Live neuron map

Hosted playback demo · no GPUs are running here. The demo uses clearly marked sample data. Open the Weight Atlas for the new real-model research measurements.

Loading the saved playback demo…

Choose a saved capture or send a request
Loading the Connectome (0.8 MB)…
How to read it, and what is measured
  • What fires. In each of the 40 layers the router sends every token to 6 of the 384 routed experts and to the layer’s shared expert. Each expert has 2,304 neurons: the inputs of its down projection, a = silu(min(w1·x, 10)) × clamp(w3·x, −10, 10) (the model’s SwiGLU clamp, so |a| ≤ silu(10) × 10 ≈ 99.995; the recorded values carry the bf16 rounding of the down projection’s input and reach about 100.4), taken before the routing weight. A neuron fires on a token when |a| is among the 115 largest of its expert’s 2,304 (the top 5 %; ties go to the lower neuron number, so when more than 115 neurons tie at the cut, as when they all sit at the clamp, the 115 lowest-numbered are reported). So every expert a token reaches fires exactly 115 neurons for it; what differs is which ones. “Fired” is relative to the expert, not an absolute threshold.
  • Dots and brightness. Experts sit where the Connectome puts them: layers are nested shells (layer 0 at the core), regions are the task groups that used them most in the task map. For the chosen phase an expert’s lit dots are the share of its 2,304 neurons that fired on at least one of the phase’s tokens (1 dot ≈ 48 neurons; on small screens 16 dots per expert, so 1 dot ≈ 144), and the lit dots are brighter and whiter the larger the share of the phase’s tokens routed to it (full at 50 %).
  • Lines are the request’s real token paths. Amber lines join the experts the same token was routed to in neighbouring layers, counted over the phase’s tokens (the 6,000 most frequent pairs are drawn; when many pairs are used equally often, as in a short prompt where most are used once, an even sample of them is drawn). Choosing a token draws its own path in white: its 6 experts in every layer, joined layer to layer, brighter for larger routing weights. The blue task-map links (off by default) are the Connectome’s co-use links, not paths.
  • While it runs. About every second the brain shows the router’s picks so far, on the same scale as the result: the prompt’s once it is read, then the answer’s, with each lit expert showing the dots of the 115 neurons it fires for one token (a lower bound); the newest tokens’ paths are drawn one after another, the current one in white, the last 24 as a fading amber trail. Which neurons fired, the token list and the tables come with the finished answer; an earlier result steps aside meanwhile (it comes back if the request fails), and a clicked expert shows the tokens routed to it so far.
  • Phases. Prompt is every token fed in before the answer, chat-template markers included; Answer is every generated token. A final end token is never fed through the model, so it has no path and is not shown; when the answer is cut at the token limit, one extra forward gives its last token a path too. Both counts each neuron once if it fired in either phase.
  • Stack perspective. Neuron Stack uses the same request, token, phase and playback as the Connectome. Layers are horizontal expert grids, with the shared expert to the right. Layer spacing and the layer/expert controls reveal individual experts. Neurons shows all 2,304 index positions, highlighting only identities stored in the capture’s ranked lists and the selected token’s strongest neuron. Grey positions do not say whether a neuron fired. Completed token paths land on the recorded strongest neuron when it is available; live routes land at expert centres. These lines are observed routing transitions, not measured neuron-to-neuron synapses.
  • Not drawn. Attention is not drawn; shared experts appear in the Stack, the table and the expert panel. The neuron table lists the 300 neurons that fired on the most tokens of the chosen phase (each phase has its own list; older results carry only the whole request’s list).
  • Cost and safety. Starting books the session in the campaign C2 ledger and runs the model on 4 × RTX PRO 6000 on Modal at about 16.33 USD per hour; the ledger books GPU time from the first container start in 10-minute steps. It turns itself off after the idle limit and at the session limit, whether or not this page is open; the workspace that books and settles it runs only for that, and only this Mac stops it, so keep the Mac awake with its lid open while it books and while it turns off. Before it starts the model for a request that might be too long, the page counts the request’s tokens on this computer: the model takes 4,096 tokens, request and answer together, and would refuse a longer request only once it had started. Sample mode (?mock=1) starts nothing and spends nothing.
  • The page and the capture were made by AI agents and have not been reviewed by a person.