INTERVENTION LAB / PARAMETER RESEARCH
Inside every connection.
Explore the complete model, then follow measured token activity down to an individual weight.
763.21Blearned parameters
35.48Mbackbone expert neurons
15,400backbone experts
40backbone layers
40 backbone layers · every expertClick a layer to enter
Structure · select a measured token belowSelectionDrag to orbit · scroll to zoom
REAL GPU MEASUREMENTS
A word takes a path.
Saved research measurements are unavailable. The structural weight explorer remains available.
THE CONNECTION / STORED VALUE + TOKEN INPUT
Read the actual weight.
layers.24.ffn.experts.34.w3.weight
Reading a small window from the pinned checkpoint…
Each cell is one actual learned coefficient. Arrow keys move between cells. FP4 nibbles and their block scales are decoded separately.
COMPLETE STRUCTURAL COVERAGE
Every module. Every coordinate.
Includes attention, routers, embeddings, Engram memory, vision, hyper-connections and MTP. Quantization scales are indexed separately from learned parameters.
Loading catalog…
FOLLOW-UP / PAIRED WEIGHT USAGE
Different prompts. Different connections.
The second analysis compares 3D prompts with matched code controls, counting unrouted positions as zero. It selects two connections per layer from discovery examples, then checks their usage on eight reserved prompt pairs.
A repeated usage pattern
The pooled contrast is 0.72397 normalized units; its paired bootstrap interval is 0.39631 to 1.0960. The exact paired sign-flip p-value is 0.0078125.
0 individual candidates pass correction across all 80 tests. Connections that share inputs are correlated; the eight prompt pairs are the statistical units.
Causal evidence stays limited
The earlier eight-site ablations give a 3D-specific excess loss change of 1.287e-5, with interval -2.882e-5 to 5.918e-5 and paired p = 0.594.
This does not establish a 3D-specific causal role. The 80 new usage candidates have not received new ablation tests.
Inspect the 80 exploratory connections
Click a connection to open its original coefficient and a saved 3D token where its input was measured. Δ compares mean absolute local contributions on reserved 3D and control prompts.
| Connection | Weight | Reserved Δ | Paired 95% interval | Adjusted p |
|---|---|---|---|---|
| -0.046875 | 1.530e-5 | 5.633e-7 to 3.171e-5 | 1.0000 | |
| -0.046875 | -7.238e-6 | -4.116e-5 to 1.896e-5 | 1.0000 | |
| 0.046875 | -6.155e-5 | -1.636e-4 to 3.670e-5 | 1.0000 | |
| 0.046875 | 5.769e-4 | 1.973e-4 to 0.0010241 | 1.0000 | |
| 0.062500 | 0.0010547 | -5.448e-5 to 0.0026170 | 1.0000 | |
| 0.031250 | 3.413e-4 | 2.126e-4 to 4.926e-4 | 0.62500 | |
| -0.062500 | 4.449e-4 | -1.222e-4 to 0.0011128 | 1.0000 | |
| -0.093750 | -2.532e-4 | -5.844e-4 to 1.174e-4 | 1.0000 | |
| 0.062500 | 0.0014482 | 8.389e-4 to 0.0021230 | 0.62500 | |
| 0.046875 | 3.021e-4 | 3.022e-5 to 6.115e-4 | 1.0000 | |
| -0.062500 | 0.0010610 | 3.990e-4 to 0.0020883 | 0.62500 | |
| 0.062500 | 2.665e-4 | 2.351e-5 to 6.400e-4 | 0.89063 | |
| -0.031250 | 0.0025297 | 5.634e-4 to 0.0049471 | 0.62500 | |
| 0.031250 | 3.646e-4 | 4.887e-5 to 8.631e-4 | 0.89063 | |
| -0.062500 | 0.0011520 | 4.722e-4 to 0.0018909 | 0.62500 | |
| 0.062500 | 0.0035060 | 4.953e-4 to 0.0075511 | 1.0000 | |
| 0.046875 | 0.0018669 | 8.857e-4 to 0.0031182 | 0.62500 | |
| -0.046875 | 5.778e-4 | 1.380e-5 to 0.0016516 | 1.0000 | |
| -0.093750 | 0.010751 | 0.0030281 to 0.020417 | 0.62500 | |
| 0.031250 | 2.934e-5 | -5.335e-5 to 1.374e-4 | 1.0000 | |
| 0.046875 | 3.051e-4 | 1.816e-4 to 4.250e-4 | 0.62500 | |
| -0.046875 | 0.0015437 | 7.725e-5 to 0.0038537 | 0.89063 | |
| 0.046875 | 4.023e-4 | 2.641e-4 to 5.684e-4 | 0.62500 | |
| 0.031250 | 0.0011558 | 5.396e-5 to 0.0027666 | 1.0000 | |
| -0.031250 | 0.0023456 | 0.0016293 to 0.0030453 | 0.62500 | |
| 0.062500 | 3.019e-4 | 1.308e-5 to 6.798e-4 | 1.0000 | |
| 0.023438 | 4.157e-4 | 6.028e-5 to 8.847e-4 | 1.0000 | |
| -0.062500 | 5.012e-4 | 8.442e-5 to 0.0010673 | 0.89063 | |
| -0.093750 | 0.0038257 | 0.0019122 to 0.0063241 | 0.89063 | |
| -0.062500 | 0.0012834 | 3.493e-4 to 0.0023819 | 0.89063 | |
| 0.031250 | 0.0026316 | -2.000e-4 to 0.0080886 | 1.0000 | |
| -0.046875 | 0.0021374 | -0.0030956 to 0.0081627 | 1.0000 | |
| -0.062500 | 0.0010063 | 6.279e-4 to 0.0014376 | 0.62500 | |
| 0.062500 | 0.0039050 | 9.460e-4 to 0.0078099 | 1.0000 | |
| 0.046875 | 0.0022721 | 1.493e-4 to 0.0050807 | 0.62500 | |
| -0.031250 | 0.0021869 | 5.698e-4 to 0.0041020 | 1.0000 | |
| -0.062500 | 9.153e-4 | -1.246e-5 to 0.0026814 | 1.0000 | |
| 0.046875 | 9.146e-4 | 7.437e-5 to 0.0021055 | 1.0000 | |
| 0.062500 | 1.170e-4 | 2.620e-5 to 2.141e-4 | 1.0000 | |
| -0.015625 | -0.0016136 | -0.0055465 to 0.0015641 | 1.0000 | |
| 0.023438 | 0.0068551 | 0.0010386 to 0.015218 | 1.0000 | |
| -0.062500 | 2.203e-4 | -5.333e-4 to 9.099e-4 | 1.0000 | |
| 0.046875 | 0.0077986 | 0.0035954 to 0.012322 | 0.62500 | |
| -0.031250 | 0.0018902 | 0.0010110 to 0.0028844 | 0.62500 | |
| 0.031250 | 0.0048359 | 0.0013926 to 0.0090408 | 0.62500 | |
| -0.046875 | 9.742e-4 | -3.337e-4 to 0.0023739 | 1.0000 | |
| 0.046875 | 0.0083971 | 0.0021529 to 0.016097 | 0.62500 | |
| 0.062500 | 0.0011141 | 5.029e-4 to 0.0021412 | 0.62500 | |
| 0.023438 | 0.025695 | 0.0054634 to 0.052746 | 0.62500 | |
| 0.046875 | 0.0017171 | 5.476e-6 to 0.0039857 | 1.0000 | |
| -0.023438 | 2.570e-4 | 7.377e-5 to 4.198e-4 | 1.0000 | |
| 0.046875 | 0.0010762 | 1.193e-4 to 0.0022840 | 1.0000 | |
| -0.093750 | 0.0019661 | 4.849e-4 to 0.0036207 | 1.0000 | |
| 0.062500 | 0.0015656 | -3.446e-4 to 0.0036599 | 1.0000 | |
| -0.046875 | 0.0065150 | 0.0024761 to 0.011243 | 0.89063 | |
| 0.093750 | 0.0025308 | 4.560e-4 to 0.0050132 | 1.0000 | |
| -0.062500 | 0.027975 | 0.0040046 to 0.061266 | 1.0000 | |
| 0.062500 | 0.0043392 | 4.203e-4 to 0.010845 | 1.0000 | |
| -0.062500 | 0.0058704 | 2.679e-5 to 0.013518 | 1.0000 | |
| 0.046875 | 0.0020543 | 8.091e-4 to 0.0041604 | 0.62500 | |
| 0.046875 | 0.0034091 | -8.656e-4 to 0.0086467 | 1.0000 | |
| -0.062500 | 0.0045860 | 8.250e-4 to 0.011462 | 0.62500 | |
| 0.062500 | 0.0034284 | 0.0015548 to 0.0058281 | 0.62500 | |
| 0.062500 | 0.0041426 | 0.0024073 to 0.0060029 | 0.62500 | |
| -0.062500 | 7.801e-4 | 9.871e-5 to 0.0016200 | 1.0000 | |
| -0.031250 | 5.260e-5 | -9.372e-5 to 2.755e-4 | 1.0000 | |
| 0.046875 | 0.0020709 | 8.657e-4 to 0.0034883 | 0.89063 | |
| -0.062500 | 0.0014006 | -0.0055822 to 0.0064099 | 1.0000 | |
| -0.062500 | -2.788e-4 | -0.0027849 to 0.0015781 | 1.0000 | |
| 0.062500 | 0.0022202 | 8.929e-4 to 0.0038400 | 0.89063 | |
| -0.062500 | 0.0043821 | 3.627e-5 to 0.0097679 | 1.0000 | |
| 0.046875 | 5.269e-4 | -7.085e-4 to 0.0022254 | 1.0000 | |
| -0.062500 | 0.011264 | 0.0071962 to 0.016145 | 0.62500 | |
| 0.046875 | 0.0030423 | -2.931e-4 to 0.0081913 | 1.0000 | |
| 0.062500 | 0.013713 | -7.153e-7 to 0.035020 | 1.0000 | |
| -0.062500 | 0.0062101 | 0.0018048 to 0.010845 | 1.0000 | |
| -0.25000 | -0.13848 | -0.52120 to 0.20615 | 1.0000 | |
| 0.062500 | 0.037959 | -0.026257 to 0.099912 | 1.0000 | |
| 0.75000 | 1.2656 | 0.42188 to 2.5313 | 1.0000 | |
| 0.37500 | -0.53041 | -2.0035 to 0.62292 | 1.0000 |
Coverage: 45,458,444 coefficient coordinates have an observed nonzero term. Cached weights cover two of 5,120 W2 output rows per measured expert. This is exploratory reuse of the original reserved prompts and teacher-forced prefill captures; prompt style and routing can explain differences. It is not a capability benchmark or a semantic map of every parameter. All 80 selected values and addresses were independently rechecked against the pinned checkpoint. This follow-up used CPU analysis, with no new GPU inference.
EVIDENCE / LIMITS
Values are facts.
Functions need experiments.
Every weight has a structural address. Local input × weight terms explain a linear operation. They do not, by themselves, establish what knowledge a weight represents or why the model chose its final word.
The size cards count expert neurons and experts in the 40-layer backbone. MTP adds 387 experts and 891,648 expert neurons. Vision and MTP weights are structurally covered; this text-only research does not exercise them. Task associations from the earlier map are observational, and the new probe results have the scope stated above.