A map of every neuron in Qwen3-8B that we have tied to a job. Pick a concept and watch its neurons light up, layer by layer, as the model works out the answer.
James Phoenix & Claude Opus 5.5 · 18 October 2026 · work in progress
In Eiffel Tower Qwen we showed that a handful of neurons carry each concept the model knows: switch them off and it can't name the concept; turn them up and it talks about it.
Since then we have run the same test on over 5,000 concepts, which we've connected to about 44,000 of the model's 442,368 neurons. This page maps those neurons to concepts.
The code will follow on GitHub.
How it works:
Our first and main approach is the following process:
Ask. For a given concept, Qwen answers about ten questions about it, such as questions whose answer is the concept's name (“What drink is brewed in a French press?” → Coffee). We keep only those questions Qwen answers correctly.
Observe. For every neuron we measure how strongly it fires and how sensitive the answer is to it. Together these estimate how much Qwen's likelihood of replying with the concept as its answer (its “concept-specific certainty”) would drop if that neuron were switched off. For about 1,400 neurons so far, we also record where in ordinary web text they fire most strongly, for contrast.
Ablate. We switch off the 200 neurons with the biggest estimated drop, one at a time, and keep the 30 that reduce the concept-specific certainty the most. As a control, we also switch off random neurons in the same layers to confirm they do not meaningfully reduce concept-specific certainty.
Label. We name neurons for convenience when we are confident in our reading of them, based on the concepts that need them, the words they push the model towards and the web text they fire most on.
This approach finds concept-linked neurons well, but tells us only a limited amount about what those neurons do, and little about neurons that respond less strongly. So we are exploring other ways of identifying neurons' “jobs”, explained further below.
All findings are work in progress, and not every process has been run for every neuron yet.
Figure 1 · Watch a concept light up
Pick a concept, a century or a fact. Its neurons light up as the question passes down through the layers. Or pick a kind of job to see every neuron we've found doing it. Hover over or tap any dot to see what that neuron does; to see one subject up close, pick it below or tap its name in the map.
Or a fact:
35
Hover over or tap a dot to see what that neuron does.
the concept's neurons (bigger = switching it off lowers Qwen's certainty more)other neurons with a checked name (–)other neurons whose job isn't clear yet
Figure 1. Each row is one of Qwen3-8B's 36 layers, first at the top. In layers 20–35 we grouped the neurons by the subject they mostly serve (from the Wikipedia section of the concepts that need them); "Shared" neurons serve several subjects. Inside the model the neurons have no such order: the grouping is ours, to make the map readable. We don't group layers 0–19, where neurons react to many things and their jobs are much less clear; there they sit in their real order. The bar on the right of each row shows how high the right answer ranks, out of the 151,936 tokens Qwen could say next, if the model had to answer at that layer: empty until the answer is in the top 1,000, full when it is first, and in colour once it is in the top 10.
Figure 2 · How the concept's neurons fit together
The 30 neurons from Figure 1, in groups. Each box holds neurons that we know do the same kind of job; the number in its corner says how many. A line from one box to another means switching off neurons in the first changes how strongly the second fires. Thicker lines are stronger links. The answer is at the top, and the bar along the bottom of each box shows how much switching off its neurons lowers Qwen's certainty in the answer. Hover over or tap a box to see its neurons. Click it once to pin it, twice to take it out of the picture, and a third time to put it back.
a group with a known jobjob not yet clear⟶ switching off the first group weakens the second⇢ dashed: it strengthens it
Figure 2. A graph of one concept, in the style of the circuit diagrams in Neural Networks Are Already Clean, built only from neurons on the map. Rows follow the layers: early layers at the bottom, then the later neurons, split by whether their weights push the answer's own word. Groups come from what we already know about each neuron, strongest evidence first: its checked name, a tested job, being shared by answers of every kind, or pushing the answer's word. Neurons with none of these go in a dashed box. Lines are measured links (point 3 under “What the map shows so far”): switching off one of the 30 changes how strongly another fires, well beyond what switching off a comparable random neuron does. We add up the links between each pair of groups and draw the strongest. Lines into the answer are as thick as the drop in Qwen's certainty when each neuron in the group is switched off, added up.
What a neuron's "job" means here
We use three kinds of evidence. A neuron's job is clearest when they agree; for many neurons we only have one or two so far.
What breaks without it. Switch the neuron off and see which answers get worse. This is the test behind every dot on the map, and it is the strongest evidence we have: it shows the model needs the neuron for that concept.
What it pushes. When a neuron fires it nudges the model towards some tokens and away from others. We can read those tokens straight from the model's weights. This works well for neurons in the last third of the model, which write the answer, and poorly for earlier ones, which work through later neurons. So for earlier neurons we also show a second reading, the J-lens (from Anthropic's Jacobian lens): the average effect that a nudge at the neuron's layer has on the model's final choice of word, across many texts. Because it follows the push through all the later layers, it is far more reliable for early neurons than the weights alone. We show it only in the layers where it passed a reliability check we set in advance: layers 0–20, 31 and 32. It is still a reading of the neuron, not a label.
What makes it fire. An AI (GPT-6-Luna) reads the web text where the neuron fires most strongly and describes the pattern; a second AI (Jev) checks that description against new text. We have done this for about 1,400 neurons so far.
Neurons fire positively or negatively, and the sign matters. Each neuron produces one number for every token, and it can be positive or negative. A positive number pushes the model one way; a negative one pushes it the exact opposite way. The two directions often do quite different jobs, so we describe them separately. Which side is called "positive" is an accident of how the model's weights are written down: flip the signs inside a neuron and the model behaves identically. So negative doesn't mean "anti"; what matters is which way the neuron pushes. About half of neurons happen to do their main job when firing negatively. The neuron that the African Union depends on pushes the model towards "African", "Africa" and "非洲" (Chinese for Africa) only when it fires negatively, which is how it fires when Qwen is asked about the African Union; its positive side pushes towards nothing in particular. When you hover over a neuron, the side it takes for the chosen concept is highlighted.
Names are for convenience. Where the evidence points clearly one way, a neuron gets a short name, like "Water and water supply". A name is a label to help you read the map, not the neuron's true meaning. We only show a name if it passes a blind check: Luna names the neuron without seeing one of the concepts that needs it, and a second AI, given only the name, has to pick that hidden concept out of five. Of the names Luna was confident about, 84% passed; a name taken from a different neuron passes about one time in eight, so a few names on the map may still be wrong. So far 1,827 neurons have a name that passed. That cost about $2 of AI time.
Many neurons have no name. We know they matter for the concepts listed, because switching them off lowers Qwen's certainty in the right answer, but we can't yet say what they do in a few words with confidence. That includes most neurons in the early layers. We show them anyway: knowing a neuron matters is a finding, even before we know why.
Kinds of job we have found
Each kind comes from a different test, so a neuron can have several, and most neurons have none we can name yet. Tap one in the figure ("Or see one kind of job") to see where those neurons sit.
What the map shows so far
Most of a concept's key neurons sit near the end. About two thirds of each concept's top five neurons are in the last eight layers, where the model writes its answer. A smaller cluster sits around layers 8–10, where the model takes in what the question is about.
Neurons are shared. Almost half the neurons on the map matter for more than one concept. The "Shared" column holds the ones that serve several subjects at once.
A concept's neurons feed each other. Switch off one of a concept's earlier key neurons and the later neuron that writes its answer fires about 12% less (the typical case), against almost nothing when we switch off a random neuron that was firing just as strongly. We then measured every pair among each concept's 30 key neurons, for all 4,194 concepts: 1.7 million pairs. About one in five passes our test (333,000 links); four in five of those support the later neuron, one in five holds it back. Figure 2 shows them for the concept you pick, between groups of its neurons.
Facts are looked up in stages. Pick "Taj Mahal: which century?" in the figure. Neurons that recognise the Taj Mahal fire early, at its name; neurons that know the question is about a century sit later, and are shared with every other "which century" question; near the end come neurons for the answer itself, such as a 17th-century neuron, shared by everything built in that century.
Some neurons hold facts, not names. A neuron in layer 28 fires for 18th-century people, places and works, whether you ask for a year, a century, a decade or a period, and switching it off makes the model worse at 18th-century dates. Next to it sit neurons that track how a date is written rather than when it was: one fires for years starting "17" but not for the words "18th century", and pushes the model towards "seven". Pick a century above to see them. These come from a separate, targeted test, so we call them "tested jobs" rather than names.
A few neurons are two-ended scales. Of the 21,510 neurons we have identified as clearly affecting concept-specific certainty (switching one off lowers it substantially for at least one concept), all but 1,567 do so firing in one direction only. A few use both: firing one way pushes towards "north", the other way towards "south". We tested this two ways: whether each end's description, read from the weights, predicts which concepts use it, and whether pushing the neuron each way actually moves the model's answers towards that end (with a random neuron pushed the same amount as a control). Three pass both: two north/south neurons and a vertical/horizontal one. Two more probably do (breakfast/dinner, gentle/tough). A male/female first-names neuron moved the answers one way only. That is one job, a scale, not two: none of the 24 neurons we pushed showed two unrelated jobs at its two ends. Only gentle pushes give clean results; stronger pushes move even a random neuron's answers.
The answer appears late. For a typical concept the right answer only reaches the top 10 at layer 33 of 36, and first place a layer later. Before that, the answer can't yet be read from the model's working state, even though many of the concept's neurons have already fired.
What it doesn't show yet
One task. Every dot comes from one kind of question: "what is this called?". A neuron can matter here and do other jobs elsewhere.
A bias towards late layers. We find neurons by how much they affect the answer, which favours the layers that write it. Early neurons are under-counted.
Attention, in detail. Each layer also has 32 attention heads, which move information between the words of the question. For every concept we switched off its 50 most important heads one at a time; those are what you see when you show heads in Figure 1. Separate studies found that heads feed a concept's neurons directly (49 of 53 routes we tested), and that a few heads shared across concepts carry the concept from earlier words to the answer: one head in layer 34 does this for 173 of 312 concepts we checked, and restoring just the top three heads brings back 56% of what the earlier words contribute, against none for three random heads. But we have mapped heads far less thoroughly than neurons.
Jobs for every dot. Many dots so far only tell you which concepts need them. Neurons that matter for just one concept can't be named yet, because the blind check needs a second concept to hide.