What's worth reading off a brain
A conversation with Adam Marblestone
We can read every weight in a large language model and still can’t really say what the model is doing. Why then would tracing every connection in a brain, a far harder problem, tell us anything we couldn’t get more cheaply somewhere else? It sounds like a reason not to bother mapping brains at all.
Adam Marblestone thinks it’s the opposite. Adam Marblestone runs Convergent Research, a nonprofit that incubates what it calls Focused Research Organizations (FROs), mid-scale science projects that are too capital-intensive and engineering-heavy for an academic lab but too far from a product at the start for venture capital.
“When many AI people or computer scientists hear ‘map the connectome,’” he told me, “I think they hear ‘we’re going to map the weight matrix. We’re going to know all these weights.’” That, he thinks, is the wrong way to picture it because the weights are not where the interesting information is. “The weights are just as much a function of what you trained it on as it is anything about the architecture of that system. So I actually don’t really care about the specific weights. What I care about is the architecture.”
Outside the trained weights
To understand why Adam doesn’t care about the weights, it helps to look at how an AI system actually gets built. When you train a neural network, the decisions about the architecture, the goal it’s optimizing for, the data it learns from all live in the code the researcher writes.
“I have some PyTorch code that sets up the architecture of the network. I write in code what the loss function is. I feed it data. And then in the end I end up with a weight matrix that lives inside this box.” As he puts it, “a bunch of the interesting stuff about what the researchers actually did isn’t in the weight matrix.”
But the brain doesn’t work that way. There’s no separate code, it has to build everything out of neurons: “The learning signals are neurons. The basic architecture is how the neurons are initialized and how they’re connected. The cost functions are specific neurons that deliver whatever that reward signal is. Everything has to do with neurons. It doesn’t have a separate programmer.”
In an AI system, the design sits in the code itself, outside the trained weights. In a brain there’s no such thing as “outside the trained weights,” so all of the design has to be built physically into the cells and their connections.
Adam develops this further in his paper, The Sweet Lesson of Neuroscience. In it, he references Steve Byrnes’ work, which recasts the entire brain as two interacting systems: a learning subsystem and a steering subsystem. The first learns from experience during the animal’s lifetime. The second is mostly hardwired and sets the goals, priorities, and reward signals that shape that learning. The learning subsystem is what learns. The steering subsystem is what decides what’s worth learning. For more on this, here is another good overview on the two subsystems.
The steering subsystem’s circuits aren’t a record of anything learned. They are the design, hardwired by evolution, with the reward signals written directly into the cell types and their wiring. That is why Adam cares about reading out the steering subsystem and not the learned weights of the cortex. The cortex’s wiring is mostly a snapshot of values it acquired through experience. The steering subsystem’s wiring is the genome’s specification itself. Patrick Mineault wrote a good piece that dives deeper into Adam’s thinking on this.
Fei Chen and Evan Macosko, the PIs from the Broad Institute who published one of the original whole-mouse-brain transcriptomes, find evidence of this in their work, Cellular Scaling Laws in the Mammalian Brain. The largest number of distinct, bespoke neuron types sits in the old deep structures, the hypothalamus and brainstem. These regions have far fewer neurons overall but a much wider variety of them. The cortex is the reverse. It is large, but built from many copies of a few repeating templates.
This fits what the framework predicts. The cortex works like a neural network, so it can learn almost anything and doesn’t need specialized hardware. It just needs many copies of the same flexible units. The steering subsystem has the opposite job. It encodes specific, hardwired goals: hunger, thirst, fear, the drive to mate, the urge to breathe. Each is something the genome has to specify in advance, because an animal can’t afford to learn them by trial and error. So if the steering subsystem really is the brain’s hardwired reward function, the bespoke cell-type diversity should pile up there too, in those deep structures rather than the cortex. And that is what the research shows.
A thread of work that Adam began a decade ago along with Ed Boyden at MIT helped start the use of expansion microscopy and in-situ sequencing to read out neural wiring. Adam later helped popularize FROs as a way to fund this kind of infrastructure-heavy science. The mapping itself is now being pushed by E11 Bio, the first FRO spun out of his Convergent Research, which is trying to drop the cost of a mouse-brain connectome from billions of dollars to low tens of millions. The hope is that mapping at high enough resolution would expose the brain’s design: not the trained values, but what specifically produced them. “Once we’ve done that, I don’t actually care that much about the specific weights,” he said.
Mapping the reward functions
A detailed enough map, the thinking goes, would expose the machinery that computes internal rewards. “It might be that the infant is trying to first establish eye contact, or it’s trying to find and pay attention to novel stimuli or something like that versus boring stimuli,” Adam said. “Am I making eye contact with the parent? Am I finding novelty? Am I controlling my environment? These are all things that the brain probably has to have some way of detecting and rewarding.”
On the Dwarkesh Patel podcast, Adam argued that the field has tended to neglect the role of these very specific reward functions. Machine learning gravitates toward mathematically simple objectives, like predicting the next token. His hunch is that evolution did the opposite, building a lot of complexity into the brain’s reward functions: many different ones for different regions, switched on at different stages of development. If he’s right, those reward functions in the brain would each be a group of cells you could in principle point to.
A detailed map is the starting point, but what Adam really wants isn’t the frozen snapshot so much as the process that generates it: “I want to understand the drivers... how it starts out and then how it learns within the lifetime.” He acknowledges that the translation from a static map to a complete description of “what is it trying to do” will be hard. “Will you be able to actually translate between a map and that information? Maybe,” he said. “But if not, I still think that having the maps is going to be a multiplier on the rate of overall neuroscience progress.” So even in the pessimistic case, where the wiring doesn’t hand you the algorithm, the map still accelerates everything else.
Why the mapping is so hard
In October 2024, the FlyWire consortium, which includes Greg Jefferis’ group and collaborators at Cambridge University, Princeton and the University of Vermont, published the first complete wiring diagram of an adult fruit fly brain: roughly 140,000 neurons and more than 54 million synapses. Producing it required slicing a single fly brain into thousands of ultrathin sections, imaging each with electron microscopes, and using machine learning to stitch the images back into a 3D reconstruction. A fruit fly has on the order of 100,000 neurons. A human has something closer to 100 billion.
To address this, E11 Bio is making comprehensive static circuit maps more scalable. The core bet is a shift in imaging physics: “The traditional way of doing static circuit mapping all the way down to the neuron and synapse level is the electron microscopes, which are extremely precise in what they can see spatially, but they are not easily scalable... If you can switch that to using a light-based microscope rather than an electron-based microscope, it’s much easier to have thick pieces of tissue that you can sort of see through and are much easier to handle.”
Light-based methods bring a second advantage: they can read out molecules. “It also gives you the advantage of being able to see molecules kind of overlaid — so what are the specific receptors and transmitters that are used by the synapses?” This is important because the brain’s connections are not uniform in the way an artificial network’s are.
“Unlike in a computer, where there’s maybe a few types of connections, there’s actually many different types of connections in an actual brain,” Marblestone said. Excitatory or inhibitory, but also varying in their time scales and in how they adapt and learn. A wiring diagram that only records which neuron connects to which, with no read-out of the receptors and transmitters at each synapse, would miss most of what tells those connection types apart. And thus, much of what distinguishes one learning rule from another.
This is the focus of a second FRO Adam is helping to catalyze, Meridial, which pushes from static snapshots toward dynamic maps. Meridial is not expected to be as comprehensive as static mapping, but the goal is to observe a subset of connections changing over time and thus to understand the rules for how synapses change. “You won’t get every connection. But even if you just look at a subset of connections, being able to understand the rules for how they change, I think that’s super AI relevant,” he said.
Training models from brain data
If the wiring encodes the design, then there could be a path whereby you could train AI systems directly on brain activity. This would mean a model learns to represent the world the way a brain does rather than only the way labeled data does. “There are some companies starting to do brain-data-based training,” Adam told me.
The question, he says, is whether brain data tells a model anything it couldn’t already figure out on its own. “What’s the delta? What’s the difference between having that information and having just the information about the world that we train on now? Are there things in that neural activity that we can’t already predict from the data that it’s seeing?” If a model can already infer how a brain would respond to an image just from the image, the recording adds nothing. Brain data is only worth collecting if it carries something we can’t get from other types of data.
A line of work on representational alignment has shown measurable gains from nudging artificial networks toward neural data. Aligning vision models to human EEG can make their representations more brain-like and more robust. Fine-tuning speech models on fMRI recordings of people listening to stories, a method Mariya Toneva and colleagues call brain-tuning, improves their downstream performance, with the largest gains on tasks that require semantic understanding. And a 2025 study found that aligning auditory models to individual fMRI recordings improved performance on downstream tasks, especially where training data was scarce.
There’s signal that brain data contains something AI can’t already extract from the world, but there are still a lot of open questions. As Adam put it, “It’s one of these things that needs to be tried.”
Why this paradigm got skipped
If reading every weight in a language model can’t explain it, why expect a brain map to do better? The objection assumes you’d be staring at its weights. But Adam believes you’d be staring at something an LLM never had: the design itself.
Adam was on the neuroscience team at Google DeepMind from roughly 2018 to 2020, when the field’s brain-inspired instincts were near their peak. “What I was working on was memory architectures, we were focused on questions like ‘what does the hippocampus do as a memory system?’ ‘Does it have some way of compressing information?’” The leading research programs leaned on reinforcement learning, training systems through reward and trial-and-error in an elaborate, brain-inspired form, full of what Adam calls “bells and whistles.”
Then LLMs took off and bypassed most of it. LLMs use a stripped-down form of reinforcement learning: reward the good outputs, adjust, repeat. They have no internal model of the world. Whereas the brain is thought to do something richer. “The way that large language models do reinforcement learning and post-training is in some ways kind of like the simplest or most brute-force way to do RL,” Adam said, “whereas people believe that the brain does model-based RL,” which is focused on building a model of how the world works and planning against it.
Model-based RL gives the brain a set of faculties that LLMs don’t have built in. “The brain has more innately built systems,” Adam said, “to predict what’s going to happen in the future, or simulate different possible events. Or to go back to different memories to use that memory to make a prediction.” The brain also has ways of working out which actions deserve credit for a reward that only comes later, the problem of temporal credit assignment, as well as value functions that estimate how good a situation is likely to turn out. “These are things that are pretty clearly built into the mammalian brain,” he said, “that LLMs don’t have a built-in architectural solution for.”
With today’s LLMs, “you’re not really building in things like a hippocampus or prefrontal cortex or a striatum or some of the brain areas that we know we have,” he noted. These systems may approximate some of those functions as emergent byproducts of training, but they don’t have them as architectural commitments.
The brain does, and that is the argument for going to look at it. Every faculty the brain has, it had to build into the structure itself, where it can in principle be found and read. Adam is not alone in believing that intelligence needs a kind of built-in cognitive structure, rather than expecting it to emerge from scale alone. Emmanuel Dupoux, Yann LeCun, and Jitendra Malik argue that today's models are missing an architecture inspired by human and animal cognition. They analyze how autonomous learning works in living organisms and propose a roadmap for reproducing it in artificial systems.
As Adam put it, “Having the maps is going to be this multiplier on the rate of overall neuroscience progress. It will help us understand the truth of how humans do it.” That truth isn’t in the weights. The learned weights of the brain’s learning subsystem are tuned over a single lifetime, particular to one brain. The thing worth reading is the wiring: the architecture and the reward circuitry built into the structure itself.
Author’s note: An LLM was used for light copy editing only (spelling, grammar, and clarity). Content, meaning, tone, and structure remain unchanged.
 | A guest post by
|
The road to human-level critical thinking in a machine requires a neurosymbolic approach -- cognitive science lights the path.