# When Neural Networks Pack More Ideas Than They Have Room For
## The Geometry of Compression Inside Artificial Minds
There’s a quiet revelation in modern neural network research that has reshaped how scientists think about what happens inside these systems. The discovery challenges something we might intuitively assume: that a network with a hundred neurons can only hold about a hundred distinct pieces of knowledge. In practice, these systems routinely hold far more than their architecture would seem to allow, and the reason has to do with geometry, sparsity, and a clever strategy networks discover on their own.
### Why This Question Matters
When researchers try to open up a trained neural network and understand what individual neurons actually represent, they run into a puzzle. The expectation might be clean one-to-one mappings — one neuron for “this is a dog,” another for “this is the color red.” What they actually find is messier. Individual neurons respond to multiple unrelated concepts simultaneously. A single unit might fire for both feline faces and the front grille of an automobile. This phenomenon, known as polysemanticity, makes interpreting what a network has learned extraordinarily difficult.
The central question becomes whether this mixing is a failure — evidence that the network hasn’t learned clean representations — or something more intentional. The answer, as it turns out, is neither accidental nor fixable by simply adding more neurons.
### The Core Insight: More Concepts Than Dimensions
Neural networks process information through layers of numbers. Each layer has a fixed number of dimensions — essentially a fixed number of slots where information can live. Intuitively, you’d expect that you can only represent as many distinct ideas as you have slots available.
But consider this: most of the time, most concepts are silent. In any given piece of text, the concept of “spaceship” might be entirely absent while “the” and “past tense” are present. This natural sparsity creates an opportunity. If two concepts rarely turn on at the same time, a network could theoretically let them share the same slot — or direction in mathematical space — because the interference between them would only matter in the vanishingly rare moments when both concepts appear simultaneously.
This is the fundamental idea behind superposition. Networks represent more features than they have dimensions by packing them at angles to each other, accepting a small amount of cross-talk as a tradeoff for tremendous representational efficiency.
### Building the Simplest Possible Experiment
To truly understand this phenomenon, there’s value in stripping everything away until you’re left with the bare essentials. The experiment involves a very simple setup: generate synthetic numerical features, compress them through a smaller bottleneck, and then reconstruct them.
Imagine you have twenty independent numerical values, each between zero and one. Most of the time — say, ninety percent — each value is exactly zero. Occasionally, a value turns on and takes on a random magnitude. These represent the sparse features a network might encounter.
The network takes these twenty values and compresses them down into just five dimensions — four fewer slots than it started with — and then attempts to reconstruct all twenty original values from those five compressed representations. The network has strictly fewer dimensions on the inside than it has features on the outside, yet it’s asked to preserve information about all twenty.
The training process minimizes a weighted error, telling the network that some features are more important to get right than others. Feature zero might be critical, while feature nineteen barely matters at all. Through standard gradient-based learning, the network adjusts its internal parameters to reduce this error as much as possible.
### What Happens When You Watch Closely
The first surprising result appears in how the network treats unimportant features. With twenty features compressed into five dimensions and a steep gradient of importance across those features, the network makes a decisive choice. It reconstructs the most valuable features with high accuracy and then, past a certain threshold, simply stops trying. The reconstruction error for the least important features doesn’t hover at some mediocre compromise — it essentially goes to zero, meaning the network has deliberately abandoned them entirely.
Nobody programmed this cutoff. It emerged naturally from the optimization process. The network found that spending capacity on low-importance features was wasteful when that capacity could be better used elsewhere.
### The Pentagon Phenomenon
The most striking result emerges when you remove importance weighting entirely and force five equally significant features through a two-dimensional bottleneck. With just five features competing for two dimensions, what geometry does the network discover?
It arranges all five feature directions as vertices of a perfect regular pentagon in the two-dimensional space. Each feature gets its own arrow pointing in a unique direction, separated from every other feature by roughly seventy-two degrees.
This arrangement has a specific mathematical property: it maximizes the minimum angle between any pair of features. In other words, it’s the configuration that minimizes the worst-case interference between any two features sharing that two-dimensional space.
There is no penalty in the loss function for symmetry. No term rewards the network for placing these directions evenly around a circle. The pentagon emerges because gradient descent, driven purely by reconstruction error, discovers that this is the optimal geometric packing. Five directions in a two-dimensional plane, each as far from every other direction as possible, is the natural solution when you must represent five things in only two dimensions.
At lower levels of sparsity — where features are active more often and therefore more likely to overlap — the network gives up on most features and only represents two or three. At higher sparsity, where features are almost never active simultaneously, all five find room, arranged in the near-perfect pentagonal geometry described above.
### Visualizing Interference Patterns
When the scale increases to forty features compressed into five dimensions, the interference patterns become visible through the network’s internal weight matrix. Computing the product of the weight matrix with its transpose reveals a structure: a dense block in one corner where important features live with strong self-reconstruction and visible cross-talk between them, and a vast empty region beyond that where features have been completely abandoned.
The boundary between these two regions aligns remarkably well with what emerges from simpler experiments, suggesting a consistent mechanism at work regardless of the specific numbers involved.
### The Phase Transition of Sparsity
Perhaps the most conceptually powerful result comes from systematically varying how often features are active while keeping the network architecture fixed at thirty features compressed into five hidden dimensions.
At zero sparsity — where features are always active and constantly interfering with each other — the network represents exactly five of thirty features, ignoring the rest entirely. Without sparsity, superposition costs more in reconstruction error than it saves.
As sparsity increases past approximately eighty percent, something dramatic happens. The network begins cramming in additional features at an accelerating rate, eventually representing twenty-four of thirty features — nearly five times its nominal dimensional capacity.
The transition is not gradual. There is a sharp regime change rather than a smooth ramp. Below a certain sparsity threshold, superposition is not worth the interference penalty. Above it, the network commits fully to the strategy of packing features at angles, exploiting the fact that most features are simultaneously inactive.
### What This Means for Understanding Real Networks
These experiments use synthetic data and tiny architectures — nothing that would require specialized hardware or months of training. Yet they reveal a phenomenon that researchers believe is central to understanding large language models, which almost certainly represent thousands of distinct concepts inside far fewer neurons than that count.
The practical implication is significant. If polysemanticity — the mixing of multiple concepts within single neurons — is not a flaw or a training deficiency but rather an inevitable consequence of representing many features in limited space, then interpretability research cannot simply ask individual neurons what they represent. The mixing is structural, built into the compression strategy itself.
This explains why specialized tools like sparse autoencoders have been developed. These techniques aim to reverse the packing process, pulling apart the superposed features and recovering individual concepts from the entangled representations. Understanding superposition helps explain why such tools are necessary in the first place.
Having watched this phenomenon unfold in a system small enough to fully examine and visualize, the reason for these research directions becomes clear in a way that description alone cannot convey.
—
## Frequently Asked Questions
**What is superposition in the context of neural networks?**
Superposition is the strategy neural networks use to represent more distinct features than they have dimensions available. By packing feature representations at acute angles to each other in activation space, networks can encode additional information at the cost of occasional interference when multiple features happen to be active simultaneously.
**Why does sparsity matter so much for this phenomenon?**
Sparsity — the property that most features are inactive at any given time — is the economic condition that makes superposition viable. If features rarely overlap in their active periods, the interference cost of shared dimensions becomes negligible. Dense activation would make superposition counterproductive because interference would dominate reconstruction accuracy.
**Is the pentagon arrangement specific to five features in two dimensions?**
The regular polygon pattern generalizes. The network discovers whatever symmetric arrangement minimizes maximum pairwise interference for the given feature count and dimension count. In each case, features spread out as evenly as possible across the available dimensional space.
**Can superposition be undone or reversed?**
Yes, and this is an active area of research. Techniques like sparse autoencoders are specifically designed to decompose superposed representations and extract individual features. These methods work by learning to identify the geometric structure of superposition and invert it.
**Does this mean neural networks are fundamentally lossy compressors?**
In a sense, yes. The tradeoff is real: representing more features than dimensions allows comes at the cost of interference between simultaneously active features. The network manages this by relying on the statistical rarity of multiple features co-occurring, but the compression is inherently lossy to some degree.
**What do these toy experiments tell us about large language models?**
They provide a mechanistic explanation for polysemanticity observed in real networks. Rather than being a mysterious failure of training, multi-concept neurons may be an inevitable geometric consequence of compression under sparsity — a strategy networks adopt when they have more concepts to encode than parameters to encode them with.
**Why were hand-derived gradients used instead of automatic differentiation?**
Deriving gradients manually forces a deeper understanding of what each computational step actually does mathematically. For small models, this exercise provides insight that would be obscured by simply calling a built-in differentiation function.
—
## Conclusion
The phenomenon of superposition reveals that neural networks are far more mathematically sophisticated in their representations than a surface-level view might suggest. When faced with a bottleneck — fewer internal dimensions than external features to represent — networks do not simply fail or truncate. Instead, they discover elegant geometric solutions that maximize representational capacity under interference constraints.
These experiments demonstrate that the geometry of neural representations is not arbitrary. Antipodal pairs, regular polygons, and sharp phase transitions in representational strategy all emerge from the same basic principle: minimize reconstruction error given limited capacity. The network’s solutions are genuinely optimal packing arrangements, not approximations or compromises.
For the broader field of interpretability, superposition provides both an explanation and a direction. It explains why individual neurons in trained networks respond to multiple unrelated concepts, and it points toward the tools needed to undo that mixing. The path forward for understanding large artificial intelligence systems runs through understanding these geometric compression strategies at their most fundamental.
What began as a curiosity — can a network really fit five things into two dimensions? — turns out to be a window into the representational principles governing systems of all sizes.
Thank you for reading
