Episode #187 – The Shannon Trap
In 1948, Claude Shannon published A Mathematical Theory of Communication and solved, in a single paper, the problem of how to transmit a signal reliably through a noisy channel. He did it by deliberately excluding meaning from his formal definition of information,a move that was correct for the engineering problem at hand and costly once the rest of the twentieth century adopted his measure as the definition of information itself. This episode is about the inversion that followed, and the proposed repair: K = Ic².
Episode Summary
The way a wrong early guess corrupts a game of charades illustrates a cognitive pattern that applies far beyond party games. When one player shouts a confident early guess, Jaws,the entire team anchors to it, and every subsequent gesture the mimer produces gets filtered through that frame. Course correction becomes impossible, not because the evidence stops arriving, but because the evidence is being read through the wrong category. In 1948, Claude Shannon shouted the equivalent of Jaws in the domain of information theory, and the field that followed him has spent seventy-five years interpreting everything through his frame.
Shannon was working at Bell Labs on a concrete engineering problem: how to transmit a signal through a noisy communication channel without the message being garbled at the receiver. He was explicit about the move he made to solve it. The semantic content of the message, he wrote, was irrelevant to the engineering task, because the pipe carrying a phone call does not care whether the callers are arguing philosophy or hitting the keypad with their foreheads. The signal must only fit inside the pipe. What Shannon formalised under the name information was really a measure of possibility space,a count of how many different messages the source could have sent. His entropy rises as the message becomes less predictable and falls as it becomes more determined. For the telephone network, this was exactly the right measurement, and it was enough to build the digital century on.
The error did not lie in Shannon’s math. It lay in what happened next. The field he created adopted his measure as the definition of information itself, and then philosophy and cosmology picked up that definition and ran with it. The map became the territory. Consider a sandcastle on a beach and, next to it, a flat stretch of sand. The two contain the same atoms and, if one lists the three-dimensional coordinates of every grain, take the same number of bits to describe. Shannon’s measure returns the same answer for both. But one is a functional structure that holds form against wind and tide, and the other is what the beach becomes when nothing is holding it. The difference between them is not in the data. It is in the constraint.
In 1961, Rolf Landauer, working at IBM, demonstrated that erasing one bit of information carries an unavoidable thermodynamic cost,roughly 3 × 10⁻²¹ joules at room temperature, a quantity so small that the result was treated as a theoretical curiosity for half a century, until Antoine Bérut and colleagues confirmed it experimentally in Nature in 2012. The implication is straightforward. A bit is not a floating conceptual unit that sits above the hardware. It is a magnetic domain on a disk, a transistor in a memory cell, a specific physical arrangement held in place against the universe’s preference for the generic. Erase the bit, and the system slides back toward that preference while the energy that was holding the arrangement dissipates as heat. A bit, in other words, is a unit of constraint. And maintaining any constraint costs energy.
Read in this frame, the second law of thermodynamics changes register without changing content. Entropy increases in a closed system is the textbook formulation, usually interpreted as things fall apart. Translated into the constraint language: constraint dissipates unless something actively maintains it. A sandcastle pays rent to the wind and the tide. A magnetic domain pays rent to thermal noise. Cease paying, and the arrangement relaxes to the generic. Heat death, under this reading, is not a universe running out of energy but a universe running out of constraint. Shannon’s measure, seen against this backdrop, is the inverse of what the later field took it to be. It reads maximum information when the instrument is pointed at pure noise.
The repair is a modified equation: K = Ic². Knowledge equals information multiplied by constraint quality squared. Shannon’s I survives. The c² is the coefficient he deliberately excluded because his phone line did not require it. With that restoration, a new question opens. What does constraint creation look like as a physical process? The answer proposed in this episode is that one four-step mechanism runs at every scale at which the physical world produces durable structure. First, energy is committed up front, before the outcome is known. Second, the transformation is made irreversible, in the sense that reversing it would cost more energy than it released. Third, the verdict is delivered by an external reference the actor does not control. Fourth, there is no path back. Wet clay entering a kiln. Two hydrogen nuclei approaching fusion at the core of a star. A mutation carried for a full generation before selection touches it. A researcher committing a year of career and a laboratory budget to a hypothesis the physical world will evaluate. A miner expending megawatts on a hash contest whose verdict is delivered by nodes he does not control. One mechanism, running in five materials.
Under this frame, time is not a backdrop against which events occur. Time is the accumulation of irreversible commitments. An old star, in this sense, is a star that has closed a great many doors behind it. Three and a half billion years of biological time is a sequence of mutations that could not be withdrawn. A calendar year spent committing to difficult projects that reality then tested contains more time, under this framework, than a year in which nothing changed, even though the calendar disagrees. Einstein’s equations tell us how many such commitments are possible along any given path through spacetime, and they do so with the kind of experimental confirmation one finds nowhere else in physics. They do not tell us what a tick is made of. A tick, this episode proposes, is an irreversible commitment. And not every tick carries equal knowledge.
Consider a large language model under the two frames. In the Shannon frame, the model looks maximally informative: trillions of parameters, an enormous possibility space, any combination of words it is asked to produce. In the constraint frame, the same model has no self-maintained constraint of its own. It is predicting the next token from the statistical distribution of language it was trained on, and the explanatory constraint in that distribution was built by the humans who wrote the original text. The model is a compression of human constraint, not a generator of new constraint. It is the sandcastle and the beach, rendered again at trillion-parameter resolution. Bitcoin, by the same logic, is the minimal example of a new constraint generator for the economic domain: energy committed up front, blocks whose production cannot be undone, verdicts delivered by nodes no miner controls, and a record that thickens with every ten-minute interval into something more expensive to revise than to accept.
The account of creativity that falls out of the argument is tighter than the one ordinarily offered. A potter shaping a clay jug creates an interior, a bounded region of possibility,where before there was only the undifferentiated lump. The Impressionists, barred from the academic studios of late nineteenth-century Paris, chose to paint outside and inherited two constraints the studios had neutralised: changing light, and weather they could not control. From those they derived techniques, broken colour and visible brushwork, above all,that could not have been discovered indoors. Cézanne came next, then cubism a generation later, each new constraint revealing an interior of the manifold that did not previously exist. Evolution also opens phase space through random variation, yet the wing does not carry an account of why it works. The periodic table does, and the account reaches into quantum mechanics and nuclear physics,domains Mendeleev could not have imagined in 1869. Creativity, in the stricter definition, is the imposition of a novel explanatory constraint that opens inaccessible configurations and whose explanation has reach beyond its origin. Reach is the line separating a useful local pattern from genuine knowledge.
Timestamps
00:00:00 – Charades and the wrong early guess
00:03:55 – Shannon at Bell Labs and the problem he was actually solving
00:05:37 – Why he deliberately excluded meaning from the math
00:07:20 – Map and territory: when the engineering tool became the ontology
00:07:33 – The sandcastle and the flat beach
00:10:14 – Landauer and the physical cost of erasing a bit
00:12:49 – Rewriting the second law in the language of constraint
00:14:09 – A meter that reads maximum information at maximum noise
00:15:14 – K = IC² as the proposed repair
00:16:30 – The four-step mechanism: pay, commit, test, no path back
00:18:04 – Stars, DNA, minds, markets, and Bitcoin as one mechanism in five materials
00:23:05 – Time as accumulated irreversible commitment
00:25:52 – Einstein’s equations describe the ticks but not what a tick is made of
00:28:47 – Coal and uranium: knowledge opens access to phase space the past couldn’t reach
00:29:45 – AI as compression of human constraint, not a generator of new constraint
00:31:45 – Bitcoin as the correct body for machine-generated economic claims
00:33:00 – Creativity as a novel explanatory constraint
00:37:20 – Evolution versus minds: reach as the line
Timestamps are approximate.
Topics Discussed
- Claude Shannon’s 1948 information theory and what it was actually built to solve
- Why Shannon deliberately excluded meaning from the formal measure of information
- The sandcastle on the beach: two objects with identical bit-counts carrying different amounts of something Shannon can’t see
- Landauer’s Principle and the thermodynamic cost of erasing one bit
- A bit as a unit of constraint rather than as an abstract unit of information
- Entropy as the absence of constraint rather than as disorder
- K = IC² as the proposed repair to the Shannon inversion
- The four-step mechanism of irreversible commitment across clay, stars, DNA, minds, markets, and Bitcoin
- Time as a count of tested commitments, not a backdrop
- Why an LLM is a compression of human constraint rather than a generator of new constraint
- Bitcoin as the first body through which machine-generated claims can act on physical reality
- Creativity as the imposition of a novel explanatory constraint with reach beyond its origin
Links & References
- Claude Shannon, “A Mathematical Theory of Communication” (Bell System Technical Journal, 1948)
- Rolf Landauer, “Irreversibility and Heat Generation in the Computing Process” (IBM Journal of Research and Development, 1961)
- Bérut et al., “Experimental verification of Landauer’s principle linking information and thermodynamics” (Nature 483, 2012)
- David Deutsch, The Beginning of Infinity (2011)
- Satoshi Nakamoto, “Bitcoin: A Peer-to-Peer Electronic Cash System”
Related Episodes
Notable Pull Quotes
“We mistook the map for the territory. The engineering tool that was discovered became the ontology.”
“We are not living in the information age. We are living in the Shannon age.”
“A bit is a unit of constraint.”
“Heat death isn’t the death of energy. It’s the death of constraint.”
“Shannon built a meter that reads maximum information when you’re looking at maximum noise.”
Post a comment: