Charles Bennett, Claude Shannon, entropy, information overload, information theory, James Clerk Maxwell, John von Neumann, Léon Brillouin, Leó Szilárd, Maxwell's Demon, Michael Kandel, paradoxes, Rolf Landauer, science fiction, Stanisław Lem, The Cyberiad, thermodynamics
For almost one hundred years, an apparent paradox in the laws of thermodynamics—and, by extension, information theory—existed in the form of Maxwell’s Demon, a thought experiment devised in the 1860s and 70s by the Scottish physicist James Clerk Maxwell. The Second Law of Thermodynamics states that in a thermodynamic process, the sum of the entropies involved will increase. In practice, this means that the temperatures of the systems involved in such a process will converge towards an equilibrium as its energy and matter dissipate evenly throughout the whole. For example, if a hot gas and a cold gas are both released into a suitable closed container, the temperature of the whole (expressed by the velocity, and therefore energy, of the atoms and/or molecules of which the gas is composed) will quickly become uniform.
In thermodynamics, entropy refers to the number of different ways in which a thermodynamic system can be arranged, and is commonly understood to be a measure of disorder. For instance, in this example used by Brian Cox, a pile of sand has a higher entropy than the same amount of sand formed into a castle:
(As an aside, in information theory, entropy refers to the amount of information contained in a message (which can be understood as a sample of a data stream). It is a measure of uncertainty—for example, a coin that always lands on heads results in no uncertainty, and therefore no information and no entropy, whereas a fair coin will continue to provide one unit of information for each flip, and therefore has higher entropy as each outcome is uncertain before it is carried out. This form of entropy is called “Shannon Entropy” (and can also be measured in “shannons”), after Claude Shannon, who introduced the concept in his seminal 1948 work A Mathematical Theory of Communication. The term is somewhat nebulous: a popular account of its origin is that it was suggested jokingly to Shannon by a colleague, John von Neumann, precisely because it was already poorly understood in other disciplines and could thus be defended more easily; in addition, Shannon’s contemporary, Norbert Wiener, treated entropy as a lack of information—the opposite of Shannon’s definition.)
Maxwell’s demon allows for a hypothetical situation in which the Second Law of Thermodynamics can be broken: a “demon” (originally described by Maxwell as a “finite being”, but an entity that can be thought of as a mechanism or device rather than a living creature) sits on a box, divided in half by a screen. The box contains a high-entropy gas of uniform temperature. However, the demon, using information about the velocity (and therefore energy) of each individual molecule within the gas, operates a window which it can open and close quickly enough to allow them, one at a time, into the other half of the box. When repeated over and over again, the demon can theoretically create a scenario in which all of the “cold” molecules end up in one of the box, with all the “hot” molecules in the other, decreasing the entropy of the system as a whole, creating a perpetual-motion machine, and thus breaking the Second Law, all by using pure information and no energy itself. Even though the scenario is highly unlikely to occur (for example, due to the immense technological requirements of building such a “demon”), the mere fact that it can exist in theoretical terms presented an apparent paradox at the heart of thermodynamics.
The paradox was challenged on numerous occasions throughout the twentieth century, first by Leó Szilárd and Léon Brillouin, who pointed out that the demon must record the information necessary to make its judgements, and that doing so requires an expenditure of energy, increasing the entropy of the system as a whole when the demon is included within it. A similar conclusion was reached by Rolf Landauer and later Charles Bennett, who proved that the demon (a “finite being”, remember) must periodically erase information within its memory to keep track of all the information that it is processing. Landauer proved that any logically irreversible operation involving information, including its erasure, requires an expenditure of information and therefore a corresponding increase in entropy (the theoretical lower limit of energy needed to erase one bit of information is now known as the Landauer limit). The demon, therefore, cannot escape the effects of entropy—the act of reducing it in the thermodynamic system that it manipulates creates a corresponding increase within its own memory.
So, where does Stanisław Lem fit into all of this? Lem was a prolific Polish author, best-known for his science-fiction works such as Solaris, but also highly regarded as a writer on the philosophy of technology. His oeuvre includes a collection of humorous short stories, published in 1967 as Cyberiada and translated by Michael Kandel into English for publication as The Cyberiad in 1974. All but one of the stories feature Trurl and Klapaucius, two robotic “constructors” of immense technical skill and ingenuity who issue forth on various “sallies” through a quasi-medieval future universe, set thousands of years after intelligent machines have liberated themselves from human control. The stories (which I highly recommend reading—the book is available through City University Library) cover numerous technological, philosophical and technological questions, including the problem of Maxwell’s Demon.
In one of their sallies, Trurl and Klapaucius are ambushed and held hostage by a pirate. This pirate is unusual in that he possesses a PhD, so rather than demand gold and other precious materials from his captives, he instead requires more valuable booty:
I gather rich mines of information, for such is my lifelong love and avocation, the result of a higher education and, I might add, a practical grasp of the situation, when you consider that, with the usual treasures untutored pirates like to hoard, there is not a blessed thing here one can buy. Information, on the other hand, satisfies one’s thirst for knowledge, and it is well known besides, that everything that is, is information; and thus for centuries now I gather it, and will continue to do so…
The pirate rejects the constructors’ offer of a single piece of valuable information (a formula for producing gold from hydrogen) in favour of the sum total of their learning, with the sole proviso that everything they tell him is true. Faced with eons of reciting their knowledge, Trurl makes the pirate an alternative offer:
If first you solemnly swear, up and down and cross your heart, that you will let us go, we will give you information, information about infinite information, that is, we will make you your very own Demon of the Second Kind, which is magical and thermodynamical, nonclassical and stochastical, and from any old barrel or sneeze it will extract information for you about everything that was, is, may be or ever will be. And there is no demon beyond this Demon, for it is of the Second Kind, and if you want it, say so now!
For anyone familiar with Maxwell’s demon, it becomes clear that the Demon of the Second Kind is its logical development: whereas Maxwell’s Demon uses information to ascertain the movement of molecules, the Demon of the Second Kind—perched over a pinhole made in a container, measuring the air that passes through—uses the movement of molecules to ascertain information, as the trillions of collisions between them that occur in every fraction of a second can theoretically give rise to complex information on a regular basis by chance. Indeed, the constructors’ technological skills are so advanced that they view Maxwell’s Demon (referred to as a Demon of the First Kind) as being totally mundane:
Oh, it’s not as interesting, it’s an ordinary thermodynamic demon, and all it does it let fast atoms out of the hole and keep in the slow. That way you get a thermodynamic perpetuum mobile, which hasn’t a thing to do with information.
The pirate accepts their offer, and they build him a tiny Demon, equipped with paper tape and a diamond-tipped pen, and put it to work on the pinhole made in a barrel; it quickly—”a million times faster” than a telegraph—produces a stream of information. The pirate is entranced and fails to notice his erstwhile captives making their escape; as he realises that the information produced, though true and factual, is also trivial and useless, he becomes physically trapped by the sheer quantity of information that is produced:
He sits there to this day, at the very bottom of his rubbish heap and bins of trash, covered with a mountain of paper, and in the dimness of that cellar the diamond pen still jumps and flickers like the purest flame, recording whatever the Demon of the Second Kind culls from dancing atoms in the rancid air that flows through the hole in the old barrel[…]and there is no hope for him, since this is the harsh sentence the constructors passed upon him for his pirately assult—unless of course the tape runs out, for lack of paper.
Lem’s book predates the mass availability of the Internet by several decades, but this cautionary tale has an even greater meaning in this era of exponential data increase. It is easier than ever before to become lost in the vast quantity of information that is produced by contemporary human society, and we face the additional problem that we cannot even trust all of it to be true, unlike the filter with which the Demon of the Second Kind was constructed. It is perhaps also the case that although the paradox of Maxwell’s Demon has been long since solved, the Information Age has produced its own conundrum—greater availability of data can result in both an increase in productivity, as users are able to find the information they need more quickly and efficiently, but also a decrease if they get bogged down in useless information or are tempted to procrastinate by the growth in information channels that has occurred. The conclusion must be that information quantity counts for nothing unless standards for quality are also adhered to—whether in terms of full and accurate metadata, information retrieval, or the proper education and training of information professionals like myself.