Journal of Human Thermodynamics 2012, 8(1):1‐120 Open Access Journal of Human Thermodynamics ISSN 1559-386X HumanThermodynamics.com/Journal.html Article Thermodynamics ≠ Information Theory: Science’s Greatest Sokal Affair Libb Thims Institute of Human Thermodynamics, Chicago, IL; [email protected] Received: 26 Jun 2012; Reviewed: 14 Dec 2012 – 18 Dec 2012; Published: 19 Dec 2012 Abstract This short article is a long-overdue, seven decades—1940 to present—delayed, inter-science departmental memorandum—though not the first—that INFORMATION THEORY IS NOT THERMODYNAMICS and thermodynamics is not information theory. We repeat again: information theory—the mathematical study of the transmission of information in binary format and or the study of the probabilistic decoding of keys and cyphers in cryptograms—is not thermodynamics! This point cannot be overemphasized enough, nor restated in various ways enough. Information theory is not statistical mechanics—information theory is not statistical thermodynamics—information theory is not a branch of physics. Claude Shannon is not a thermodynamicist. Heat is not a binary digit. A telegraph operator is not a Maxwell’s demon. The statistical theory of radiography and electrical signal transmission is NOT gas theory. The equations used in communication theory have absolutely nothing to do with the equations used in thermodynamics, statistical mechanics, or statistical thermodynamics. The logarithm is not a panacea. Boltzmann was not thinking of entropy in 1894 as ‘missing information’. Concisely, the truncated abstract of this article is: thermodynamics ≠ information theory. In pictorial form, the abstract of this article is: Journal of Human Thermodynamics, 2012, Vol. 8(1) 2 Neumann-Shannon anecdote “Neumann’s proposal to call Shannon’s function defining information by the name ‘entropy’ opened a Pandora’s box of intellectual confusion.” — Antoine Danchin (2001), French geneticist1 The roots of this article trace to what known as the Neumann-Shannon anecdote, a famous, or rather infamous, depending on point of view, discussion between American mathematician and communications engineer Claude Shannon, shown below left, and Hungarian-born American mathematician and chemical engineer John Neumann, shown below right, which occurred in the period between fall 1940 to spring 1941, during Shannon’s postdoctoral year at the Institute for Advanced Study, Princeton, New Jersey, during which time Shannon was wavering on whether or not to call his new logarithm formulation, shown below, of communications in signal transmissions by the name of either information—as in binary coded information sent via a telegraph transmission—or uncertainty—of the Heisenberg uncertainty type, i.e. of ‘uncertainty’ in the mind of someone about to receive a message?2 Several alternately worded versions of the anecdote exist.3 The following is a popular version (with image) from the 2010 book Biomedical Informatics by German computer processing scientist and cognitive researcher Andreas Holzinger:4 Claude Shannon John Neumann Shannon, prior to this famously-repeated 1940 conversation, had developed a long-standing passion for signal transmission. The earliest mention of this was when as a boy in circa 1926 he used a telegraph machine to send Morris code messages to his neighborhood friend a ½-mile away through the barbed wire that ran along the road, as summarized below:5 Journal of Human Thermodynamics, 2012, Vol. 8(1) 3 American Claude Shannon (1916‐2001), as a boy, circa 1926, in Gaylord, Michigan, hooked up a telegraph machine to the barbed wire fence (left) that ran along his country road so that he could communicate with his neighborhood friend a ½‐mile away, using Morse code (center): a 1836 communication system developed by American Samuel Morse, according to which the telegraph system (right) could be used to send coded pulses—dots (short hold) or dashes (long hold)—of electric current along a wire which controlled an electromagnet that was located at the receiving end of the telegraph system.5 The barbed wire photo shown, to note, is of a Texas area farm rather than a Michigan area farm—put the point is still the same: tapping out ‘hey, what are you doing’ on a single button telegraph board—or for that matter typing your daily email on a modern 61-button Dvorak Simplified Keyboard, and the various coding formulas underlying the transmission of these communications are NOT measures of Boltzmann entropy. To clarify, however, before proceeding further, there is a real physical-chemical perspective in which entropy—of the real thermodynamic variety—seems to be a quantitative factor in the sending of messages between people, such as via in person verbal speech, post, telegraph, email, text messaging, Skype, etc., but this is an advanced subject of human chemical thermodynamics, which holds firstly that each person is a surface-attached reactive molecule, a twenty-six-element human molecule to be specific, and secondly that the binding and or debinding of molecules—people—is completely ‘determined’, in the words of American-born Canadian biophysicalN4 chemist Julie Forman-Kay, by ‘the free energy change of the interaction, composed of both enthalpic and entropic terms’.6 Hence, any interaction between two people, such as typing out ‘LOL’ in text message, is a factor—composed of both enthalpic and or entropic terms—involved in the reactionary chemical binding or debonding of two people.196 To clarify further, in this sense, we are not referring to the ‘physical irregularities and functional degradation, an unreliability resulting from the physical effect of the second law in the real world of signal transmissions’, in the 2011 words of American anthropological neuroscientist Terrence Deacon, that REAL entropy differs from SHANNON bit entropy, but rather the way in which entropy plays a role in spontaneous direction of chemical reactions, atomic or human.220 This, however, is a subject beyond the scope of the present article.7 The objective herein is to show that the so-called Shannon entropy (information theoretic entropy)— or specifically the one time Shannon uses the term ‘entropy’ in his 1945 article ‘A Mathematical theory of Cryptography’ and 152 times Shannon uses the term ‘entropy’ in his 1948 article ‘A Mathematical Theory of Communication’, namely: conditional entropy, entropy of two possibilities, entropy of a Journal of Human Thermodynamics, 2012, Vol. 8(1) 4 source, entropy of a joint event, the uncertainty (or entropy) of a joint event—here we see Shannon vacillating on terminology (an inconsistency point we will dig into in the following article)—entropy of a source per symbol of text, entropy per second, entropy per symbol of blocks, relative entropy, entropy of the approximations to English, entropy per unit time, entropy of symbols on a channel, entropy of the channel input per second, true entropy, actual entropy, maximum possible entropy, zero entropy, maximum entropy source, joint entropy of input and output, the entropy of the output when the input is known and conversely, entropy as a measure of uncertainty, input entropy to a channel, entropy of a continuous distribution, entropy of discrete set of probabilities, entropy of an averaged distribution, entropy of a one-dimensional Gaussian distribution, the entropy as a measure in a an absolute way of the randomness of a chance variable, new entropy, old entropy, entropy of an ensemble of functions, entropy of an ensemble per degree of freedom, maximum possible entropy of white noise, entropy of a continuous stochastic process, entropy power, maximum entropy for a given power, entropy power of a noise, entropy loss in linear filters, change of entropy with a change of coordinates, entropy power factor, entropy power gain (in decibels), final power entropy, initial power entropy, output power entropy, entropy power loss, entropy of a sum of two ensembles, entropy power of a distribution, definite entropy of noise, entropy of a received signal less the entropy of noise, maximum entropy of a received signal, greatest possible entropy for a power, entropy (per second) of a received ensemble, noise entropy, noise entropy power, entropy power of the noise, maximum entropy for the power, entropy power of the received signal, entropy power of the sum of two ensembles, entropy powers, individual entropy powers, maximum entropy power of a sum, entropy of the maximum distribution, entropy of the transmitted ensemble, entropy of the white noise, entropy of an underlying stochastic process, maximum possible entropy for a power, have absolutely positively unequivocally NOTHING to do with thermodynamics, specifically the original entropy of Clausius or the ideal gas entropy of Boltzmann or the statistical mechanical entropy of Gibbs. The above usages are but a cesspool of misadopted terminological confusion. Historically, to continue, the amount of ‘Pandora’s box’ confusion, as French geneticist Antoine Danchin puts it, surrounding the falsely-assumed notion that Shannon entropy equals Boltzmann entropy is immense.1 American evolutionary chemist Jeffrey Wicken, in 1987, cogently summarized things as follows (his italics, quote slightly rewritten, square brackets added):8 “It is a mistaken belief that Shannon’s function—called ‘entropy’ by namesake misadoption in information theory (Shannon and Weaver 1949)—has resulted in a true generalization of the Carnot‐Clausius state function [dQ/T] treatment and the Boltzmann‐Gibbs statistical [H function] treatment of the original [thermodynamic] entropy formulation of heat [Q], thus, in a sense, freeing it from disciplinary framework of thermodynamics for use in probability distributions in general— which is a major impediment to productive communication [in science].” In addition, this is not the first time this ‘mistaken belief’ issue has been pointed out. In 1953, British electrical engineer and cognitive scientist Colin Cherry, in his On Human Communication: a Review, a Survey, and a Criticism, gave his reflective opinion:9 Journal of Human Thermodynamics, 2012, Vol. 8(1) 5 “I had heard of ‘entropies’ of language, social systems, and economic systems and its use in various method‐starved studies.