Entropy E 2859 45. Tani J, Fukumura N (1997) Self-organizing internal represen- Nonadditive entropy It usually refers to the basis of tation in learning of navigation: A physical experiment by the nonextensive statistical mechanics. This entropy, de- mobile robot Yamabico. Neural Netw 10(1):153–159 noted Sq,isnonadditive for q 1. Indeed, for two 46. Tani J, Nolfi S (1999) Learning to perceive the world as articu- ¤ lated: An approach for hierarchical learning in sensory-motor probabilistically independent subsystems A and B,it satisfies S (A B) S (A) S (B)(q 1). For his- systems. Neural Netw 12:1131–1141 q C ¤ q C q ¤ 47. Tani J, Nishimoto R, Namikawa J, Ito M (2008) Co-developmen- torical reasons, it is frequently (but inadequately) re- tal learning between human and humanoid robot using a dy- ferred to as nonextensive entropy. namic neural network model. IEEE Trans Syst Man Cybern B. q-logarithmic and q-exponential functions Denoted Cybern 38:1 ln x (ln x ln x), and ex (ex ex ), respectively. 48. Varela FJ, Thompson E, Rosch E (1991) The Embodied mind: q 1 D q 1 D Cognitive science and human experience. MIT Press, Cam- Extensive system So called for historical reasons. A more bridge appropriate name would be additive system. It is a sys- 49. van Gelder TJ (1998) The dynamical hypothesis in cognitive sci- tem which, in one way or another, relies on or is con- ence. Behav Brain Sci 21:615–628 nected to the (additive) Boltzmann–Gibbs entropy. Its 50. Vaughan E, Di Paolo EA, Harvey I (2004) The evolution of con- trol and adaptation in a 3D powered passive dynamic walker. basic dynamical and/or structural quantities are ex- In:PollackJ,BedauM,HusbandP,IkegamiT,WatsonR(eds) pected to be of the exponential form. In the sense of Proceedings of the Ninth International Conference on the Sim- complexity, it may be considered a simple system. ulation and Synthesis of Living Systems. MIT Press, Cambridge Nonextensive system So called for historical reasons. A more appropriate name would be nonadditive sys- tem. It is a system which, in one way or another, relies Entropy on or is connected to a (nonadditive) entropy such as S (q 1). Its basic dynamical and/or structural quan- q ¤ CONSTANTINO TSALLIS1,2 tities are expected to asymptotically be of the power- 1 Centro Brasileiro de Pesquisas Físicas, law form. In the sense of complexity, it may be consid- Rio de Janeiro, Brazil ered a complex system. 2 Santa Fe Institute, Santa Fe, USA Definition of the Subject Article Outline Thermodynamics and statistical mechanics are among Glossary the most important formalisms in contemporary physics. Definition of the Subject They have overwhelming and intertwined applications in Introduction science and technology. They essentially rely on two ba- Some Basic Properties sic concepts, namely energy and entropy.Themathemati- Boltzmann–Gibbs Statistical Mechanics cal expression that is used for the first one is well known On the Limitations of Boltzmann–Gibbs Entropy to be nonuniversal; indeed, it depends on whether we are and Statistical Mechanics say in classical, quantum, or relativistic regimes. The sec- The Nonadditive Entropy S q ond concept, and very specifically its connection with the A Connection Between Entropy and Diffusion microscopic world, has been considered during well over Standard and q-Generalized Central Limit Theorems one century as essentially unique and universal as a physi- Future Directions cal concept. Although some mathematical generalizations Acknowledgments of the entropy have been proposed during the last forty Bibliography years, they have frequently been considered as mere prac- tical expressions for disciplines such as cybernetics and Glossary control theory, with no particular physical interpretation. Absolute temperature Denoted T. What we have witnessed during the last two decades is the Clausius entropy Also called thermodynamic entropy. growth, among physicists, of the belief that it is not neces- Denoted S. sarily so. In other words, the physical entropy would ba- Boltzmann–Gibbs entropy Basis of Boltzmann–Gibbs sically rely on the microscopic dynamical and structural statistical mechanics. This entropy, denoted SBG,isad- properties of the system under study. For example, for sys- ditive. Indeed, for two probabilistically independent tems microscopically evolving with strongly chaotic dy- subsystems A and B,itsatisfiesS (A B) S (A) namics, the connection between the thermodynamical en- BG C D BG C SBG(B). tropy and the thermostatistical entropy would be the one 2860 E Entropy found in standard textbooks. But, for more complex sys- later on generalized into tems (e. g., for weakly chaotic dynamics), it becomes ei- ther necessary, or convenient, or both, to extend the tradi- S k f (q; p)ln[f (q; p)] dq dp ; (4) D tional connection. The present article presents the ubiqui- “ tous concept of entropy, useful even for systems for which where (q; p) is called the -space and constitutes the phase no energy can be defined at all, within a standpoint re- space (coordinate q and momentum p) corresponding to flecting a nonuniversal conception for the connection be- one particle. tween the thermodynamic and the thermostatistical en- Boltzmann’s genius insight – the first ever mathemat- tropies. Consequently, both the standard entropy and its ical connection of the macroscopic world with the micro- recent generalizations, as well as the corresponding statis- scopic one – was, during well over three decades, highly tical mechanics, are here presented on equal footing. controversial since it was based on the hypothesis of the existence of atoms. Only a few selected scientists, like Introduction the English chemist and physicist John Dalton, the Scot- The concept of entropy (from the Greek !, en tish physicist and mathematician James Clerk Maxwell, trepo, at turn, at transformation) was first introduced in and the American physicist, chemist and mathematician 1865 by the German physicist and mathematician Rudolf Josiah Willard Gibbs, believed in the reality of atoms and Julius Emanuel Clausius, Rudolf Julius Emanuel in or- molecules. A large part of the scientific establishment was, der to mathematically complete the formalism of classi- at the time, strongly against such an idea. The intricate cal thermodynamics [55], one of the most important the- evolution of Boltzmann’s lifelong epistemological strug- oretical achievements of contemporary physics. The term gle, which ended tragically with his suicide in 1906, may was so coined to make a parallel to energy (from the Greek be considered as a neat illustration of Thomas Kuhn’s o&, energos, at work), the other fundamental con- paradigm shift, and the corresponding reaction of the sci- cept of thermodynamics. Clausius connection was given entific community, as described in The Structure of Sci- by entific Revolutions. There are in fact two important for- malisms in contemporary physics where the mathematical ıQ dS ; (1) theory of probabilities enters as a central ingredient. These D T are statistical mechanics (with the concept of entropy as where ıQ denotes an infinitesimal transfer of heat. In a functional of probability distributions) and quantum other words, 1/T acts as an integrating factor for ıQ. mechanics (with the physical interpretation of wave func- In fact, it was only in 1909 that thermodynamics was tions and measurements). In both cases, contrasting view- finally given, by the Greek mathematician Constantin points and passionate debates have taken place along more Caratheodory, a logically consistent axiomatic formula- than one century, and continue still today. This is no sur- tion. prise after all. If it is undeniable that energy is a very deep In 1872, some years after Clausius proposal, the Aus- and subtle concept, entropy is even more. Indeed, energy trian physicist Ludwig Eduard Boltzmann introduced concerns the world of (microscopic) possibilities,whereas a quantity, that he noted H, which was defined in terms entropy concerns the world of the probabilities of those of microscopic quantities: possibilities, a step further in epistemological difficulty. In his 1902 celebrated book Elementary Principles of H f (v)ln[f (v)] dv ; (2) Statistical Mechanics, Gibbs introduced the modern form • of the entropy for classical systems, namely where f (v)dv is the number of molecules in the veloc- ity space interval dv. Using Newtonian mechanics, Boltz- S k d f (q; p)ln[Cf(q; p)] ; (5) D mann showed that, under some intuitive assumptions Z (Stoßzahlansatz or molecular chaos hypothesis)regarding where represents the full phase space of the system, thus thenatureofmolecularcollisions,H does not increase containing all coordinates and all momenta of its elemen- with time. Five years later, in 1877, he identified this quan- tary particles, and C is introduced to take into account the tity with Clausius entropy through kH S,wherek is finite size and the physical dimensions of the smallest ad- a constant. In other words, he established that missible cell in -space. The constant k is known today to be a universal one, called Boltzmann constant, and given S k f (v)ln[f (v)] dv ; (3) 23 D by k 1:3806505(24) 10 Joule/Kelvin. The studies • D Entropy E 2861 of the German physicist Max Planck along Boltzmann and This violation is one of the mathematical manifesta- Gibbs lines after the appearance of quantum mechanical tions that, at the microscopic level, the state of any concepts, eventually led to the expression physical system exhibits its quantum nature. Expansibility Also S (p ; p ;:::;p ; 0) S (p ; p ; BG 1 2 W D BG 1 2 S k ln W ; (6) :::; D pW ), i. e., zero-probability events do not modify our information about the system. which he coined as Boltzmann entropy. This expression is Maximal value SBG is maximized at equal probabilities, carved on the stone of Boltzmann’s grave at the Central i.