Axioms and Uniqueness Theorem for Tsallis Entropy

19 June 2000

Physics Letters A 271Ž. 2000 74±79 www.elsevier.nlrlocaterpla

Axioms and uniqueness theorem for Tsallis entropy

Sumiyoshi Abe) College of Science and Technology, Nihon UniÕersity, 7-24-1 Narashinodai, Funabashi, Chiba 274-8501, Japan Received 19 January 2000; received in revised form 2 May 2000; accepted 2 May 2000 Communicated by C.R. Doering

Abstract

The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy and a complete proof of the uniqueness theorem for the Tsallis entropy is presented. This improves the discussion of dos Santosw J. Math Phys. 38Ž. 1997 4104x . q 2000 Elsevier Science B.V. All rights reserved.

PACS: 05.20.yy

s r 1. Introduction by the equiprobability distribution, i.e., pi 1 W Ž.is1, 2, PPP , W and the value itself is SslnW, The concept of entropy is the fundamental ingre- which is the celebrated Boltzmann formula. wx dient in information theory and statistical mechanics. To characterize what the entropy is, Shannon 1 and Khinchinwx 2 investigated its axiomatic founda- Given a probability distribution Ä4piWis1, 2, PPP , sat- F s PPP W s tion. The Shannon±Khinchin axioms are given as isfying 0 piiiŽ.1, 2, , W and Ý s1 pi 1, the Boltzmann±Shannon entropy, i.e., the ordinary follows: information entropy, is defined by

W PPP sy wxI. SpŽ., p , PPP , p is continuous with re- SpŽ.12, p , , pWiikpÝ ln p ,1 Ž.12 W is1 spect to all its arguments and takes its maximum for s r s the equiprobability distribution pi 1 WiŽ 1, 2, where k is a positive constant, which is henceforth PPP , W ., set equal to unity for simplicity. This quantity is a wxII . SAwxwx, B sSAqSBA, positive and concave function of Ä4piWis1, 2, PPP , .It wx PPP III . SpŽ.12, p , , p W ,0 also fulfills the H-theorem. Its maximum is realized s PPP SpŽ.12, p , , pW .

In the second axiom, SAwx, B and SA wxare the ) Fax: q81-47-464-9342. entropies of the composite system Ž.A, B with the s E-mail address: [email protected]Ž. S. Abe . joint probability distribution pAijŽ., BiŽ 1, 2,

PPP , W; js1, 2, PPP , U . and the subsystem A Boltzmann±Shannon entropy, it takes its maximum s s r s with the marginal probability distribution pAiŽ. for the equiprobability distribution pi 1 WiŽ U PPP s y y1 Ý js1 pAijŽ., B , respectively. SBA stands for 1, 2, , W .:ItsvalueisSq Ž.1 q the conditional entropy associated with the condi- Ž.W 1yq y1 , which is a monotonically increasing tional probability distribution function of W. However, the additivity is violated. For statistically independent systems, A and B, S pBAŽ.s pAŽ.Ž., BpA. q ij ij i yields Writing the entropy of pBAas SBA , the ijŽ. i wxwxwxs q conditional entropy is given by SAqqq, B SA SB W q y wx wx s Ž.A s Ž.1 qSqq AS B,6 Ž. SBA ²:SBAiiiÝ pAŽ. SBA. s i 1 which is referred to as the pseudoadditivity. Clearly, 2 Ž. the additivity holds only in the limit q™1. Eq.Ž. 6 In the special case when A and B are statistically has been carefully discussed in Ref.wx 5 . independent, SBAsSBwx, leading to the addi- In a recent paperwx 6 , dos Santos has made an tivity: interesting discussion about uniqueness of the Tsallis entropy in Eq.Ž. 5 . He has shown that that if a SAwxwxwx, B sSAqSB.3 Ž. quantity is:wx i continuous with respect to the proba- wx wx The uniqueness theorem 1,2 states that a quantity bility distribution Ä4piWis1, 2, PPP , ; ii a monotoni- satisfying the axiomswx I ± w III x is uniquely equal to S cally increasing function of W in the case of the s r s in Eq.Ž. 1 . equiprobability distribution pi 1 WiŽ 1, 2, We note that there is a correspondence relation PPP , W .;wx iii satisfies the pseudoadditivity in Eq. between the Bayes multiplication law and the axiom Ž.6 for the statistically independent systems, A and wxII : B; andwx iv fulfills the relation s l wx pAijŽ.Ž., B pA i p ijŽ. BA SA, B PPP SpqŽ.12, p , , pW s wxq SA SBA.4Ž. q sSpŽ.Ž., p q pS Now, in the field of statistical mechanics, there is qL M L q a growing interest in nonextensive generalization of =Ž.p rp , p rp , PPP , p rp Boltzmann±Gibbs theory. In particular, the one initi- 1 L 2 LWL wx1 q ated by Tsallis 3,4 is receiving much attention. qŽ.ŽpSprp , p rp , PPP , p rp ., This formalism is based on the following single- Mq1 M 2 MWM W W parameter generalization of the Boltzmann±Shannon where p s Ý sL p and p sÝ s q p , then it Li1 iMiWL 1 i entropy: is identical to the Tsallis entropy. Here, comparing the set of the Shannon±Khinchin PPP wx w x wx w x SpqŽ.12, p , , pW axioms I ± III with that of i ± iv , we find that the W complete parallelism is missing between the two. 1 q s p y1 q)0. 5 The main reason for this is due to the absence of the y Ý Ž.i Ž . Ž. 1 q is1 concept of `nonextensive conditional entropy'. ™ In this Letter, we present the axioms for the In the limit q 1, Sq converges to S in Eq.Ž. 1 Ž.with k'1 . This quantity is also positive and con- Tsallis entropy by introducing the nonextensive concave, and satisfies the H-theorem. In addition, as the ditional entropy. Then we prove the uniqueness theorem for the Tsallis entropy. Our proof essentially follows a natural generalization of the line presented in Ref.wx 2 . This improves the discussion given in wx 1 A comprehensive list of references can currently be obtained Ref. 6 and establishes the complete parallelism with from http:rrtsallis.cat.cbpf.brrbiblio.htm the Shannon±Khinchin axioms. 76 S. AberPhysics Letters A 271() 2000 74±79

2. Nonextensive conditional entropy In fact, the pseudoadditivity in Eq.Ž. 6 is recovered in the special case when A and B are statistically In nonextensive statistical mechanics, it is known independent each other. wx4 that the average of a physical quantity Q s Ä4Q i is1, 2, PPP , W is given in terms of the normal- ized q-expectation value: 3. Axioms and uniqueness theorem for Tsallis W q entropy Qp W Ý iiŽ. s ' is1 Now, the set of the axioms we present for the ²:Q q Ý QPii W ,7Ž. is1 q Tsallis entropy is the following: Ý Ž.pi is1 wxI.) SpŽ., p , PPP , p is continuous with ' q W q q 12 W where PiiiŽ.p Ý s1 Ž.pi is the escort distribu- respect to all its arguments and takes its maximum tion associated with p wx7 . To be consistent with the s r s i for the equiprobability distribution pi 1 WiŽ nonextensive formalism, we use this concept to gen- 1, 2, PPP , W ., eralize the definition in Eq.Ž. 2 . For this purpose, we wx) wxwxs q II . SAqqq, B SA SBA calculate the Tsallis entropy of the conditional prob- q y wx Ž.1 qSqq AS BA, ability distribution wx) PPP III . SpqŽ.12, p , , pW ,0 U s PPP 1 q SpqŽ.12, p , , pW . SBA s pBA y1. 8 qiy Ý ijŽ. Ž. 1 q ½5js1 wx) wx) Theorem. A quantity satisfying I ± III is From this quantity, we define the nonextensive con- uniquely equal to the Tsallis entropy. ditional entropy as follows: Proof. First, let us consider the equiprobability dis- s r s PPP tribution pi 1 WiŽ.1, 2, , W and put Ž.A SBAs²:SBA 11 1 qqiq PPP s Sqq,, ,:LWŽ..12 Ž. W ž/WW W q Ý pAiqiŽ. SBA wx) s From III , it follows that s i 1 .9 W Ž. 11 1 q s PPP Ý pAiŽ. LWqqŽ. S ,, ,,0 s ž/WW W i 1 11 Using the definition of the Tsallis entropy, we find FS ,,PPP , that Eq.Ž. 9 can be expressed as q ž Wq1 Wq1 wxwxy SAqq, B SA 11 SBAs .10 , sLWq1, 13 q q y wx Ž. q q qŽ.Ž. 1 Ž.1 qSq A W 1 W 1 /

An important point here is that, with this definition, which means that LWqŽ.is a nondecreasing func- a natural nonextensive generalization of the corre- tion of W. spondence relation in Eq.Ž. 4 is established in conformity with the pseudoadditivity in Eq.Ž. 6 : Consider m statistically independent systems, A 1, A , PPP , A , each of which contains rŽ.G2 pAŽ.Ž., B spA p BAl 2 m ij i ijŽ. equally likely events. Then, we have = wx SAq , B 11 1 wxs PPP s wxq SAqkS q ,, , SAqqSBA ž/rr r q y wx s F F Ž.1 qSqq ASBA.11 Ž. LrqŽ.Ž1 k m ..14 Ž . S. AberPhysics Letters A 271() 2000 74±79 77

Usingwx II) for these independent systems, we find We note that, in deriving these inequalities, the following Ansatz has to be made: SAwx, A , PPP , A q 12 m - q y q y - 0 1 Ž.Ž.Ž.Ž.1 qLqq r,1 1 qL s 1 m m ky1 k s Ž.1yqLr Ž. ) Ý ž/k q for q 1.Ž. 22 ks1 Later, we shall see that this is in fact justified. Also, 1 m s 1q Ž.Ž.1yqL r y1. Ž. 15 from Eq.Ž. 18 , it is evident that 1yq ½5q m ln sm1 wxPPP s m Since SAq 12, A , , A mqLrŽ., we have FFq.23Ž. n lnrnn 1 m m s q y y LrqqŽ.1 Ž1 qL .Ž. r 1. Combining this with Eq.Ž. 21 , we have 1yq ½5 Ž.16 q y ln 1 Ž.Ž.1 qLq s ln s 1 yF.24 q y Ž. Similarly, for any other positive integers n and s ln 1 Ž.Ž.1 qLq r lnrn equal to or larger than 2, we have Since n can be arbitrarily large, we obtain 1 n n s q y y LsqqŽ.½51 Ž1 qL . Ž. s 1. q y 1yq ln 1 Ž.Ž.1 qLq r Ž.17 lnr q y It is always possible to take m, r, n, and s which ln 1 Ž.Ž.1 qLq s s :slŽ.q ,25 Ž . satisfy ln s m F n F mq1 r s r .18Ž. where lŽ.q is a separation constant dependent on q. Therefore, we find Since Lq is a nondecreasing function, we have the inequalities 1 s lŽ.q y m F n F mq1 LrqŽ.r 1. Ž 26 . LrqqqŽ.Ls Ž.Lr Ž .,19 Ž. 1yq which lead to Clearly, lŽ.1 s0.Next, let us consider any rational 1 numbers q y m y 1 Ž.Ž.1 qLq r 1 1yq ½5 gi p s Ž.Ž.is1, 2, PPP , W ,27 i g 1 n F 1q Ž.Ž.1yqL s y1 y ½5q s PPP 1 q where gii Ž.1, 2, , W are any positive in- s W tegers and g Ýis1 gi.The system A is assumed to 1 mq1 F 1q Ž.Ž.1yqL r y1. Ž. 20 be described by the probability distribution 1yq ½5q s r Ä4piig g is1, 2, PPP , W . We construct the system B - - dependent on A as follows. B contains g events, Here, it is necessary to examine two cases, 0 q 1 PPP ) which are partitioned into W groups: B12, B , , and q 1, separately. After simple algebra, we find F F BWj. B Ž.1 j W has g jevents. Once the ith that in both cases the following inequalities hold: s event A iiof the system A was found, i.e., A A , q y then, in the system B, g events of the group B s m ln 1 Ž.Ž.1 qLq s m 1 iji FFq.21 r q y Ž. have the same conditional probability 1 g i and all nnnln 1 Ž.Ž.1 qLq r the events of the other groups B j/ i have the vanish- 78 S. AberPhysics Letters A 271() 2000 74±79

ing probability. Sq of B thus constructed is calcu- A remaining task is to determine lŽ.q . For this lated to be purpose, it is sufficient to calculate the nonextensive 11 1 conditional entropy using the form in Eq.Ž. 31 and s PPP s wx) SBAqiqS ,, , Lg qiŽ. impose II on it. Consequently, we find ž/ggii g i lŽ.q s1yq.32 Ž . 1 l s Ž.g Ž.q y1. Ž. 28 1yq i At this stage, we also see that the Ansatz in Eq.Ž. 22 is in fact justified. Thus, we see that S satisfying Therefore, the nonextensive conditional entropy is q wxI) ±wx III) is uniquely equal to the Tsallis entropy in given by Eq.Ž.Ž 5 . Q.E.D. .

Ž.A SBAs²:SBA qqiq 4. Concluding remarks W q Ž.pSBA Ý iq i We have constructed the nonextensive conditional s is1 W entropy in conformity with the Bayes multiplication q Ý Ž.p i law and the pseudoadditivity of the Tsallis entropy. is1 We have generalized the Shannon±Khinchin axioms W for the Boltzmann±Gibbs entropy to the nonexten- q lŽ.q Ý Ž.Ž.pgii sive systems. Based on the proposed set of axioms, 1 s syi 1 1. we have proved the uniqueness theorem for the y W 1 q q Tsallis entropy. p Ý Ž.i Ž. is1 Recently, the nonextensive nonadditive conditional entropy has been discussed in the quantum Ž.29 context in Ref.wx 8 . There, it has been shown to give On the other hand, the composite system Ž.A, B rise to the strongest criterion for separability of the F F consists of the events ABijsi Ž.1 i W . For a density matrix of a bipartite spin-1r2 system for given i, the number of possible events ABijsi validity of local realism. We also mention that char- F F Ž.1 i W is g i, and therefore the total number of acterization of the Tsallis entropy has been consid- W s wx events in the composite system is Ýis1 g i g. The ered in Ref. 9 from the viewpoint of the concept of probability of finding the event ABijsiiis p `composability', which means that the entropy of the = r r Ž.1 g i , which is the equiprobability 1 g. There- total system composed of statistically independent fore, Sq of the composite system is subsystems is expressed as a certain function of the 1 entropies of such subsystems.Ž The additivity of the wxs s lŽ.q y SAqq, B LgŽ.g 1. Ž 30 . Boltzmann±Shannon entropy and the pseudoadditiv- 1yq ity of the Tsallis entropy are the actual examples.. wx) Substituting Eqs.Ž. 29 and Ž. 30 into II and using The authors of Ref.wx 9 has shown that if a quantity Eq.Ž. 27 , we have satisfies the composabilityŽ and some other supple- W q mentary conditions. , then it is given by the Tsallis Ý Ž.pi entropy with q)1. 1 s SAwxsyi 1 1. 31 q y W Ž. 1 q qql Ž.q Ý Ž.pi is1 Acknowledgements s PPP This holds for any rational pii Ž.1, 2, , W , but actually for any probability distribution The author would like thank Professor A.K. Ra-

Ä4p i is1, 2, PPP , W due to the assumption of continuity jagopal for discussions about diverse topics of inwx I) . nonextensive statistical mechanics. This work was S. AberPhysics Letters A 271() 2000 74±79 79 supported in part by the GAKUJUTSU-SHO Pro- wx3 C. Tsallis, J. Stat. Phys. 52Ž. 1988 479. wx gram of College of Science and Technology, Nihon 4 C. Tsallis, R.S. Mendes, A.R. Plastino, Physica A 261Ž. 1998 University. 534. wx5 R.P. Di Sisto, S. Martõnez,Â R.B. Orellana, A.R. Plastino, A. Plastino, Physica A 265Ž. 1999 590. wx6 R.J.V. dos Santos, J. Math. Phys. 38Ž. 1997 4104. References wx7 C. Beck, F. Schlogl,È Thermodynamics of Chaotic Systems: An Introduction, Cambridge University Press, Cambridge, 1993. wx wx1 C.E. Shannon, W. Weaver, The Mathematical Theory of 8 S. Abe, A.K. Rajagopal, Nonadditive Conditional Entropy and r Communication, University of Illinois Press, Urbana, 1963. its Significance for Local Realism, e-print quant-ph 0001085. wx Ž. wx2 A.I. Khinchin, Mathematical Foundations of Information The- 9 M. Hotta, I. Joichi, Phys. Lett. A 262 1999 302. ory, Dover, New York, 1957.