arXiv:cond-mat/0005538 31 May 2000 PACS is presented. nonextensive of concept the on conditional based systems nonextensive the to way natural a The Axioms number: Shannon- entropy This 05. College Khinchin improves 20 and for .-y and a of complete Funabashi axioms Science the Tsallis discussion of for uniqueness proof of proof and Technology Sumiyoshi , the Chiba ordinary 1 the 274-8501 dos uniqueness Abe entropy Santos. information , Nihon , Japan theorem University entropy theorem for are the , generalized Tsallis entropy in 1. Introduction

The concept of entropy is the fundamental ingredient in information theory and statistical mechanics. Given a probability distribution {}p satisfying i i=1 2,,L , W

W 0≤p(,,,) i = 1 2 L W and ∑ p = 1, the Boltzmann-Shannon entropy, i.e., the i i=1 i ordinary information entropy, is defined by

W ()L = − S p1, p 2 , , pW k∑ p i ln pi , (1) i=1

where k is a positive constant, which is henceforth set equal to unity for simplicity. This quantity is a positive and concave function of {}p . It also fulfills the H-theorem. i i=1 2,,L , W = Its maximum is realized by the equiprobability distribution, i.e., pi 1/ W

(,,,) i= 1 2 L W and the value itself is SW= ln , which is the celebrated Boltzmann formula. To characterize what the entropy is, Shannon [1] and Khinchin [2] investigated its axiomatic foundation. The Shannon-Khinchin axioms are given as follows:

()L [] S p1,,, p 2 pW is continuous with respect to all its arguments and takes

= its maximum for the equiprobability distribution pi 1/ W

(,,,) i= 1 2 L W ,

[] SABSASBA[], = [] + [],

()LL= () [¡] S p1,,,,,,, p 2 pWW0 S p 1 p 2 p .

In the second axiom, SAB[], and SA[] are the entropies of the composite system

2 () () =LL = AB, with the joint probability distribution p i j A,(,,,;,,,) B i1 2 W j 1 2 U

U and the subsystem A with the marginal probability distribution p() A= ∑ p() A, B , ij =1 i j respectively. SBA[] stands for the conditional entropy associated with the conditional

= () () probability distribution pi j() B A p i j A, B p i A . Writing the entropy of pi j () B A

as SBA[]i , the conditional entropy is given by

W = ()A = () SBA[] SBA[]i ∑ pASBAi[] i . (2) i=1

In the special case when A and B are statistically independent, SBASB[] = [], leading to the additivity:

SABSASB[], = [] + []. (3)

The uniqueness theorem [1,2] states that a quantity satisfying the axioms []-[¡] is uniquely equal to S in eq. (1). We note that there is a correspondence relation between the Bayes multiplication law and the axiom []:

() = () ↔ = [] + pABi j,, pApBA i i j () SAB[] SA SBA[]. (4)

Now, in the field of statistical mechanics, there is a growing interest in nonextensive generalization of Boltzmann-Gibbs theory. In particular, the one initiated by Tsallis [3-5] is receiving much attention. This formalism is based on the following single-parameter

3 generalization of the Boltzmann-Shannon entropy:

W 1  q  S() p,,, pL p = ∑ ()p −1 ()q > 0 . (5) q1 2 W−  i  1 q  i=1 

→ ≡ In the limit q 1, Sq converges to S in eq. (1) (with k 1). This quantity is also positive and concave, and satisfies the H-theorem. In addition, as the Boltzmann-Shannon = entropy, it takes its maximum for the equiprobability distribution pi 1/ W

= L =() − −1 1−q − (,,,) i1 2 W : Its value is Sq 1 q() W 1 , which is a monotonically increasing function of W . However, the additivity is violated. For statistically independent systems, A and B, Sq yields

= [] + [] + − [][] SABq[],() SA q SB q1 qSASB q q , (6)

which is referred to as the pseudoadditivity. Clearly, the additivity holds only in the limit q → 1. Equation (6) has been carefully discussed in Ref. [6].

In a recent paper [7], dos Santos has made an interesting discussion about uniqueness of the Tsallis entropy in eq. (5). He has shown that that if a quantity is [‡] continuous with respect to the probability distribution {}p , [·] a monotonically increasing i i=1 2,,L , W

= = L function of W in the case of the equiprobability distribution p i 1/(,,,) W i 1 2 W , and [] satisfies the pseudoadditivity in eq. (6) for the statistically independent systems,

()L = () + A and B, and [ ¶ ] fulfills the relation S q p1,,,, p 2 p W S q p L p M

()q ()LL+ ()q () = pSpppp L q1/,/,,//,/,,/ L 2 L pp W L pSpppp M q1 M 2 M pp W M , where pL

WL = W ∑ = pi and pM∑ = + p i , then it is identical to the Tsallis entropy. i 1 i WL 1

4 Here, comparing the set of the Shannon-Khinchin axioms []-[¡] with that of [‡]-

[¶], we find that the complete parallelism is missing between the two. The main reason for this is due to the absence of the concept of “nonextensive conditional entropy”. In this paper, we present the axioms for the Tsallis entropy by introducing the nonextensive conditional entropy. Then we prove the uniqueness theorem for the Tsallis entropy. Our proof essentially follows a natural generalization of the line presented in Ref. [2]. This improves the discussion given in Ref. [7] and establishes the complete parallelism with the Shannon-Khinchin axioms.

2. Nonextensive conditional entropy

In nonextensive statistical mechanics, it is known [4] that the average of a physical quantity QQ= {} is given in terms of the normalized q-expectation value: i i=1 2,,L , W

W ()q W ∑ Qi p i = ≡ i=1 QQPq ∑ i i W , (7) i=1 ()q ∑ pi i=1

W where P≡ ()() pq ∑ p q is the escort distribution associated with p [8]. To be i i i=1 i i consistent with the nonextensive formalism, we use this concept to generalize the definition in eq. (2). For this purpose, we calculate the Tsallis entropy of the conditional probability distribution

5 U 1  q  SBA[] = ∑ []p() B A −1. (8) q i− i j 1 q  j =1 

From this quantity, we define the nonextensive conditional entropy as follows:

W ∑ []p() Aq S[] B A ()A i q i = = i=1 SBASBAq[] q[] i W . (9) q () q ∑ []pi A i=1

Using the definition of the Tsallis entropy, we find that equation (9) can be expressed as

SABSA[], − [] SBA[] = q q . (10) q + − [] 1() 1 q Sq A

An important point here is that, with this definition, a natural nonextensive generalization of the correspondence relation in eq. (4) is established in conformity with the pseudoadditivity in eq. (6):

() = () pi j A, B p i A p i j () B A (11) ↔ = [] + + − [] SABq[],(). SA q SBA q[] 1 qSASBA q q[]

In fact, the pseudoadditivity in eq. (6) is recovered in the special case when A and B are statistically independent each other.

6 3. Axioms and uniqueness theorem for Tsallis entropy

Now, the set of the axioms we present for the Tsallis entropy is the following:

()L []* S q p1,,, p 2 p W is continuous with respect to all its arguments

= and takes its maximum for the equiprobability distribution pi 1/ W

(,,,) i= 1 2 L W , = [] + + − [] []* SABq[],() SA q SBA q[] 1 qSASBA q q[],

()LL= () [¡]* S q p1,,,,,,, p 2 p W0 S q p 1 p 2 p W .

Theorem: A quantity satisfying []*-[¡]* is uniquely equal to the Tsallis entropy.

= = L Proof: First, let us consider the equiprobability distribution p i 1/(,,,) W i 1 2 W and put

 1 1 1  S ,,,:L = LW(). (12) q WWW  q

From [¡]*, it follows that

 1 1 1  LWS() = ,,,,L 0 q q  WWW  (13)  1 1 1 1  ≤ S ,,,,,L =LW() +1 q  WWWW+1 +1 +1 +1 q

() which means that LWq is a nondecreasing function of W .

7 L Consider m statistically independent systems, AAA 1,,, 2 m , each of which contains r ()≥ 2 equally likely events. Then, we have

 1 1 1 SAS[] = ,,,L = L() r ()1 ≤k ≤ m . (14) q k q r r r q

Using []* for these independent systems, we find

m m k SAAA[],,,()L = ∑   1 − qk −1[] L() r q1 2 m   q k =1 k

1 m = {}[]1+() 1 − q L() r − 1 . (15) 1 − q q

L = ()m Since S q[] A1,,, A 2 A m L q r , we have

1 m L() r m = {}[]1+() 1 − q L() r − 1 . (16) q 1 − q q

Similarly, for any other positive integers n and s equal to or larger than 2, we have

1 n L() s n = {}[]1+() 1 − q L() s − 1 . (17) q 1 − q q

It is always possible to take m,,, r n and s which satisfy

rm≤ s n ≤ r m+1 . (18)

8 Since Lq is a nondecreasing function, we have the inequalities

m ≤ n ≤ m+1 Lq () r Lq () s Lq () r , (19)

which lead to

1 m 1 n {}[]1+()() 1 − q L() r − 1 ≤ {}[]1+ 1 − q L() s − 1 1 − q q 1 − q q

1 m+1 ≤ {}[]1+() 1 − q L() r − 1 . (20) 1 − q q

Here, it is necessary to examine two cases, 0 1, separately. After simple algebra, we find that in both cases the following inequalities hold:

+ − () m ln[]1 ( 1 q ) Lq s m 1 ≤ ≤ + . (21) + − () n ln[]1 ( 1 q ) Lq r n n

We note that, in deriving these inequalities, the following Ansatz has to be made:

< + − () + − () < > 0 1(),() 1q Lq r 1 1 q L q s 1 for q 1. (22)

Later, we shall see that this is in fact justified. Also, from eq. (18), it is evident that

m ln s m 1 ≤ ≤ + . (23) n ln r n n

Combining this with eq. (21), we have

9 + − () ln[]1 ( 1 q ) Lq s ln s 1 − ≤ . (24) + − () ln[]1 ( 1 q ) Lq r ln r n

Since n can be arbitrarily large, we obtain

ln[]1+ ( 1 − q ) L() r ln[] 1+ ( 1 − q ) L() s q= q := λ()q , (25) ln r ln s

where λ()q is a separation constant dependent on q. Therefore, we find

1 λ() L() r = []r q −1 . (26) q 1 − q

Clearly, λ()1= 0.

Next, let us consider any rational numbers

g p = i (,,,) i= 1 2 L W , (27) i g

W where g (,,,)i= 1 2 L W are any positive integers and g= ∑ g .The system A is i i=1 i assumed to be described by the probability distribution {}p= g/ g . We i i i=1,,, 2 L W construct the system B dependent on A as follows. B contains g events, which are L ≤ ≤ partitioned into W groups: BBB 1,,, 2 W . Bj ()1 j W has gj events. Once the = ith event Ai of the system A was found, i.e., AAi , then, in the system B, gi

events of the group B j= i have the same conditional probability 1/ gi and all the events of

10 the other groups B j≠ i have the vanishing probability. Sq of B thus constructed is calculated to be

  1 1 1 1  λ()q  SBAS[] =  ,,,L  = L() g = ()g −1 . (28) q i q   q i−  i  gi g i g i 1 q

Therefore, the nonextensive conditional entropy is given by

W q ∑ ()p S[] B A ()A i q i = = i=1 SBASBAq[] q[] i W q q ∑ ()pi i=1

W  λ  q () ()q ∑ ()pi gi  = 1  i=1 −  W 1 . (29) 1 − q  () q   ∑ pi   i=1 

() On the other hand, the composite system AB, consists of the events ABi j= i ≤ ≤ ≤ ≤ ()1 i W . For a given i, the number of possible events ABi j= i ()1 i W is gi , and

W therefore the total number of events in the composite system is ∑ g= g. The i=1 i × () probability of finding the event ABi j= i is pi1/ g i , which is the equiprobability 1/ g.

Therefore, Sq of the composite system is

1 λ() S[] A, B= L() g = []g q −1 . (30) q q 1 − q

Substituting eqs. (29) and (30) into []* and using eq. (27), we have

11 W  q   ∑ ()p  1 i [] =  i=1 −  SAq W 1 . (31) 1 − q  () q+λ() q  ∑ pi   i=1 

= L This holds for any rational pi (,,,)i 1 2 W , but actually for any probability distribution {}p due to the assumption of continuity in []*. i i=1 2,,L , W

A remaining task is to determine λ()q . For this purpose, it is sufficient to calculate the nonextensive conditional entropy using the form in eq. (31) and impose []* on it.

Consequently, we find

λ()q=1 − q. (32)

At this stage, we also see that the Ansatz in eq. (22) is in fact justified. Thus, we see that

Sq satisfying []*-[¡]* is uniquely equal to the Tsallis entropy in eq. (5). (Q.E.D.)

4. Concluding remarks

We have constructed the nonextensive conditional entropy in conformity with the Bayes multiplication law and the pseudoadditivity of the Tsallis entropy. We have generalized the Shannon-Khinchin axioms for the Boltzmann-Gibbs entropy to the nonextensive systems. Based on the proposed set of axioms, we have proved the uniqueness theorem for the Tsallis entropy. Recently, the nonextensive (nonadditive) conditional entropy has been discussed in the

12 quantum context in Ref. [9]. There, it has been shown to give rise to the strongest criterion for separability of the density matrix of a bipartite spin-1/2 system for validity of local realism. We also mention that characterization of the Tsallis entropy has been considered in Ref. [10] from the viewpoint of the concept of “composability”, which means that the entropy of the total system composed of statistically independent subsystems is expressed as a certain function of the entropies of such subsystems. (The additivity of the Boltzmann-Shannon entropy and the pseudoadditivity of the Tsallis entropy are the actual examples.) The authors of Ref. [10] has shown that if a quantity satisfies the composability (and some other supplementary conditions), then it is given by the Tsallis entropy with q > 1.

Acknowledgments

The author would like thank Professor A. K. Rajagopal for discussions about diverse topics of nonextensive statistical mechanics. This work was supported in part by the GAKUJUTSU-SHO Program of College of Science and Technology, Nihon University.

13 References

[1] C. E. Shannon and W. Weaver, The Mathematical Theory of Communication

(University of Illinois Press, Urbana, 1963).

[2] A. I. Khinchin, Mathematical Foundations of Information Theory

(Dover, New York, 1957). [3] C. Tsallis, J. Stat. Phys. 52 (1988) 479. [4] C. Tsallis, R. S. Mendes and A. R. Plastino, Physica A 261 (1998) 534. [5] A comprehensive list of references can currently be obtained from http://tsallis.cat.cbpf.br/biblio.htm [6] R. P. Di Sisto, S. Martínez, R. B. Orellana, A. R. Plastino and A. Plastino, Physica A 265 (1999) 590. [7] R. J. V. dos Santos, J. Math. Phys. 38 (1997) 4104. [8] C. Beck and F. Schlögl, Thermodynamics of Chaotic Systems: An Introduction (Cambridge University Press, Cambridge, 1993). [9] S. Abe and A. K. Rajagopal, “Nonadditive Conditional Entropy and Its Significance for Local Realism”, e-print (quant-ph/0001085). [10] M. Hotta and I. Joichi, Phys. Lett. A 262 (1999) 302.

14