Axioms and Uniqueness Theorem for Tsallis Entropy
Total Page:16
File Type:pdf, Size:1020Kb
Axioms and uniqueness theorem for Tsallis entropy Sumiyoshi Abe College of Science and Technology, Nihon University, Funabashi, Chiba 274-8501, Japan arXiv:cond-mat/0005538 31 May 2000 The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy and a complete proof of the uniqueness theorem for the Tsallis entropy is presented. This improves the discussion of dos Santos. PACS number: 05.20.-y 1 1. Introduction The concept of entropy is the fundamental ingredient in information theory and statistical mechanics. Given a probability distribution {}p satisfying i i=1 2,,L , W W 0≤p(,,,) i = 1 2 L W and ∑ p = 1, the Boltzmann-Shannon entropy, i.e., the i i=1 i ordinary information entropy, is defined by W ()L = − S p1, p 2 , , pW k∑ p i ln pi , (1) i=1 where k is a positive constant, which is henceforth set equal to unity for simplicity. This quantity is a positive and concave function of {}p . It also fulfills the H-theorem. i i=1 2,,L , W = Its maximum is realized by the equiprobability distribution, i.e., pi 1/ W (,,,) i= 1 2 L W and the value itself is SW= ln , which is the celebrated Boltzmann formula. To characterize what the entropy is, Shannon [1] and Khinchin [2] investigated its axiomatic foundation. The Shannon-Khinchin axioms are given as follows: ()L [] S p1,,, p 2 pW is continuous with respect to all its arguments and takes = its maximum for the equiprobability distribution pi 1/ W (,,,) i= 1 2 L W , [] SABSASBA[], = [] + [], ()LL= () [¡] S p1,,,,,,, p 2 pWW0 S p 1 p 2 p . In the second axiom, SAB[], and SA[] are the entropies of the composite system 2 () () =LL = AB, with the joint probability distribution p i j A,(,,,;,,,) B i1 2 W j 1 2 U U and the subsystem A with the marginal probability distribution p() A= ∑ p() A, B , ij =1 i j respectively. SBA[] stands for the conditional entropy associated with the conditional = () () probability distribution pi j() B A p i j A, B p i A . Writing the entropy of pi j () B A as SBA[]i , the conditional entropy is given by W = ()A = () SBA[] SBA[]i ∑ pASBAi[] i . (2) i=1 In the special case when A and B are statistically independent, SBASB[] = [], leading to the additivity: SABSASB[], = [] + []. (3) The uniqueness theorem [1,2] states that a quantity satisfying the axioms []-[¡] is uniquely equal to S in eq. (1). We note that there is a correspondence relation between the Bayes multiplication law and the axiom []: () = () ↔ = [] + pABi j,, pApBA i i j () SAB[] SA SBA[]. (4) Now, in the field of statistical mechanics, there is a growing interest in nonextensive generalization of Boltzmann-Gibbs theory. In particular, the one initiated by Tsallis [3-5] is receiving much attention. This formalism is based on the following single-parameter 3 generalization of the Boltzmann-Shannon entropy: W 1 q S() p,,, pL p = ∑ ()p −1 ()q > 0 . (5) q1 2 W− i 1 q i=1 → ≡ In the limit q 1, Sq converges to S in eq. (1) (with k 1). This quantity is also positive and concave, and satisfies the H-theorem. In addition, as the Boltzmann-Shannon = entropy, it takes its maximum for the equiprobability distribution pi 1/ W = L =() − −1 1−q − (,,,) i1 2 W : Its value is Sq 1 q() W 1 , which is a monotonically increasing function of W . However, the additivity is violated. For statistically independent systems, A and B, Sq yields = [] + [] + − [][] SABq[],() SA q SB q1 qSASB q q , (6) which is referred to as the pseudoadditivity. Clearly, the additivity holds only in the limit q → 1. Equation (6) has been carefully discussed in Ref. [6]. In a recent paper [7], dos Santos has made an interesting discussion about uniqueness of the Tsallis entropy in eq. (5). He has shown that that if a quantity is [‡] continuous with respect to the probability distribution {}p , [·] a monotonically increasing i i=1 2,,L , W = = L function of W in the case of the equiprobability distribution p i 1/(,,,) W i 1 2 W , and [] satisfies the pseudoadditivity in eq. (6) for the statistically independent systems, ()L = () + A and B, and [ ¶ ] fulfills the relation S q p1,,,, p 2 p W S q p L p M ()q ()LL+ ()q () = pSpppp L q1/,/,,//,/,,/ L 2 L pp W L pSpppp M q1 M 2 M pp W M , where pL WL = W ∑ = pi and pM∑ = + p i , then it is identical to the Tsallis entropy. i 1 i WL 1 4 Here, comparing the set of the Shannon-Khinchin axioms []-[¡] with that of [‡]- [¶], we find that the complete parallelism is missing between the two. The main reason for this is due to the absence of the concept of “nonextensive conditional entropy”. In this paper, we present the axioms for the Tsallis entropy by introducing the nonextensive conditional entropy. Then we prove the uniqueness theorem for the Tsallis entropy. Our proof essentially follows a natural generalization of the line presented in Ref. [2]. This improves the discussion given in Ref. [7] and establishes the complete parallelism with the Shannon-Khinchin axioms. 2. Nonextensive conditional entropy In nonextensive statistical mechanics, it is known [4] that the average of a physical quantity QQ= {} is given in terms of the normalized q-expectation value: i i=1 2,,L , W W ()q W ∑ Qi p i = ≡ i=1 QQPq ∑ i i W , (7) i=1 ()q ∑ pi i=1 W where P≡ ()() pq ∑ p q is the escort distribution associated with p [8]. To be i i i=1 i i consistent with the nonextensive formalism, we use this concept to generalize the definition in eq. (2). For this purpose, we calculate the Tsallis entropy of the conditional probability distribution 5 U 1 q SBA[] = ∑ []p() B A −1. (8) q i− i j 1 q j =1 From this quantity, we define the nonextensive conditional entropy as follows: W ∑ []p() Aq S[] B A ()A i q i = = i=1 SBASBAq[] q[] i W . (9) q () q ∑ []pi A i=1 Using the definition of the Tsallis entropy, we find that equation (9) can be expressed as SABSA[], − [] SBA[] = q q . (10) q + − [] 1() 1 q Sq A An important point here is that, with this definition, a natural nonextensive generalization of the correspondence relation in eq. (4) is established in conformity with the pseudoadditivity in eq. (6): () = () pi j A, B p i A p i j () B A (11) ↔ = [] + + − [] SABq[],(). SA q SBA q[] 1 qSASBA q q[] In fact, the pseudoadditivity in eq. (6) is recovered in the special case when A and B are statistically independent each other. 6 3. Axioms and uniqueness theorem for Tsallis entropy Now, the set of the axioms we present for the Tsallis entropy is the following: ()L []* S q p1,,, p 2 p W is continuous with respect to all its arguments = and takes its maximum for the equiprobability distribution pi 1/ W (,,,) i= 1 2 L W , = [] + + − [] []* SABq[],() SA q SBA q[] 1 qSASBA q q[], ()LL= () [¡]* S q p1,,,,,,, p 2 p W0 S q p 1 p 2 p W . Theorem: A quantity satisfying []*-[¡]* is uniquely equal to the Tsallis entropy. = = L Proof: First, let us consider the equiprobability distribution p i 1/(,,,) W i 1 2 W and put 1 1 1 S ,,,:L = LW(). (12) q WWW q From [¡]*, it follows that 1 1 1 LWS() = ,,,,L 0 q q WWW (13) 1 1 1 1 ≤ S ,,,,,L =LW() +1 q WWWW+1 +1 +1 +1 q () which means that LWq is a nondecreasing function of W . 7 L Consider m statistically independent systems, AAA 1,,, 2 m , each of which contains r ()≥ 2 equally likely events. Then, we have 1 1 1 SAS[] = ,,,L = L() r ()1 ≤k ≤ m . (14) q k q r r r q Using []* for these independent systems, we find m m k SAAA[],,,()L = ∑ 1 − qk −1[] L() r q1 2 m q k =1 k 1 m = {}[]1+() 1 − q L() r − 1 . (15) 1 − q q []L = ()m Since S q A1,,, A 2 A m L q r , we have 1 m L() r m = {}[]1+() 1 − q L() r − 1 . (16) q 1 − q q Similarly, for any other positive integers n and s equal to or larger than 2, we have 1 n L() s n = {}[]1+() 1 − q L() s − 1 . (17) q 1 − q q It is always possible to take m,,, r n and s which satisfy rm≤ s n ≤ r m+1 . (18) 8 Since Lq is a nondecreasing function, we have the inequalities m ≤ n ≤ m+1 Lq () r Lq () s Lq () r , (19) which lead to 1 m 1 n {}[]1+()() 1 − q L() r − 1 ≤ {}[]1+ 1 − q L() s − 1 1 − q q 1 − q q 1 m+1 ≤ {}[]1+() 1 − q L() r − 1 . (20) 1 − q q Here, it is necessary to examine two cases, 0<q < 1 and q > 1, separately. After simple algebra, we find that in both cases the following inequalities hold: + − () m ln[]1 ( 1 q ) Lq s m 1 ≤ ≤ + . (21) + − () n ln[]1 ( 1 q ) Lq r n n We note that, in deriving these inequalities, the following Ansatz has to be made: < + − () + − () < > 0 1(),() 1q Lq r 1 1 q L q s 1 for q 1. (22) Later, we shall see that this is in fact justified.