arXiv:math-ph/0505078v1 30 May 2005 PACS: e words: e Key Tsallis for generalization pseudo-a no the exists preserves which there averages, that Kolmogorov-Nagumo show we per omgrvNgm vrgs yimposing Shan by generalized averages, [1] R´enyiKolmogorov-Nagumo in entropy. Tsallis of property eti ls fpyia ytm hteti ogrneinteract systems. long-range multi-fractal entail and pr that memories thermostatistical systems physical describing of in class measure certain useful a as considered 3 2 1 ∗ R´enyi int literature. extensive as known an measure, inform having of generalized now measures generalized R´enyi [1,2], of Alfred theory to the of point starting The form classical the the of pseudo-additivity of entropy, property non characteristic a is additivity As Abstract nteohrhn,hwvr sli n[]pooe non-logarithmic a as proposed known [3] measure, in entropic Tsallis of however, alization hand, other the On p by ge and the averages with Kolmogorov-Nagumo entropy additivity particular in in averaging averages, linear ized replacing by derived is en fKlooo-auoaeae under averages Kolmogorov-Nagumo of means rpitsbitdt leirSine2Otbr2018 October 2 Science Elsevier to submitted Preprint eateto optrSineadAtmto,Ida Institu Indian Automation, and Science Computer of Department [email protected] (Tel:+91-80-22932779) [email protected] [email protected] author corresponding ognrlzblt fTalsEtoyby Entropy Tsallis of Nongeneralizability 54.r 97.c 02.70.Rr 89.70.+c, 65.40.Gr, fteifrainmeasures. information the of mekrDukkipati, Ambedkar omgrvNgm vrgs ´nietoy sli entr Tsallis R´enyi entropy, averages, Kolmogorov-Nagumo pseudo-additivity hlb Bhatnagar Shalabh aglr-602 India. Bangalore-560012, 1 .Nrih Murty, Narsimha M. x + q etoyo sli nrp hc is which entropy Tsallis or -entropy q additivity y α etoyo R´enyi which entropy, or -entropy = x + 3 y +(1 sacntan.I hspa- this In constraint. a as nomto esr,Shan- measure, information o nrp ymasof means by entropy non − q dditivity. ) toy ymasof means by ntropy, xy sacharacteristic a is os long-term ions, eo Science, of te ∗ priso a of operties , to sdue is ation 2 opy sn the osing roduced gener- neral- Tsallis and R´enyi entropy measures are two possible different generalization of the Shannon entropy but are not generalizations of each other.

To understand these generalizations, the so called Hartley information mea- sure [4], of a single stochastic event plays a fundamental role. It can be inter- preted either as a measure of how unexpected the event is, or as a measure of the information yielded by the event. In a system with the finite configu- n ration space x = {xk}k=1, Hartley information measure of a single event with probability pk is defined as 1 H(pk) = ln , k =1, . . . n . (1) pk

Hartley information measure satisfies: (1) H is nonnegative: H(pk) ≥ 0 (2) H 1 is additive: H(pipj)= H(pi)+ H(pj) (3) H is normalized: H( 2 ) = 1. These are both necessary and sufficient [5]. Hartley information measure can be viewed as a random variable and hence we use the notation H =(H1,...Hn).

Shannon entropy is defined as an average Hartley information: [6]

n n S(p)= hHi = pkHk = − pk ln pk . (2) kX=1 kX=1

The characteristic property of Shannon entropy is additivity, i.e., for two in- dependent probability distributions p and r we have

S(pr)= S(p)+ S(r) , (3) where pr is the joint distribution of p and r.

R´enyi in [1,2] used a well known idea in mathematics that the linear mean, though most widely used, is not the only possible way of averaging, but one can define the mean with respect to an arbitrary function [7,8] to general- ize the Shannon entropy. In the general theory of means, a mean of x = (x1, x2,...,xn) with respect to a probability distribution p = (p1,p2,...,pn) is defined as [7]

W −1 hxiψ = ψ pkψ (xk) , (4) " # kX=1 where ψ is continuous and strictly monotonic (increasing or decreasing) in which case it has an inverse ψ−1 which satisfies the same conditions; ψ is generally called the Kolmogorov-Nagumo function associated with the mean

2 (4) 4 . If, in particular, ψ is linear, then (4) reduces to the expression of linear n averaging, hxi = k=1 pkxk. The mean of form (4) is also referred as quasi- linear mean. P In the definition of Shannon entropy (2), if the linear average of Hartley infor- mation is replaced with the generalized average of the form (4), the information corresponding to the probability distribution p with respect to KN-function ψ will be n n −1 1 −1 Sψ(p)= ψ pkψ ln = ψ pkψ (Hk) , (5) " pk !# " # kX=1 kX=1 where H =(H1,...Hn) is the Hartley information measure associated with p.

If we impose the constraint of additivity in Sψ, then ψ should satisfy [1]

hx + Ciψ = hxiψ + C, (6)

for any x =(x1,...,xn) and a constant C.

R´enyi employed the above formalism to define an one-parameter family of measures of information (α-entropies)

n 1 α Sα = ln pk , (7) 1 − α ! kX=1 where the KN-function ψ is chosen in (5) as ψ(x)= e(1−α)x, choice motivated by well known theorem in the theory of means (Theorem 89, [7]) that (6) can hold only for linear and exponential functions. R´enyi entropy is an one- parameter generalization of Shannon entropy in the sense that, the limit α → 1 in (7) retrieves Shannon entropy.

On the other hand, Tsallis entropy is given by [3]

n q 1 − k=1 pk Sq(p)= , (8) qP− 1 where q is called nonextensive index (q is positive in order to ensure the con- cavity of Sq). Tsallis entropy too, like R´enyi entropy, is an one-parameter 4 A. N. Kolmogorov [9] and M. Nagumo [10] were the first to investigate the charac- teristic properties of general means. They considered only the case of equal weights; the generalization to arbitrary weights and the characterization of means of form (4) are due to B. de Finetti [11], B. Jessen [12], T. Kitagawa [13], J. Acz´el [8] and many others

3 generalization of Shannon entropy in the sense that q → 1 in (8) retrieves Shannon entropy. The entropic index q characterizes the degree of nonexten- sivity reflected in the pseudo-additivity property

Sq(pr)= Sq(p)+q Sq(r)= Sq(p)+ Sq(r)+(1 − q)Sq(p)Sq(r) , (9) where p and r are independent probability distributions.

Though the derivation of Tsallis entropy, when it was proposed in 1988 is slightly different, one can understand this generalization using q- (see 11) function: where one would first generalize, logarithm in the Hartley information with q-logarithm and define q-Hartley information measure H = (H1,..., Hn) as f f f 1 Hk = H(pk) = lnq , k =1, . . . n , (10) pk f f where q-logarithm is defined as x1−q − 1 ln (x)= , (11) q 1 − q

which satisfies pseudo-additivity lnq(xy) = lnq x+q lnq y and in the limit q → 1 we have lnq → ln x. Tsallis entropy (8) defined as the average of q-Hartley information i.e [14]:

1 Sq(p)= H = lnq . (12) * pk + D E f Now a natural question arises whether one could generalize Tsallis entropy in the similar lines of derivation of R´enyi entropy i.e., by replacing linear average in (12) by KN-averages under the pseudo-additivity.

The class of information measures that represent the KN-average of q-Hartley information measure is written as

W W 1 −1 1 −1 Sψ(p)= lnq = ψ pkψ lnq = ψ pkψ Hk .(13) * pk + " pk !# " # ψ kX=1 kX=1   e f By the pseudo-additivity constraint, ψ should satisfy

Sψ(pr)= Sψ(p)+q Sψ(r) (14)

e e e or

4 n n −1 1 ψ pirjψ lnq  pirj ! Xi=1 jX=1  n  n −1 1 −1 1 = ψ piψ lnq +q ψ rjψ lnq , (15) " pi !#  rj ! Xi=1 jX=1   where p and r are independent probability distributions and pr denotes the joint probability distribution of p and r.

Equivalently, we need

n n ψ−1 p r ψ Hp + Hr  i j i q j  i j X=1 X=1    n f f  n = ψ−1 p ψ Hp + ψ−1 r ψ Hr , (16) i i q  j j  "i # j X=1   X=1   f  f  whereHp and Hr represents the q-Hartley information of probability distribu- tions p and r respectively. f f Note that (16) must hold for arbitrary finite discrete probability distributions p r r pi and rj and for arbitrary numbers Hk and Hk . If we choose Hk = J inde- pendently of j then (16) yields that f f f n n −1 p −1 p ψ pkψ Hk +q J = ψ pkψ Hk +q J (17) " # " # kX=1   kX=1   f f In general ψ satisfies (17) only if ψ satisfies

hx +q Ciψ = hxiψ +q C, (18)

for any x =(x1,...,xn), which can be rearranged as

hx + C + (1 − q)xCiψ = hxiψ + C + (1 − q)hxiψC (19) or

h(1+(1 − q)C)x + Ciψ = (1+(1 − q)C)hxiψ + C. (20)

Since q is independent of other quantities, ψ should satisfy the equation of form (By B = (1+(1 − q)C))

hBx + Ciψ = Bhxiψ + C. (21)

5 Finally ψ must satisfy

hx + Ciψ = hxiψ + C (22) and

hBxiψ = Bhxiψ , (23)

for any x = (x1,...,xn) and for any constants B and C, for Sψ to preserve the pseudo-additivity. e From the generalized theory of means (22) is satisfied only when ψ is linear or exponential, but the requirement (23) is satisfied only when ψ is linear and it is not satisfied when ψ is exponential.

Hence ψ is linear in which case (13) is nothing but Tsallis entropy.

This establishes the nongeneralizability of Tsallis entropy by means of KN- averages under pseudo-additivity.

References

[1] A. R´enyi, Some fundamental questions of , MTA III. Oszt. K¨ozl. 10 (1960) 251–282, (reprinted in [15], pp. 256-552).

[2] A. R´enyi, On measures of entropy and information, in: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley-Los Angeles, 1961, pp. 547–561, (reprinted in [15], pp. 565-580).

[3] C. Tsallis, Possible generalization of Boltzmann Gibbs statistics, J. Stat. Phys. 52 (1988) 479.

[4] R. V. L. Hartley, Transmission of information, Bell System Technical Journal 7 (1928) 535.

[5] J. Aczel, Z. Daroczy, On Measures of Information and Their Characterization, Academic Press, New York, 1975.

[6] C. E. Shannon, A mathematical theory of communication, Bell System Technical Journal 27 (1948) 379.

[7] G. H. Hardy, J. E. Littlewood, G. P´olya, Inequalities, Cambridge, 1934.

[8] J. Acz´el, On mean values, Bull. Amer. Math. Soc. 54 (1948) 392–400.

6 [9] A. Kolmogorov, Sur la notion de la moyenne, Atti della R. Accademia Nazionale dei Lincei 12 (1930) 388–391.

[10] M. Nagumo, Uber¨ eine klasse von mittelwerte, Japanese Journal of Mathematics 7 (1930) 71–79.

[11] B. de Finetti, Sul concetto di media, Giornale di Istituto Italiano dei Attuarii 2 (1931) 369–396.

[12] B. Jessen, Uber¨ die verallgemeinerung des arthmetischen mittels, Acta Sci. Math. 5 (1931) 108–116.

[13] T. Kitagawa, On some class of weighted means, Proceedings Physico- Mathematical Society of Japan 16 (1934) 117–126.

[14] C. Tsallis, F. Baldovin, R. Cerbino, P. Pierobon, Introduction to nonextensive statistical mechanics and thermodynamics, in: F. Mallamace, H. E. Stanley (Eds.), The Physics of Complex Systems: New Advances and Perspectives, Vol. 155, Enrico Fermi International School of Physics, 2003.

[15] P. Tur´an (Ed.), Selected Papers of Alfr´ed R´enyi, Akademia Kiado, Budapest, 1976.

7