Invariance Defines Scaling Relations and Probability Patterns
Total Page:16
File Type:pdf, Size:1020Kb
UC Irvine UC Irvine Previously Published Works Title Common probability patterns arise from simple invariances Permalink https://escholarship.org/uc/item/1104s484 Journal Entropy, 18(5) ISSN 1099-4300 Author Frank, SA Publication Date 2016-05-01 DOI 10.3390/e18050192 License https://creativecommons.org/licenses/by/4.0/ 4.0 Peer reviewed eScholarship.org Powered by the California Digital Library University of California Common probability patterns arise from simple invariances Steven A. Frank1 Department of Ecology and Evolutionary Biology, University of California, Irvine, CA 92697{2525 USA Shift and stretch invariance lead to the exponential-Boltzmann probability distribution. Rotational invari- ance generates the Gaussian distribution. Particular scaling relations transform the canonical exponential and Gaussian patterns into the variety of commonly observed patterns. The scaling relations themselves arise from the fundamental invariances of shift, stretch, and rotation, plus a few additional invariances. Prior work described the three fundamental invariances as a consequence of the equilibrium canonical ensemble of statis- tical mechanics or the Jaynesian maximization of information entropy. By contrast, I emphasize the primacy and sufficiency of invariance alone to explain the commonly observed patterns. Primary invariance naturally creates the array of commonly observed scaling relations and associated probability patterns, whereas the classical approaches derived from statistical mechanics or information theory require special assumptions to derive commonly observed scales. 1. Introduction 2 9.1. Statistical mechanics 10 9.2. Jaynesian maximum entropy 10 2. Background 3 9.3. Conclusion 11 2.1. Probability increments 3 2.2. Parametric scaling relations 3 10. References 11 3. Shift invariance and the exponential form 3 Appendix A: Technical issues and extensions 12 3.1. Conserved total probability 3 1. Conserved total probability 12 3.2. Shift-invariant canonical coordinates 4 2. Conserved average values: eqn 7 12 3.3. Entropy as a consequence of shift invariance 4 3. Primacy of invariance 12 3.4. Example: the gamma distribution 4 4. Measurement theory 13 4. Stretch invariance and average values 5 Appendix B: Invariance and the common 4.1. Conserved average values 5 canonical scales 13 4.2. Alternative measures 5 5. Rotational invariance of the base scale 13 4.3. Example: the gamma distribution 5 6. General form of base scale invariance 13 7. Example: linear-log invariance of the base scale 14 5. Consequences of shift and stretch invariance 5 8. The family of canonical scales 14 5.1. Relation between alternative measures 5 9. Example: extreme values 15 5.2. Entropy 5 10. Example: stretched exponential and L´evy 15 5.3. Cumulative measure 6 5.4. Affine invariance and the common scales 6 6. Rotational invariance and the Gaussian radial measure 6 6.1. Gaussian distribution 7 6.2. Radial shift and stretch invariance 7 6.3. Transforming distributions to canonical Gaussian form 7 arXiv:1602.03559v2 [math.PR] 1 Apr 2016 6.4. Example: the gamma distribution 7 6.5. Example: the beta distribution 8 7. Rotational invariance and partitions 8 7.1. Overview 8 7.2. Rotational invariance of conserved probability 8 7.3. Partition of the canonical scale 9 7.4. Partition into location and rate 9 8. Summary of invariances 9 9. The primacy of invariance and symmetry 10 a)web: http://stevefrank.org git • master @ arXiv2-0-e7d119f-2016-04-01 (2016-04-05 00:57Z) • safrank 2 It is increasingly clear that the symmetry [invari- What is primary in the relation between these two ance] group of nature is the deepest thing that we equations: equilibrium statistical mechanics or shift in- understand about nature today. I would like to variance? The perspective of statistical mechanics, with suggest something here that I am not really cer- eqn 2 as the primary equilibrium outcome, dominates treatises of physics. tain about but which is at least a possibility: that Jaynes4,5 questioned whether statistical mechanics is specifying the symmetry group of nature may be sufficient to explain why patterns of nature often follow all we need to say about the physical world, be- the form of eqn 2. Jaynes emphasized that the same yond the principles of quantum mechanics. probability pattern often arises in situations for which The paradigm for symmetries of nature is of physical theories of particle dynamics make little sense. In Jaynes' view, if most patterns in economics, biology, course the group symmetries of space and time. and other disciplines follow the same distributional form, These are symmetries that tell you that the laws then that form must arise from principles that transcend of nature don't care about how you orient your the original physical interpretations of particles, energy, laboratory, or where you locate your laboratory, and statistical mechanics6. or how you set your clocks or how fast your lab- Jaynes argued that probability patterns derive from oratory is moving (Weinberg 1, p. 73). the inevitable tendency for systems to lose information. By that view, the equilibrium form expresses minimum information, or maximum entropy, subject to whatever For the description of processes taking place in constraints may act in particular situations. In maximum nature, one must have a system of reference (Lan- entropy, the shift invariance of the equilibrium distribu- dau and Lifshitz 2, p. 1). tion is a consequence of the maximum loss of information under the constraint that total probability is conserved. Here, I take the view that shift invariance is pri- 1 INTRODUCTION mary. My argument is that shift invariance and the conservation of total probability lead to the exponential- I argue that three simple invariances dominate much of Boltzmann form of probability distributions, without the observed pattern. First, probability patterns arise from need to invoke Boltzmann's equilibrium statistical me- invariance to a shift in scaled measurements. Second, chanics or Jaynes' maximization of entropy. Those sec- the scaling of measurements satisfies invariance to uni- ondary special cases of Boltzmann and Jaynes follow from form stretch. Third, commonly observed scales are often primary shift invariance and the conservation of proba- invariant to rotation. bility. The first part of this article develops the primacy Feynman3 described the shift invariant form of proba- of shift invariance. bility patterns as Once one adopts the primacy of shift invariance, one is faced with the interpretation of the measurement scale, q (E) q (E + a) E. We must abandon energy, because we have discarded = ; (1) q E0 q E0 + a the primacy of statistical mechanics, and we must aban- don Jaynes' information, because we have assumed that in which q (E) is the probability associated with a mea- we have only general invariances as our basis. surement, E. Here, the ratio of probabilities for two dif- We can of course end up with notions of energy and ferent measurements, E and E0, is invariant to a shift by information that derive from underlying invariance. But a. Feynman derived this invariant ratio as a consequence that leaves open the problem of how to define the canon- of Boltzmann's equilibrium distribution of energy levels, ical scale, E, that sets the frame of reference for measure- E, that follows from statistical mechanics ment. We must replace the scaling relation E in the above q (E) = λe−λE : (2) equations by something that derives from deeper gener- ality: the invariances that define the commonly observed Here, λ = 1= hEi is the inverse of the average measure- scaling relations. ment. In essence, we start with an underlying scale for obser- Feynman presented the second equation as primary, vation, z. We then ask what transformed scale, z 7! Tz ≡ arising as the equilibrium from the underlying dynamics E, achieves the requisite shift invariance of probability of particles and the consequent distribution of energy, E. pattern, arising from the invariance of total probability. He then mentioned in a footnote that the first equation It must be that shift transformations, Tz 7! a+Tz, leave of shift invariance follows as a property of equilibrium. the probability pattern invariant, apart from a constant However, one could take the first equation of shift invari- of proportionality. ance as primary. The second equation for the form of the Next, we note that a stretch of the scale, Tz 7! bTz, probability distribution then follows as a consequence of also leaves the probability pattern unchanged, because shift invariance. the inverse of the average value in eqn 2 becomes λ = git • master @ arXiv2-0-e7d119f-2016-04-01 (2016-04-05 00:57Z) • safrank 3 1=b hTzi, which cancels the stretch in the term λE = λTz. 2.2 Parametric scaling relations Thus, the scale Tz has the property that the associated probability pattern is invariant to the affine transforma- A probability pattern, qzd z, may be considered as a tion of shift and stretch, T 7! a + bT . That affine z z parametric description of two scaling relations, qz and z, invariance generates the symmetry group of scaling rela- with respect to the parameter z. Geometrically, qzd z tions that determine the commonly observed probability is a rectangular area defined by the parametric height, patterns7{9. qz, with respect to the parameter, z, and the parametric The final part of this article develops rotational in- width, d z, with respect to the parameter, z. variance of conserved partitions. For example, the We may think of z as a parameter that defines a curve Pythagorean partition T = x2(s)+y2(s) splits the scaled z along the path ( z; qz), relating a scaled input measure, measurement into components that add invariantly to T z z, to a scaled output probability, qz. The followings sec- for any value of s. The invariant quantity definesp a circle tions describe how different invariances constrain these in the xy plane with a conserved radius Rz = Tz that scaling relations.