The Categorical Origins of Entropy
Total Page:16
File Type:pdf, Size:1020Kb
The categorical origins of entropy Tom Leinster University of Edinburgh These slides: www.maths.ed.ac.uk/tl/ Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in science Entropy of one kind or another is important in very many branches of science: Entropy in category theory, algebra and topology The point of this talk Entropy is notable by its absence from category theory, algebra and topology. However, we will see that entropy is inevitable in pure mathematics: it is there whether we like it or not. p nq ∆ R Ó Ó CATEGORICAL MACHINE Ó Shannon entropy Image: J. Kock The point of this talk Entropy is notable by its absence from category theory, algebra and topology. However, we will see that entropy is inevitable in pure mathematics: it is there whether we like it or not. p nq ∆ R Ó Ó internal algebras in a categorical algebra for an operad Ó Shannon entropy Image: J. Kock Plan 1. The definition of entropy 2. Operads and their algebras 3. Internal algebras 4. The theorem 5. Low-tech corollary 1. The definition of entropy The definition of entropy p q t u Let p p1;:::; pn be a probability° distribution on 1;:::; n . P r sn That is, let p 0; 1 with i pi 1. The Shannon entropy of p is ¸n Hppq ¡ pi log pi i1 (where 0 log 0 0). It measures disorder, or information, or expected surprise, or uniformity,.... For a fixed n: the maximum entropy is Hp1{n; 1{n;:::; 1{nq log n the minimum entropy is Hp0;:::; 0; 1; 0;:::; 0q 0. Changing the base of the logarithm scales H by a constant factor. 2. Operads and their algebras The definition of operad A [symmetric] operad O is a sequence pO q P of sets n n N P together with: θ O3 (i) composition: for each k; n1;:::; nk P N, a function ¤¤¤ ¤¤¤ φ1 ¤ ¤ ¤ φn O ¢ O ¢ ¤ ¤ ¤ ¢ O ÝÑ O ¤¤¤ n k1 kn k1 kn ¤¤¤ 1 n 1 n pθ; φ ; : : : ; φ q ÞÝÑ θ ¥ pφ ; : : : ; φ q θ (ii) unit: an element1 P O1 [(iii) symmetry: for each n, an action of Sn on On] θ satisfying monoid-like axioms. Examples of operads 1. The terminal operad O 1 has On t¤u for all n. # M if n 1; 2. Given monoid M, get operad OpMq with pOpMqqn H otherwise: 3. The operad of simplices ∆, with ( n¡1 ∆n probability distributions on t1;:::; nu ∆ : Composition: given p pp ;:::; p q; q1 pq1;:::; q1 q;:::; qn pqn;:::; qn q; 1 n 1 k1 1 kn define p ¥ pq1;:::; qnq pp q1;:::; p q1 ;:::; p qn;:::; p qn q: 1 1 1 k1 n 1 n kn E.g.: p , q1 , q2 . 1 2 Then p ¥ pq ; q q plooooooomooooooon1{12;:::; 1{12; looooooooomooooooooon1{104;:::; 1{104q P ∆58. 6 52 Algebras for an operad Fix an operad O. An O-algebra is a set A together with a map θ : An ÝÑ A for each n P N and θ P On, satisfying action-like axioms: (i) composition, (ii) unit, [(iii) symmetry.] Examples: a. An algebra for the [symmetric] operad 1 is a [commutative] monoid. b. An OpMq-algebra is an M-set. d c. Let A R be a convex set. Then A becomes a ∆-algebra as follows: given p P ∆n, define n p: A ÝÑ ° A p 1 nq ÞÝÑ i a ;:::; a i pi a : Categorical algebras for an operad Fix an operad O. Extending the definition in the obvious way, we can consider O-algebras in any category A with finite products (not just Set). A categorical O-algebra is an O-algebra in Cat. Explicitly, it is a category A together with a functor θ : An ÝÑ A for each n P N and θ P On, satisfying action-like axoims. Examples: a. A categorical algebra for the nonsymmetric operad 1 is a strict monoidal category. b. A categorical OpMq-algebra is a category with an M-action. d c. Let A be a convex submonoid of pR ; ; 0q. Viewing A as a one-object category, it is a categorical ∆-algebra in the way defined above. Maps between categorical algebras for an operad Fix an operad O and categorical O-algebras B and A. A lax map B ÝÑ A is a functor G : B ÝÑ A together with a natural transformation G n Bn / An θ ðγθ θ B / A G for each n P N and θ P On, satisfying axioms. Explicitly: it's G together with a map 1 n 1 n γθ;b1;:::;bn : θpGb ;:::; Gb q ÝÑ Gpθpb ;:::; b qq 1 n for each θ P On and b ;:::; b P B, satisfying naturality and axioms on: (i) composition, (ii) unit, [(iii) symmetry.] 3. Internal algebras Internal algebras in a categorical algebra for an operad Fix an operad O and a categorical O-algebra A. Write 1 for the terminal categorical O-algebra. Definition (Batanin): An internal algebra in A is a lax map 1 ÝÑ A. Explicitly: it's an object a P A together with a map γθ : θpa;:::; aq ÝÑ a for each n P N and θ P On, satisfying axioms on (i) composition, (ii) unit, [(iii) symmetry.] Examples: a. Let O 1 (nonsymmetric). Let A be a strict monoidal category. An internal O-algebra in A is just a monoid in A. b. Let O OpMq. Let A be a category with an M-action. An internal O-algebra in A is an object a P A with a map γm : m ¤ a ÝÑ a for each m P M, satisfying action-like axioms. Internal algebras in a categorical algebra for an operad We fixed an operad O and a categorical O-algebra A. Consider the case where A has only one object, i.e. is a monoid A. An internal algebra in A then consists of a function γ : On ÝÑ A for each n P N, satisfying axioms on (i) composition, (ii) unit, [(iii) symmetry.] Topologizing everything Everything so far can be done internally to a category E with finite limits (instead of Set). So then, On P E , A is an internal category in E , etc. We take E Top. Explicitly, this means that throughout, we add a condition (iv) continuity to the conditions (i)|(iii) that appear repeatedly. 4. The theorem The theorem Recall: we have p q the (symmetric, topological) operad ∆ ∆n nPN of simplices the (symmetric, topological) categorical ∆-algebra R pR; ; 0q. We just saw that an internal algebra in the categorical ∆-algebra R consists of functions ∆n ÝÑ R (n P N) satisfying certain axioms. Theorem The internal algebras in the categorical ∆-algebra R are precisely the scalar multiples of Shannon entropy. The theorem Theorem The internal algebras in the categorical ∆-algebra R are precisely the scalar multiples of Shannon entropy. Explicitly, this says: take a sequence of functions γ : ∆n ÝÑ R (n P N). Then γ cH for some c P R if and only if γ satisfies: ¨ ° (i) composition: γ p ¥ pq1;:::; qnq γppq p γpqi q ¨ i i (ii) unit: γ p1q 0 ¨ ¨ (iii) symmetry: γ pp1;:::; pnq γ ppσp1q;:::; pσpnqq (σ P Sn) (iv) continuity: each function γ is continuous. Proof: This explicit form is equivalent to a 1956 theorem of Faddeev. l 5. Low-tech corollary (with John Baez and Tobias Fritz) The free categorical algebra containing an internal algebra Thought: A monoid in a monoidal category A is the same thing as a lax monoidal functor 1 ÝÑ A. But it's also the same as a strict monoidal functor D ÝÑ A, where D (finite ordinals) is the free monoidal category containing a monoid. We can try to imitate this for algebras for other operads, such as ∆. Fact: The free categorical ∆-algebra containing an internal algebra is (nearly) the category FinProb in which: an object pX ; pq is a finite set X with a probability measure p the maps are the measure-preserving maps (`deterministic processes'). Thus, an internal ∆-algebra in R is a functor FinProb ÝÑ pR; ; 0q satisfying certain axioms. An explicit characterization of entropy Corollary (with John Baez and Tobias Fritz) Let L: tmaps in FinProbuÑR be a function that `measures information loss', that is, satisfies: g Lpg ¥ f q Lpf q Lpgq ¤ ÝÑf ¤ ÝÑ ¤ 1 1 f Lpλf ` p1 ¡ λqf q λLpf q p1 ¡ λqLpf q : // : f 1 Lpf q 0 if f is invertible L is continuous. Then there is some c P R such that ¡ © ¨ L pX ; pq ÝÑf pY ; qq c ¤ Hppq ¡ Hpqq for all f . Summary Summary Given an operad O and an O-algebra A in Cat, there is a general concept of internal algebra in A.