Lecture 4. Entropy and Markov Chains
Total Page:16
File Type:pdf, Size:1020Kb
preliminary version : Not for di®usion Lecture 4. Entropy and Markov Chains The most important numerical invariant related to the orbit growth in topological dynamical systems is topological entropy.1 It represents the exponential growth rate of the number of orbit segments which are distinguishable with an arbitrarily high but ¯nite precision. Of course, topological entropy is invariant by topological conjugacy. For measurable dynamical systems, an entropy can be de¯ned using the invariant measure. It gives an indication of the amount of randomness or complexity of the system. The relation between measure{theoretical entropy and topological entropy is given by a variational principle. 4.1 Topological Entropy We will follow the de¯nition given by Rufus Bowen in [Bo, Chapter 4]. Let X be a compact metric space. De¯nition 4.1 Let S ½ X, n 2 N and " > 0. S is a (n; "){spanning set if for every x 2 X there exists y 2 S such that d(f j(x); f j(y)) · " for all 0 · j · n. It is immediate to check that the compactness of X implies the existence of ¯nite spanning sets. Let r(n; ") be the least number of points in an (n; "){spanning set. If we bound the time of observation of our dynamical system by n and our precision in making measurements is " we will see at most r(n; ") orbits. Exercise 4.2 Show that if X admits a cover by m sets of diameter · " then r(n; ") · mn+1. De¯nition 4.3 The topological entropy htop(f) of f is given by 1 htop(f) = lim lim sup log r(n; ") : [4:1] "!0 n!+1 n In the previous de¯nition one cannot replace lim sup with lim since there exist examples of maps for which the limit does not exist. However one can replace it with lim inf still obtaining the topological entropy (see [Mn1], Proposition 7.1, p. 237). Exercise 4.4 Show that the topological entropy for any di®eomorphism of a compact manifold is always ¯nite. 2 ¡i Exercise 4.5 Let X = fx 2 ` (N) ; jxij < 2 for all i 2 Ng, f((xi)i2N) = ¡1 n (2xi+1)i2N. Let k 2 N. Show that for this system r(n; k ) > k thus htop(f) = 1. 1 According to Roy Adler [BKS, p. 103] \topological entropy was ¯rst de¯ned by C. Shannon [Sh] and called by him noiseless channel capacity. preliminary version ! Exercise 4.6 Show that the topological entropy of the p{adic map of Exercise 2.36 is log p. Remark 4.7 The topological entropy of a flow 't is de¯ned as the topological entropy of the time{one di®eomorphism f = '1. Exercise 4.8 Show that : (i) the topological entropy of an isometry is zero ; if h is an isometry the topological entropy of f equals that of h¡1 ± f ± h. ¡1 (ii) if f is a homeomorphism of a compact space X then htop(f) = htop(f ); m (iii) htop(f ) = jmjhtop(f). Exercise 4.9 Let X be a metric space and f a continuous endomorphism of X. We say that a set A is (n; "){separated if for all x; y 2 X there exists a 0 · j · n such that d(f j(x); f j(y)) > ". We denote s(n; ") the maximal cardinality of an (n; "){separated set. Show that : (i) s(n; 2") · r(n; ") · s(n; "); 1 (ii) htop(f) = lim"!0 lim supn!+1 n log s(n"); (iii) if X is a compact subset of Rl and f is Lipschitz with Lipschitz constant K then htop(f) · l log K. Proposition 4.10 The topological entropy does not depend on the choice of the metric on X provided that the induced topology is the same. The topological entropy is invariant by topological conjugacy. Proof. We ¯rst show how the second statement is a consequence of the ¯rst. Let f; g be topologically conjugate via a homeomorphism h. Let d denote a ¯xed metric 0 0 ¡1 ¡1 on X and d denote the pullback of d via h : d (x1; x2) = d(h (x1); h (x2)). Then h becomes an isometry so htop(f) = htop(g) (see Exercise 4.8). Let us now show the ¯rst part. Let d and d0 be two di®erent metrics on X which induce the same topology and let rd(n; ") and rd0 (n; ") denote the minimal cardinality of a (n; ")-spanning set in the two metrics. We will denote htop;d(f) and htop;d0 (f) the corresponding topological entropies. Let " > 0 and consider the set D" of all pairs (x1; x2) 2 X £ X such that 0 d(x1; x2) ¸ ". This is a compact subset of X £ X thus d takes a minimum 0 0 0 ± (") > 0 on D". Thus any ± ("){ball in the metric d is contained in a "{ball in 0 the metric d. From this one gets rd0 (n; ± (")) ¸ rd(n; ") thus htop;d0 (f) ¸ htop;d(f). Interchanging the role of the two metrics one obtains the opposite inequality. ¤ Exercise 4.11 Show that if g is a factor of f then htop(g) · htop(f). An alternative but equivalent de¯nition of topological entropy is obtained consid- ering all possible open covers of X and their re¯nements obtained by iterating f. 2 C. Carminati and S. Marmi { An Introduction to Dynamical Systems De¯nition 4.12 If ®; ¯ are open covers of X theire join ® _ ¯ is the open cover by all sets of the form A \ B, where A 2 ® and B 2 ¯. An open cover ¯ is a re¯nement of ®, written ® < ¯, if every member of ¯ is a subset of a member of ®. Let ® be an open cover of X and let N(®) be the number of sets in a ¯nite subcover of ® with smallest cardinality. We denote f ¡1® the open cover consisting of all sets f ¡1(A) where A 2 ®. Exercise 4.13 If fangn2N is a sequence of real numbers such that an+m · an +am for all n; m then limn!+1 an=n exists and equals infn2N an=n. [Hint : n = kp+m, an ap am n · p + kp .] Theorem 4.14 The topological entropy of f is given by à ! n¡1 _ 1 ¡i htop(f) = sup lim log N f ® : [4:2] ® n!1 n i=0 For its proof see [Wa, pp. 173-174]. 4.2 Entropy and information. Metric entropy. In order to de¯ne metric entropy and to make clear its analogy with the formula [4.2] of topological entropy we will preliminarly introduce some general consider- ations on the relationship between entropy and information (see [Khi]). Suppose that one performs an experiment which we will denote ® which has m 2 N possible mutually esclusive outcomes A1;:::;Am (e.g. throwing a coin m = 2 or a dice m = 6). Assume that each possible outcome Ai happens with a Pm probability pi 2 [0; 1], i=1 pi = 1 (in an experimental situation the probability will be de¯ned statistically). In a probability space (X; A; ¹) this corresponds to the following setting : ® is a ¯nite partition X = A1 [ ::: [ Am mod(0), Ai 2 A, ¹(Ai \ Aj) = 0, ¹(Ai) = pi. We want to de¯ne a function (called entropy) which measures the uncertainity associated to a prediction of the result of the experiment (or, equivalently, which measures the amount of information which one can gain from performing the experiment). Let ¢(m) denote the standard m-simplex of Rm, Xm (m) m ¢ = f(x1; : : : ; xm) 2 R j xi 2 [0; 1]; xi = 1g: i=1 De¯nition 4.15A continuous function H(m) : ¢(m) ! [0; +1] is called an entropy if it has the following properties : 3 preliminary version ! (m) (1) symmetry : 8 i; j 2 f1; : : : ; mg H (p1; : : : ; pi; : : : ; pj; : : : ; pm) = H(p1; : : : ; pj; : : : ; pi; : : : ; pm) ; (2) H(m)(1; 0;:::; 0) = 0 ; (m) (m¡1) (3) H (0; p2; : : : ; pm) = H (p2; : : : ; pm) 8 m ¸ 2, 8 (p2; : : : ; pm) 2 (m¡1) ¢ ; ¡ ¢ (m) (m) (m) 1 1 (4) 8 (p1; : : : ; pm) 2 ¢ one has H (p1; : : : ; pm) · H m ;:::; m where 1 equality is possible if and only if pi = m for all i = 1; : : : ; m ; (ml) (5) Let (¼11; : : : ; ¼1l; ¼21; : : : ; ¼2l; : : : ; ¼m1; : : : ; ¼ml) 2 ¢ ; for all (p1; : : : ; pm) 2 ¢(m) one must have (ml) (m) H (¼1l; : : : ; ¼1l; ¼21; : : : ; ¼ml) =H (p1; : : : ; pm)+ µ ¶ Xm ¼ ¼ + p H(l) i1 ;:::; il : i p p i=1 i i In the above de¯nition : (2) says that if some outcome is certain then the entropy is zero ; (3) says that no information is gained from impossible outcomes (i.e. outcomes with probability zero) ; (4) says that the maximal uncertainity of the outcome is obtained when the possible results have the same probabilitly ; (5) describes the behaviour of entropy when independent distinct experiences are performed. Let ¯ denote another experiment with possible outcomes B1;:::;Bl (i.e. another partition of (X; A; ¹)). Let ¼ij be the probablility ¼ij of Ai and Bj. The conditional probability of Bj is prob (Bj j Ai) = (i.e. pi ¹(Ai \Bj)). Clearly the uncertainity of the outcome of the experiment³ ¯ once´ (l) ¼i1 ¼il one has already performed ® with outcome Ai is given by H ;:::; . pi pi Theorem 4.16 An entropy is necessarily a positive multiple of Xm H(p1; : : : ; pm) = ¡ pi log pi : [4:3] i=1 Here we adopt the convention 0 log 0 = 0. The above theorem and its proof are taken from [Khi, pp.