FRACTALS LINDENMAYER SYSTEMS November 22, 2013 Rolf Pfeifer Rudolf M. Füchslin RECAP HIDDEN MARKOV MODELS What Letter Is Written Here? What Letter Is Written Here? What Letter Is Written Here? The Idea Behind Hidden Markov Models First letter: Maybe „a“, maybe „q“ Second letter: Maybe „r“ or „v“ or „u“ Take the most probable combination as a guess! Hidden Markov Models Sometimes, you don‘t see the states, but only a mapping of the states. A main task is then to derive, from the visible mapped sequence of states, the actual underlying sequence of „hidden“ states. HMM: A Fundamental Question What you see are the observables. But what are the actual states behind the observables? What is the most probable sequence of states leading to a given sequence of observations? The Viterbi-Algorithm We are looking for indices M1,M2,...MT, such that P(qM1,...qMT) = Pmax,T is maximal. 1. Initialization ()ib 1 i i k1 1(i ) 0 2. Recursion (1 t T-1) t1(j ) max( t ( i ) a i j ) b j k i t1 t1(j ) i : t ( i ) a i j max. 3. Termination Pimax,TT max( ( )) qmax,T q i: T ( i ) max. 4. Backtracking MMt t11() t Efficiency of the Viterbi Algorithm • The brute force approach takes O(TNT) steps. This is even for N = 2 and T = 100 difficult to do. • The Viterbi – algorithm in contrast takes only O(TN2) which is easy to do with todays computational means. Applications of HMM • Analysis of handwriting. • Speech analysis. • Construction of models for prediction. Only few processes are really Markov processes (neither writing nor speech is), but often, models based on Markov processes are good approximations. END RECAP FRACTALS Natural Geometry Geometry in text books Geometry in nature Fractals: Informal Definition • Termed coined by Benoit Mandelbrot • Geometry without smoothness Structure on all scales (detail persists when zoomed arbitrarily) • Geometrical objects generally with non-integer dimension • Self-similarity (contains infinite copies of itself) Fractals in the Human Body Kidney Lung Cortical surface (?) The Length of Borders • Lewis Fry Richardson: Probability of war between two adjacent countries proportional to length of border? • Checking the theory required gathering data about border lengths. • Surprising finding: There are strongly varying numbers in the literature. The Border of Great Britain The Border of Great Britain The closer you look, the longer the border. And the growth doesn‘t stop! A Slightly Different View on Dimension One-Dimensional Objects 1 Nc 1 : Diameter of disk N : Number of disks c1 : Constant Two-dimensional Objects N ? : Diameter of disk N : Number of disks c2 : Constant Two-dimensional Objects 2 1 Nc 2 : Diameter of disk N : Number of disks c2 : Constant Definition of the Fractal Dimension D 1 Nc The number of disks necessary for covering : Diameter of disk a structure grows with shrinking λ. N : Number of disks c : Constant D: Hausdorff Dimension log(N ( )) log( c ) log( N ( )) log( N ( )) D lim lim lim 011 0 0 log( ) log( ) log( ) Example: The Sierpinski Triangle Construction of the Sierpinski-triangle A Sierpinski triangle contains a whole copy of itself in its parts. Example: The Sierpinski Triangle Construction of the Sierpinski-triangle log(N ( )) D lim 0 log( ) Example: The Sierpinski Triangle Hausdorff Dimension: log(3n ) log(3) D lim 1.585 n 1 log( ) log(2) 2n Area of Sierpinski triangle 2 2 n 1 AN lim3 n 0 n 2 Boundary length of ST: 1 L3 KN lim3 K 3n n 2n Example: Cantor Dust Take out the middle third! log(N ( )) D lim 0 log( ) Example: Cantor Dust Take out the middle third! log(2n ) log(2) D lim n 1 log( ) log(3) 3n 1 L lim(no elements = 2n ) (length element = ) n 3n Example: The Mandelbrot Set Example: The Mandelbrot Set x x22 y a z z2 c n1 n n nn1 y2 x y b zc0, C n1 n n 0 z x iy, c a ib The Mandelbrot Set Three-Dimensional L-Systems Compressibility • The Mandelbrot looks complex. • The algorithm describing the Mandelbrot-set is very short. • Procedures generating fractal structures give a very compressed form of storing complex-looking shapes. • Directly storing these pictures is actually impossible. 2 znn1 z c zc0 0, C Self-Similarity and Scale-Invariance • “When each piece of a shape is geometrically similar to the whole, both the shape and the cascade that generate it are called self-similar.” (B. Mandelbrot) • Contains infinite copies of itself • Scaling/Scale = The value measured for a property does not depend on the resolution at which it is measured • Two types of invariance: - Geometrical - Statistical Fractals in Reality • Strict self-similarity is mostly found in mathematical examples. • Statistical interpretation of self-similarity: If a part of system can be zoomed up, and this part shows the same statistical properties as the whole system, one speaks of self-similarity. • In reverse (and more important), if coarse-graining does not change the statistical properties of a system, one speaks of self-similarity. Non-Fractal Random distribution of spheres with uniform radius. Fractal Random distribution of spheres with varying random radius (power law distribution). Self-Similarity and Coarse-Graining Coarse graining = or formation of blocks with averaged properties Self-Similarity in Reality ρ= 0.5 ρ= 0.55 SELF SIMILARITY ρ= 0.6 Physically important in the description of ρ= 0.7 phase transition. Fractal Dimension of Time Series Univ. Zürich: G. Wieser, P. F. Meier, Y. Shen, HR. Moser. R. Füchslin EEG EEG during epileptic seizure Some statistical measures such as the fractal correlation dimension D2 decrease shortly before and during an epiliptic seizure. D2 can be used as a diagnostic measure. Random Numbers • Question: Is a random number self similar? LINDENMAYER-SYSTEMS: STUDYING DEVELOPMENT USING FORMAL LANGUAGES A Real Puzzle • Nature is full of well-structured objects. • These objects are not assembled using global control and a blue-print, but emerge from local behavior. External control Self-organization Self-Assembly is Powerful, but …. Even if self-assembly processes may lead to non-trivial and finite structures with global shape and mesoscopic pattern induced by microscopic interactions, it is not the way how nature works. Paul. W. Rothemund Developmental Representation vs. Blueprint • All higher living organism develop from a fertilized egg (a zygote) into their adult form. This process is, at least to a large degree, controlled by their respective genome. • Is it that the genome contains a sort of "blueprint" of the organism? Developmental Representation vs. Blueprint • All higher living organism develop from a fertilized egg (a zygote) into their adult form. This process is, at least to a large degree, controlled by their respective genome. • It is NOT TRUE that the genome contains a sort of "blueprint" of the organism. Rather, the genome contains instructions which lead to molecules that in the interaction with the environment lead to organisms. Developmental Representation vs. Blueprint Developmental process is influence by: • An initial seed. • The (probably time-dependent) interactions of the building blocks of a body • The environment and the physical and chemical laws ruling this environment. Developmental representations are iterative in the sense that they tell you how to proceed if there is already something there. Developmental Representation vs. Blueprint The genome does not contain all the information it needs to build your body. Development requires embodiment! Instructions for construction = Developmental representation + Laws of the environment Formal Languages Are Not Enough • How to describe development by a formal system? • Problem: The languages we know do not necessarily lead to globally structured outcomes with repetitive patterns. • Reason: External decision of location where a replacement rule is applied. On Growth and Form: L-Systems • The patterns observed in multicellular algae are the result of developmental processes • Mathematical formalism introduced in 1968 by Aristid Lindenmayer. • Productions are rewriting rules which state how new symbols (or cells) can be produced from old symbols (or cells) L-Systems / Rewriting Systems • Lindenmayer systems belong to the general class of parallel grammars or parallel rewriting systems. • Difference to grammars as we know them: In a parallel rewriting system, rules are applied to all possible instances simultaneously. L-systems are subsets of languages. • Most practical L-systems are related to context-free languages. • Context-free grammars suit the emulation of maturation and division. A Context-free language AVV,() The Cantor Set as an L-System Non-Terminals (variables): AB, Terminals (constants) : none Start : A Rules : A ABA B BBB The Cantor Set as an L-System Non-Terminals (variables): AB, Terminals (constants) : none Start : A Rules : A ABA B BBB 1. A 2. ABA 3. ABABBBABA 4. ABABBBABABBBBBBBBBABABBBABA 5. ..... Anabaena Catenula A Bio-Inspired L-System Anabeana catenula: Two types of polar cells, photosynthesis and nitorgen fixation Variables : AABB, , , Constants : none Start : A Rules : A AB A BA BA BA Visualization: Turtle Graphics + - F Move forward by Rotate by angle δ Rotate by angle -δ distance F Bracket notation: [ : Store position and direction ] : Go back to position and direction of matching [ Visualization; Turtle Graphics Simple system: Axiom (Start): F Rule: F→F[-F][+F] Angle: 30° More Plants Professional Visualization Generalizations ? Generalizations • Stochasticity • Contex sensitive rules • Delay times • Reaction-diffusion systems Homology of Structure Potential Advantages of Development • Compact description • Supports symmetry • Supports modularity • Supports reuse of mechanisms in different contexts. • Supports scalability • Decentralized control by self-orgnaization and parallelism. • Turns out to be robust • Enables adaptivity • Change of structure can be achieved by small changes. • Change of structure by change of timing (heterochrony). • Evolvability .