The El Farol Bar Problem for next generation systems Athanasios Papakonstantinou Dissertation submitted for the MSc in Mathematics with Modern Applications Department of Mathematics August 2006 Supervisor Dr. Maziar Nekovee, BT Research Contents 1. Chaos, Complexity and Irish Music 1 1.1. Overview ................................ 1 1.2. The El Farol Bar Problem (EFBP) ................. 4 1.3. Modelling the original problem ................... 5 1.4. Game Theory Definitions ....................... 7 2. Previous Approaches to the El Farol 9 2.1. Overview ................................ 9 2.2. Approaches to EFBP ......................... 9 2.2.1. Minority Game ........................ 9 2.2.2. Evolutionary Learning .................... 11 2.2.3. Stochastic Adaptive Learning . 13 3. Analysis and Extension of the Stochastic Algorithm 23 3.1. Overview ................................ 23 3.2. Taxing/Payoffs Algorithms ...................... 23 3.2.1. Overview ........................... 23 3.2.2. Fairness and Efficiency .................... 29 3.3. Budget Algorithms .......................... 31 3.3.1. Overview ........................... 31 3.3.2. Fairness and Efficiency .................... 34 4. Case Study: Multiple Bars in Santa Fe 37 5. Conclusions 43 A. C Code for the El Farol Bar Problem 47 ii List of Figures 1.1. Bar attendance in the first 100 weeks [1]. .............. 5 2.1. Behaviour of the average attendance (Top) and of the fluctuations (bottom) in the El Farol problem with L = 60 seats,a ¯ = 1/2 and m = 2, 3, 6 from left to right [2]. ................... 10 2.2. Mean weekly attendance for all 300 trials[3]. 12 2.3. The attendance in a typical trial[3]. 12 2.4. The normalised one-step transition matrix[3]. 13 2.5. The overall attendance and the probabilities for each of the M agents for ‘partial information’, ‘full information’ and ‘signs’ algo- rithms. ................................. 17 2.6. histograms for partial and full info algorithms. 18 2.7. fairness and efficiency plots for original algorithms. 20 2.8. standard deviation off attendance for original algorithms. 21 3.1. histograms for partial and full info taxing algorithms. 25 3.2. The overall attendance and the probabilities for each of the M agents for ’partial’ information tax, modified tax algorithms and the full information tax algorithm. 26 3.3. fairness and efficiency plots for taxing algorithms. 30 3.4. standard deviation off attendance for taxing algorithms. 31 3.5. histograms for budget algorithms (b = 10, w = 0/6). 32 3.6. The overall attendance and the probabilities for each of the M agents for budget (budget= 10) and modified budget (budget= 10, wait=3, 6) algorithms. ........................ 33 3.7. standard deviation off attendance for budget algorithms. 34 3.8. fairness and efficiency plots for budget algorithms. 35 4.1. The overall attendance and the probabilities for each of the M agents for each bar. .......................... 38 4.2. The cumulative attendance for all three bars. 39 4.3. Agent attendances for each bar ................... 40 4.4. payoff probabilities for each bar and cumulative payoff probability for the 3-bar version ......................... 41 iv List of Tables 1.1. The Prisoner’s Dilemma in matrix form with utility pay-offs. 8 2.1. Behaviour of the ‘partial information’ algorithm. 16 2.2. Behaviour of the ‘full information’ algorithm. 16 2.3. Behaviour of the ‘signs’ algorithm. 18 2.4. standard deviation and mean of payoff probabilities for original algorithms. .............................. 19 3.1. Behaviour of the ‘tax’ algorithm with ctax = 6. 24 3.2. Simulation results for various values of ctax, cp. 25 3.3. standard deviation and mean of payoff probabilities for taxing al- gorithms. ............................... 29 3.4. Behaviour of the ‘modified budget’ algorithm for budget=10 and staying at home=6. .......................... 32 3.5. Simulation results for budget algorithms. 32 3.6. standard deviation and mean of payoff probabilities for budget algorithms. .............................. 34 4.1. mean and std for all bars. ...................... 39 vi Acknowledgements I would like to dedicate this dissertation to my parents Costantinos and Valentini Papakonstantinou . I want to thank them for giving me the opportunity to study in the UK and for always supporting me in all my decisions. Special thanks to my supervisor in BT Research, Maziar Nekovee, for his help and guidance. He was a very valuable source of information and always available when I needed him. Also Keith Briggs in BT Research was very helpful and this is very much appreciated. Futhermore, I would like to thank all the friends in York and in Ipswich for this last year, as well as all friends in Greece. vi Chapter 1 Chaos, Complexity and Irish Music 1.1 Overview There is a common misconception that complexity and chaos are synonymous. Besides the nonlinearities that occur in both systems, the other properties those two areas of mathematics share are disambiguity in definitions and having many interesting and different applications. Although a definition of chaos that everyone would accept does not exist, almost everyone agrees to three properties a chaotic system should have: Chaos is aperiodic long-term behaviour in a deterministic system that exhibits sensitive dependence on initial conditions[4]. A time evolving property such as the move of tectonic plates or planets, the temperature or any weather characteristic or even the price movements in stock markets or dreams and emotions [5] may display chaotic behaviour. Chaos is often related to complexity, but does not follow from it in all cases. Chaos might be occurring when studying phenomena as they progress in time, but when the same phenomena are examined from a microscopic point of view then, the interaction of the various parts of which the system is consisted creates patterns and not ‘erratic’ chaotic behaviour. This is where complexity enters. It is rather difficult to define complexity in mathematical terms, although there is a measure of complexity there is no other way to give mathematical definition. Dictionaries might be useful in this quest for defining complexity. According to an online Dictionary by Oxford University Press, complex is an ad- jective used to describe nouns which ‘are consisted of different and connected parts’. An even more precise definition is ‘consisted of interconnected or interwo- ven parts.’[6] That means, that in order to understand the behaviour of a complex system we should understand the behaviour of each part, as well as how they in- teract to form the behaviour of the whole. Our incapacity to describe the whole 1 2 CHAPTER 1. CHAOS, COMPLEXITY AND IRISH MUSIC without describing each part combined with the necessity to relate each part with another when describing makes the study of complex systems very difficult. Finally, based on [6] an attempt to formalise all the above definitions can be made: a complex system is a system formed out of many components whose be- haviour is emergent,that is, the behaviour of the system cannot be simply inferred only from the behaviour of its components. The amount of information needed to describe the behaviour of such a system is a measure of its complexity. If the number of the possible states the system could have is Ω and it is needed to specify in which state it is in, then the number of binary digits needed to specify this state is related to the number of the possible states: I = log2(Ω) (1.1.1) In order to realise which state the system is in, all the possible states must be examined. The fact that the unique representation of each state requires a number equal to the number of the states, leads to the conclusion that the number of states of the representation is equal to the number possible states of the system. For a string of N bits, there are 2N possible states, therefore: Ω = 2N ⇔ N = I (1.1.2) There are many applications of complex systems, in statistical physics, me- teorology, geology, biology, engineering, economics, even social sciences and psy- chology. In all sciences we could find systems that could be dismantled in their core components and study each part and the system as a whole simultaneously. It is very interesting trying to examine and forecast the behaviour of systems that consist of human beings, systems like a family, or a business, or even a gov- ernment. Humans have the capacity to learn and to constantly evolve, making models that deal with them unrealistic and of no use if this property is not taken into consideration. Therefore the need to model this human behaviour created the complex adaptive systems (CAS). The most common definition and univer- sally approved is the one given by one of its founders, John H. Holland of Santa Fe Institute: ‘A Complex Adaptive System (CAS) is a dynamic network of many agents (which may represent cells, species, individuals, firms, nations) acting in parallel, constantly acting and reacting to what the other agents are doing. The control of a CAS tends to be highly dispersed and decentralised. If there is to be any coherent behaviour in the system, it has to arise from competition and cooperation among the agents themselves. The overall behaviour of the system is the result of a huge number of decisions made every moment by many individual agents’[7]. It is even more intriguing that CAS do not appear only in human networks but wherever there is a system with interacting elements. It could be cells in a cellular automaton, ions in a spin glass or even cells in an immune system[8]. A complex adaptive system besides the property of complexity, has the properties of emergence and self-organisation. 1.1. OVERVIEW 3 Emergence occurs when agents that operate in the same environment start to interact which each other. The number of the interactions increases when the number of agents increases, this leads to the appearance of new types of behaviour.