Think Complexity.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
Think Complexity Allen B. Downey Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo Think Complexity by Allen B. Downey Copyright © 2012 Allen Downey. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/institutional sales department: (800) 998-9938 or [email protected]. Editors: Mike Loukides and Meghan Blanchette Indexer: Allen Downey Production Editor: Kristen Borg Cover Designer: Karen Montgomery Proofreader: Katie DePasquale Interior Designer: David Futato Illustrator: Robert Romano March 2012: First Edition. Revision History for the First Edition: 2012-02-21 First release See http://oreilly.com/catalog/errata.csp?isbn=9781449314637 for release details. Think Complexity is available under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License (http://creativecommons.org/licenses/by-nc-sa/3.0/legalcode). The author maintains an online version at http://thinkcomplex.com/thinkcomplexity.pdf. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. Think Complexity, the image of a black eagle, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information con- tained herein. ISBN: 978-1-449-31463-7 [LSI] 1329856783 Table of Contents Preface ..................................................................... vii 1. Complexity Science ...................................................... 1 What Is This Book About? 1 A New Kind of Science 2 Paradigm Shift? 3 The Axes of Scientific Models 4 A New Kind of Model 6 A New Kind of Engineering 7 A New Kind of Thinking 8 2. Graphs ............................................................... 11 What’s a Graph? 11 Representing Graphs 13 Random Graphs 17 Connected Graphs 17 Paul Erdős: Peripatetic Mathematician, Speed Freak 18 Iterators 18 Generators 20 3. Analysis of Algorithms .................................................. 21 Order of Growth 22 Analysis of Basic Python Operations 24 Analysis of Search Algorithms 26 Hashtables 27 Summing Lists 31 pyplot 33 List Comprehensions 34 4. Small World Graphs .................................................... 37 Analysis of Graph Algorithms 37 iii FIFO Implementation 38 Stanley Milgram 39 Watts and Strogatz 40 Dijkstra 41 What Kind of Explanation Is That? 43 5. Scale-Free Networks .................................................... 45 Zipf’s Law 45 Cumulative Distributions 46 Continuous Distributions 48 Pareto Distributions 49 Barabási and Albert 51 Zipf, Pareto, and Power Laws 52 Explanatory Models 54 6. Cellular Automata ..................................................... 57 Stephen Wolfram 58 Implementing CAs 59 CADrawer 61 Classifying CAs 62 Randomness 63 Determinism 64 Structures 65 Universality 67 Falsifiability 69 What Is This a Model Of? 70 7. Game of Life .......................................................... 73 Implementing Life 74 Life Patterns 76 Conway’s Conjecture 77 Realism 77 Instrumentalism 79 Turmites 80 8. Fractals .............................................................. 81 Fractal CAs 82 Percolation 84 9. Self-Organized Criticality ................................................ 87 Sand Piles 87 Spectral Density 88 Fast Fourier Transform 90 iv | Table of Contents Pink Noise 91 Reductionism and Holism 92 SOC, Causation, and Prediction 95 10. Agent-Based Models ................................................... 97 Thomas Schelling 97 Agent-Based Models 98 Traffic Jams 99 Boids 100 Prisoner’s Dilemma 102 Emergence 103 Free Will 104 11. Case Study: Sugarscape ................................................ 107 The Original Sugarscape 107 The Occupy Movement 108 A New Take on Sugarscape 108 Pygame 108 Taxation and the Leave Behind 109 The Gini Coefficient 109 Results with Taxation 110 Conclusion 112 12. Case Study: Ant Trails .................................................. 115 Introduction 115 Model Overview 115 API Design 117 Sparse Matrices 118 wx 118 Applications 120 13. Case Study: Directed Graphs and Knots ................................... 121 Directed Graphs 121 Implementation 122 Detecting Knots 122 Knots in Wikipedia 124 14. Case Study: The Volunteer’s Dilemma .................................... 125 The Prairie Dog’s Dilemma 125 Analysis 126 The Norms Game 127 Results 128 Improving the Chances 129 Table of Contents | v A. Call for Submissions ................................................... 131 B. Reading List ......................................................... 133 Index ..................................................................... 135 vi | Table of Contents Preface Why I Wrote This Book This book is inspired by boredom and fascination: boredom with the usual presentation of data structures and algorithms, and fascination with complex systems. The problem with data structures is that they are often taught without a motivating context; the problem with complexity science is that it is usually not taught at all. In 2005, I developed a new class at Olin College where students read about topics in complexity, implement experiments in Python, and learn about algorithms and data structures. I wrote the first draft of this book when I taught the class again in 2008. For the third offering, in 2011, I prepared the book for publication and invited the students to submit their work in the form of case studies for inclusion in the book. I recruited nine professors at Olin to serve as a program committee and choose the re- ports that were ready for publication. The case studies that met the standard are in- cluded in this book. For the next edition, we invite additional submissions from readers (see Appendix A). Suggestions for Teachers This book is intended as a scaffold for an intermediate-level college class in Python programming and algorithms. My class uses the following structure: Reading Complexity science is a collection of diverse topics. There are many interconnec- tions, but it takes time to see them. To help students see the big picture, I give them readings from popular presentations of work in the field. My reading list and sug- gestions on how to use it are in Appendix B. Exercises This book presents a series of exercises; many of them ask students to reimplement seminal experiments and extend them. One of the attractions of complexity is that the research frontier is accessible with moderate programming skills and under- graduate mathematics. vii Discussion The topics in this book raise questions in the philosophy of science, and lend themselves to further reading and classroom discussion. Case studies In my class, we spend almost half the semester on case studies. Students participate in an idea generation process, form teams, and work for 6–7 weeks on a series of experiments, which they then present in the form of a publishable 4–6 page report. An outline of the course and my notes are available at https://sites.google.com/site/comp modolin. Suggestions for Autodidacts In 2009–10, I was a Visiting Scientist at Google, working in their Cambridge office. One of the things that impressed me about the software engineers I worked with was their broad intellectual curiosity and drive to expand their knowledge and skills. I hope this book helps people like them explore a set of topics and ideas they might not encounter otherwise, practice programming skills in Python, and learn more about data structures and algorithms (or review material that might have been less engaging the first time around). Some features of this book intended for autodidacts are: Technical depth There are many books about complex systems, but most are written for a popular audience. They usually skip the technical details, which is frustrating for people who can handle it. This book presents the mathematics and other technical content you need to really understand this work. Further reading Throughout the book, I include pointers to further reading, including original papers (most of which are available electronically), related articles from Wikipe- dia,1 and other sources. Exercises and (some) solutions For many of the exercises, I provide code to get you started, and solutions if you get stuck or want to compare your code to mine. 1. Some professors have an allergic reaction to Wikipedia, on the grounds that students may depend too heavily on an unreliable source. Since many of my references are Wikipedia articles, I want to explain my thinking. First, the articles on complexity science and related topics tend to be very good; second, they are written at a level that is accessible after you have read this book (but sometimes not before); and finally, they are freely available to readers all over the world. If there is a danger in sending readers to these references, it is not that they are unreliable, but that the