Prediction Markets: Theory and Applications A dissertation presented by Michael Edward Ruberry to The School of Engineering and Applied Sciences in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the subject of Computer Science Harvard University Cambridge, Massachusetts November 2013 c 2013 - Michael Edward Ruberry All rights reserved. Dissertation Advisor: Professor Yiling Chen Michael Edward Ruberry Prediction Markets: Theory and Applications Abstract In this thesis I offer new results on how we can acquire, reward, and use accurate predictions of future events. Some of these results are entirely theoretical, improving our understanding of strictly proper scoring rules (Chapter 3), and expanding strict properness to include cost functions (Chapter 4). Others are more practical, like developing a practical cost function for the [0, 1] interval (Chapter 5), exploring how to design simple and informative prediction markets (Chapter 6), and using predictions to make decisions (Chapter 7). Strict properness is the essential property of interest when acquiring and rewarding predictions. It ensures more accurate predictions are assigned higher scores than less accurate ones, and incentivizes self-interested experts to be as accurate as possible. It is a property of associations between predictions and the scoring functions used to score them, and Chapters 3 and 4 are developed using convex analysis and a focus on these associations; the relevant mathematical background appears in Chapter 2, which offers a relevant synthesis of measure theory, functional analysis, and convex analysis. Chapters 5{7 discuss prediction markets that are more than strictly proper. Chap- ter 5 develops a market for the [0, 1] interval that provides a natural interface, is computable, and has bounded worst-case loss. Chapter 6 offers a framework to un- derstand how we can design markets that are as simple as possible while still providing iii Abstract an accurate prediction. Chapter 7 extends the classical prediction elicitation setting to describe decision markets, where predictions are used to advise a decision maker on the best course of action. iv Contents Title Page . ii Abstract . iii Table of Contents . v Citations to Previously Published Work . viii Acknowledgments . ix Dedication . x 1 Introduction 1 1.1 Convex Functions and Relations . 4 1.2 Scoring Rules . 4 1.3 Cost Functions . 8 1.4 A Cost Function for Continuous Random Variables . 10 1.5 Simple and Informative Markets . 11 1.6 Making Decisions with Expert Advice . 14 2 Mathematical Background 16 2.1 Measures, Measurable Spaces, Sets and Functions . 17 2.1.1 Measurable Spaces and Sets . 17 2.1.2 Measures . 19 2.1.3 Measurable Functions . 21 2.1.4 Lebesgue Measure as a Perspective . 23 2.2 Banach Spaces and Duality . 28 2.2.1 Banach Spaces . 28 2.2.2 Duality . 31 2.3 Convex Functions and their Subdifferentials . 32 2.3.1 Functions and Relations . 32 2.3.2 Convex Functions . 33 2.3.3 The Subdifferential . 34 2.3.4 Refining the Subdifferential . 37 2.3.5 G^ateauxdifferential . 39 2.3.6 Cyclic Monotonicity and the Subdifferential . 41 v Contents 2.3.7 Conjugate Functions . 42 2.3.8 Supporting Subgradients . 44 3 Scoring Rules 48 3.1 Scoring Rules, Formally . 51 3.2 Prediction Markets and Scoring Rules . 53 3.3 Characterizing Strictly Proper Scoring Rules . 55 3.4 Gneiting and Raftery's characterization . 61 4 Cost Functions 65 4.1 Scoring Relations . 67 4.2 Strictly Proper Cost Functions . 70 4.3 Cost Functions in Duality . 74 5 Practical Cost Functions 81 5.1 Cost Functions as Futures Markets . 81 5.1.1 Cost Function Prediction Markets . 83 5.1.2 Prices and the Reliable Market Maker . 85 5.1.3 Bounded Loss and Arbitrage . 89 5.2 A Cost Function for Bounded Continuous Random Variables . 92 5.2.1 Unbiased Cost Functions . 93 5.2.2 A New Cost Function . 95 5.3 Practical Cost Functions in Review . 103 6 Designing Informative and Simple Prediction Markets 106 6.1 Related Work . 112 6.2 Formal Model . 114 6.2.1 Modeling Traders' Information . 114 6.2.2 Market Scoring Rules . 115 6.2.3 Modeling Traders' Behavior . 117 6.3 Information Aggregation . 118 6.3.1 Aggregation . 121 6.4 Designing Securities . 123 6.4.1 Informative Markets . 124 6.4.2 Always Informative Markets . 125 6.4.3 Fixed Signal Structures . 129 6.4.4 Constrained Design . 130 6.5 Designing Markets in Review . 132 7 Decision Making 134 7.1 Introduction . 135 7.2 Prediction and Decision Markets . 138 7.3 Eliciting Predictions for Strictly Proper Decision Making . 145 vi Contents 7.3.1 Eliciting Predictions and Decision Making . 146 7.3.2 Scoring Predictions . 148 7.3.3 Incentives and Strict Properness . 149 7.4 Strictly Proper Decision Making . 154 7.4.1 Strictly Proper Decision Markets . 155 7.4.2 Strictly Proper Decision Making with a Single Expert . 161 7.5 Recommendations for Decision Making . 162 7.5.1 A Model for Recommendations . 163 7.5.2 Characterizing Recommendation Rules . 165 7.5.3 Quasi-Strict Properness and Strictly Proper Recommendation Rules . 169 7.6 Decision Making in Review . 171 8 Conclusion 174 8.1 Strict Properness . 174 8.1.1 Valuing the Class of Elicitable Predictions . 179 8.1.2 Relaxing No Arbitrage . 180 8.2 Simple and Informative Markets . 180 8.3 Expert Advice and Decision Making . 181 8.4 In Conclusion . 183 vii Citations to Previously Published Work Portions of the mathematical background and discussion of scoring relations in Chap- ter 2, as well as the entirety of Chapter 3, was developed from or appears in \Cost Function Market Makers for Measurable Spaces", Yiling Chen, Mike Ruberry, and Jenn Wortman Vaughan, Proceedings of the 14th ACM Con- ference on Electronic Commerce (EC), Philadelphia, PA, June 2013. Most of Chapter 4 previously appeared in \Designing Informative Securities", Yiling Chen, Mike Ruberry, and Jenn Wortman Vaughan, Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (UAI), Catalina Island, CA, August 2012. Most of Chapter 5 previously appeared in the following journal paper, which precedes the published conference paper listed below \Eliciting Predictions and Recommendations for Decision Making", Yiling Chen, Ian A. Kash, Mike Ruberry, Victor Shnayder, Revise and resubmit to ACM Transactions on Economics and Computation (TEAC), February 2013. \Decision Markets with Good Incentives", Yiling Chen, Ian A. Kash, Mike Ruberry, and Victor Shnayder, Proceedings of the 7th Workshop on Inter- net and Network Economics (WINE), Singapore, December 2011. viii Acknowledgments Thank you to my big-hearted adviser Yiling Chen and the other members of my thesis committee, Jenn Wortman Vaughan and David Parkes. Thank you to Joan Feigenbaum, who introduced me to economic computation. Thank you to my parents, Ed and Sarah Ruberry. Thank you to my co-authors, Yiling Chen, Jenn Wortman Vaughan, Sven Seuken, Ian Kash, Victor Shnayder, Jon Ullman and Scott Kominers. Thank you to those who introduced me to research, Jay Budzik, Sara Owsley and Ayman Shamma. Thank you Shadi, for pushing me on. ix We choose to go to the moon in this decade and to do these other things not because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win. |President John F. Kennedy Come, my friends, 'Tis not too late to seek a newer world. Push off, and sitting well in order smite The sounding furrows; for my purpose holds To sail beyond the sunset, and the baths Of all the western stars, until I die. It may be that the gulfs will wash us down; It may be we shall touch the Happy Isles, And see the great Achilles, whom we knew. Though much is taken, much abides; and though We are not now that strength which in old days Moved earth and heaven, that which we are, we are, One equal temper of heroic hearts, Made weak by time and fate, but strong in will To strive, to seek, to find, and not to yield. |Lord Alfred Tennyson's Ulysses This thesis is dedicated to my father, Edward Ruberry, who resolutely seeks and accepts the greatest challenges. From his son, Mike. x 1 Introduction 1 1: Introduction All appearances being the same, the higher the barometer is, the greater the probability of fair weather. { John Dalton, 17931 . there has been vague demand for [probabilistic weather] forecasts for sev- eral years, as the usual inquiry made by the farmers of this district has always been, \What are the chances of rain?" { Cleve Hallenbeck, 19202 Verification of weather forecasts has been a controversial subject for more than half a century. There are a number of reasons why this problem has been so perplexing to meteorologists and others but one of the most impor- tant difficulties seems to be in reaching an agreement on the specification of a scale of goodness for weather forecasts. Numerous systems have been pro- posed but one of the greatest arguments raised against forecast verification is that forecasts which may be the \best" according to the accepted system of arbitrary scores may not be the most useful forecasts. { Glenn W. Brier, 19503 One major purpose of statistical analysis is to make forecasts for the future and provide suitable measures for the uncertainty associated with them. { Gneiting & Raftery, 20074 1From [27], see also [61] for a discussion of the history of probabilistic weather forecasts. 2From [46]. 3All of Brier's quotes are from [16]. 4From [43].