the melbourne review Vol 3the Number melbourne 1 May 2007 review
Prediction markets for business and public policy
Andrew Leigh and Justin Wolfers
Recent successes in prediction markets, n 2006, Google decided that it wanted to obtain better forecasts predicting both public events and corporate Iof what the company and its outcomes, have generated substantial interest competitors were planning to do. among social scientists, policy makers and the As with most firms, better predictions of product launch dates, office business community. While these markets have openings and so on are of significant their limitations, they may be useful strategic importance to Google. But instead of using the standard as a supplement to more primitive mechanisms approach — forecasting the future for predicting the future. by asking designated experts —
the melbourne review Vol 3 Number 1 May 2007
Google opted to set up a market, market, the price of a financial Prediction markets — an overview in which any Google staff member security or prediction market contract Perhaps the best-known type could bet on the chances of an event reflects all available information. of prediction market is election- coming true. If a Google staffer had Thus efficient prediction market betting markets. The Iowa Electronic strong information that an event prices hold the promise of yielding Market, established by political would come true, then regardless of efficient and unbiased forecasts; scientists at the University of Iowa her rank in the company, she could indeed, the market price may be in 1988, is perhaps the world’s bet on that outcome. The results not only the single best predictor of best known election market. These — based on the aggregated bets of the event but also the most efficient academics operate an electronic thousands of Google staff members aggregator of available information, market in which traders can purchase — were strong predictors of the such that no combination of available ‘futures contracts’ that consist of a actual outcomes. Where the trading polls or other information can be promise to pay $1 if the candidate volume was high, the forecasts were used to improve on the market- wins the popular vote. Thus the even more precise. And, as closing generated forecast. This statement price of this contract reflects the time drew closer, the markets became does not require that all individuals probability of a candidate winning steadily more accurate.1 in a market be rational, as long as the election.3 Assuming market This example describes an the marginal trade in the market efficiency, these prices should yield increasingly important information- is motivated by rational traders. Of assessments that reflect all available aggregation tool, known as ‘prediction course, it is unlikely that prediction information — including polls, the markets’.2 Analytically, these are markets are literally efficient but state of the economy, and recent markets where participants trade a number of recent successes in policy pronouncements. in contracts whose payoff depends on unknown future events — just as in any financial or betting market. The defining feature of a prediction market is that the The defining feature of a prediction price of these contracts can be directly interpreted as a market is that the price of these market-generated forecast of some unknown quantity. contracts can be directly interpreted as a market-generated forecast of some unknown quantity. Much of the these markets, both with regard For US presidential elections, the enthusiasm for prediction markets to predicting public events and Iowa Electronic Market has tended to derives from the efficient markets corporate outcomes, have generated be more accurate than opinion polls. hypothesis: in a truly efficient substantial interest among social Research has suggested that this is scientists, policy makers and the because the betting market focuses business community. on the underlying dynamics of the We begin by outlining the types of race, and is therefore better able to prediction markets currently on offer parse out events that occur several and then discuss their accuracy. We months before the election, but will 4 review what is known about how not change the outcome. Conversely, to design an effective market and the betting market responds rapidly the constraints of the present legal to occurrences that affect the regime. Finally, we discuss the kinds underlying dynamics of the race of inferences that can be drawn from (such as the appointment of a new prediction markets and conclude with campaign manager), even if these our own prognostications on what the events elicit relatively little response future might hold. in the media or polls.
the melbourne review Vol 3the Number melbourne 1 May 2007 review
In Australia, looser regulation of Table 1: Some Popular Prediction Markets sports betting in the Australian Capital Territory and the Northern Market Focus Typical turnover on an Territory has led several bookmakers event ($US) — including Betfair, Centrebet, Iowa Electronic Small-scale election Tens of thousands International All Sports, SportingBet Markets markets. Similar of dollars
the melbourne review Vol 3 Number 1 May 2007
of economic statistics, including in 2001, and only two out of four In a corporate context, the Hollywood employment, retail sales, industrial major pollsters in 2004.6 Stock Exchange predicts opening production, and business confidence. weekend box-office success and these Perhaps more interesting in terms Some markets also forecast private- predictions have been quite accurate. of how well prediction markets sector returns. The Hollywood Further, this market has been about can aggregate information is the as accurate at forecasting Oscar Stock Exchange allows people to performance of markets at the level 9 use virtual currency to speculate of the individual district. Typically, winners as an expert panel. Some on movie-related questions like districts are sufficiently small that firms have also begun to experiment opening-weekend performance, total there is little interest (or funding) for with internal prediction markets. An box office returns, and likely Oscar local polling. Yet when Australian internal market at Hewlett-Packard winners. In several cases (such as bookmakers started betting on produced more accurate forecasts the Google example above), private district-level races, we found that of printer sales than the firm’s own firms have found innovative ways to they were extremely accurate. In processes.10 Gerhard Ortner described use prediction markets as a business 2001, the Centrebet favourite won an experiment at Siemens in which forecasting tool. in 43 out of 47 marginal seats.7 an internal market predicted that the firm would definitely fail to deliver on a software project on time, even when Arguably, the most important issue with these markets traditional planning tools suggested is their performance as predictive tools. In the political that the deadline could be met.11 domain, the Iowa Electronic Markets have both yielded While the Hollywood markets very accurate predictions for US elections and also have drawn many participants outperformed the major pollsters. simply on the basis of their entertainment value, the HP and Siemens experiences suggested that How accurate are prediction market In 2004, the Centrebet favourite motivating employees to trade was forecasts? won in 24 out of 32 marginals.8 a major challenge. In each case, the However, comparing the performance firms ran real money exchanges, Arguably, the most important of markets with a mechanistic with only a relatively small issue with these markets is their application of poll-based forecasting trading population, and subsidised performance as predictive tools. In the may not provide a particularly participation in the market, by either political domain, the Iowa Electronic compelling comparison. A more endowing traders with a portfolio, Markets have both yielded very or matching initial deposits. The accurate predictions for US elections relevant starting point might be to predictive performance of even these and also outperformed the major compare the predictions of markets very thin markets was quite striking. pollsters.5 In a comparison of election- with those of independent analysts. Yet, as the Google example discussed eve forecasts, the Iowa markets For example, a survey of 10 experts in the opening paragraph of this predicted candidates’ vote shares with published on the Sunday before the article suggests, internal markets an average absolute error of 2004 federal election found that with hundreds of traders are likely to 1.5 percentage points, while the three thought Mark Latham would have even greater forecasting power. average forecast error for Gallup win, while seven thought John polls was 2.1 percentage points. In Howard would win, but with In another recent prediction market, Australia, election-eve betting markets a smaller majority than in 2001. traders in ‘Economic Derivatives’ correctly predicted that the Coalition None forecast the true result — predict the likelihood that economic would win the 2001 and 2004 federal a Howard victory with an increased data released later in the week will elections. By contrast, a Coalition win majority. By contrast, the betting take on specific values. The traditional was predicted by the election-eve polls market was more confident of a approach to aggregating forecasts of only two out of three major pollsters Howard victory in 2004 than in 2001. is to simply take an average or a
10 the melbourne review Vol 3the Number melbourne 1 May 2007 review
‘consensus estimate’ from a survey of One intriguing question is how much around 30 professional forecasters. difference it makes whether prediction Comparing average market and markets are run with real money consensus forecasts for four variables or with some form of play-money. (non-farm payrolls, retail trade, Legal restrictions on gambling have unemployment claims and business led some groups like NewsFutures. confidence) shows that the market- com to adopt play-money exchanges, based predictions of these economic with those who amass the largest indicators are not only extremely play fortunes eligible for prizes. One close to the corresponding ‘consensus’ suggestive experiment compared the forecast, but also typically closer to predictive power of the prices from the actual outcome, outperforming the real-money and play-money exchanges 12 experts on average. over the 2003 NFL football season, When several markets run finding that both yielded predictions simultaneously on the same event, that were approximately equally 16 arbitrage across markets becomes accurate. Interestingly, both sets possible. Yet — consistent with of prices also outperformed all but market efficiency — most studies a dozen of 3,000 people in an online find little evidence of unexploited contest, and also easily outperformed profit opportunities. For example, the average assessments of these our analysis of data from five betting ‘experts.’ One practical advantage exchanges in the 2004 Australian of play-money contracts is that they election suggested only one potential offer more freedom to experiment with arbitrage opportunity throughout different kinds of contracts. On play- the entire campaign (and even that money exchanges, such as Foresight would have yielded only a 2 per cent had little effect.14 During the 2004 Exchange, one often sees quite loosely profit margin). Despite significant US Presidential election campaign, worded ‘contracts’ such as that a fluctuations, betting odds appear to one trader spent tens of thousands of ‘scientific study will conclude that co-move very closely. dollars trying to shorten the odds of a astrology is a statistically significant Kerry Presidency on Tradesports.com predictive method to describe an Market manipulation represents but prices recovered to their previous individual’s personality traits’. a very serious concern about the level within an hour.15 predictive ability of these markets. Some prediction markets will work Yet the profit motive has usually Market design better when they concern events proven sufficient to ensure that that are widely discussed, since attempts at manipulating prediction The success of prediction markets, trading on such events will have markets are unsuccessful. There like any market, can depend on higher entertainment value and there have been several known attempts their design and implementation. will be more information on whose at manipulation of these markets, but For a prediction market to work interpretation traders can disagree. none of them had much discernible well, contracts must be clear, easily Ambiguous public information may effect on prices, except during a short understood, and easily adjudicated. be better in motivating trade than transition phase. For example, one This requirement for clarity can private information, especially if the experimenter placed random $500 sometimes turn out to be complex. private information is concentrated, bets on the Iowa Electronic Markets to For example, in the course of since a cadre of highly informed trace their effect.13 Another researcher Siemens’ internal prediction market traders can easily drive out the partly attempted to manipulate betting on on whether a software project would informed, repressing trade to the horse races by cancelling large wagers be delivered to the client on schedule, point that the market barely exists. at the last moment but this, ultimately, the client changed the deadline. Indeed, attempts to set up markets on
11 the melbourne review Vol 3 Number 1 May 2007
topics where there are insiders with agencies already handle significant substantial information advantages sums of money for elections and have typically failed. For instance, major sporting events, relaxation the Centrebet market on who will of the regulation governing such lead the Liberal Party to the 2007 markets would bring little risk but federal election has generated very a significant public benefit. little trade despite the inherent interest in these questions. The Making inferences from prediction markets same has proved true for prediction How might business people or markets on when a US Supreme policy makers use the results from Court justice will choose to retire. prediction markets in subsequent analysis? The most direct form Legal constraints of inference involves using these In the United States, the legal predictions directly. For instance, environment has forced onshore in the Hewlett Packard experiments prediction markets to make discussed above, the markets elicited compromises that have limited their expectations of future printer sales, attractiveness. The Iowa Electronic which were of direct interest for Markets agreed to limit positions a joint venture by Deutsche Bank and internal planning purposes. to $500 in order to receive a ‘no Goldman Sachs that runs markets action’ letter from the Commodity in economic numbers, has been Some analyses have tried to link the Futures Trading Commission. This operating since 2002. Its traders are time series of expectations elicited in compromise limits the scope and large institutions, and its markets prediction markets with time series depth of the markets, and possibly have attracted volumes of hundreds of other variables. For instance, in their efficiency. Platforms such of millions of dollars. While the US a paper we co-authored with Eric as NewsFutures, the Hollywood Government has taken a strong Zitzewitz, we tracked movements anti-gambling stance, it seems that Stock Exchange, and the Foresight in the ‘Saddam Security’, a contract innovation will continue apace with Exchange operate using play- traded on the Tradesports exchange new contracts being launched instead money, albeit play-money that can that paid $10 if Saddam Hussein was on financial markets. sometimes be converted into prizes. ousted by a given date.17 Interpreting Thus these platforms evade anti- In Australia, the legal environment movements in the Saddam Security gambling legislation by operating as has prevented prediction markets as an index for the risk of war, we games of skill. from establishing themselves in most estimated the extent to which oil States. In addition, there is a concern prices and the S&P 500 index co- The consulting firms running that, for major bookmakers such moved with the Saddam Security. corporate prediction markets as Centrebet, trading on economic Our conclusions (publicly released have taken the same approach as experimental economists on campuses: they have subsidised …attempts to set up markets on topics where there are participation, allowing everyone to leave a winner, albeit to varying insiders with substantial information advantages have degrees. Finally, two markets that typically failed. we are aware of, Economic Derivatives and Hedgestreet, have obtained regulatory approval to derivatives would bring them into on the eve of the war) suggested that offer trading of innovative futures conflict with the Australian Stock the Iraq war would lead to a $10 per contracts. Hedgestreet started in Exchange and the Australian Futures barrel increase in oil prices, and a 15 2004, and Economic Derivatives, Exchange. Since Australian betting per cent decline in stock prices.
12 the melbourne review Vol 3the Number melbourne 1 May 2007 review
Applying a similar methodology, Joel which party is more likely to bring Robin Hanson has called these Slemrod and Timothy Greimel linked down unemployment. Suppose we contingent markets ‘decision markets’, the price of a Steve Forbes election offer two contracts. The first pays arguing (in a paper entitled ‘Shall We security in the 1996 Republican $100 if Labor wins the 2007 federal Vote on Values but Bet on Beliefs?’) primary market with a rising interest election and unemployment falls in that these expectations should be used rate premium on municipal bond 2008, and if Labor loses the election, to guide decision-making. Relatively prices, because Forbes’s signature all monies are returned. The second simple contingent markets, as well as issue was a ‘flat tax’ that would contract pays $100 if the Coalition more complex combinatorial markets, are as yet virtually untested, and a Since Australian betting agencies already handle useful focus for further research. There significant sums of money for elections and major may be important and interesting sporting events, relaxation of the regulation governing applications in domains where these markets yield useful insights. (There such markets would bring little risk but a significant remains an important caveat: these public benefit. markets reveal expected correlations but correlation need not imply have eliminated the tax exemption wins the 2007 federal election and causation. For instance, it may be that for municipal bond interest.18 In a unemployment falls in 2008, and voters are more willing to vote Liberal similar vein, Erik Snowberg, Justin if the Coalition loses, all monies if they believe that unemployment is Wolfers and Eric Zitzewitz compare are returned. The price difference likely to fall anyway.) movements in stock and bond prices between the two contracts may tell with unexpected movements in us something about the market’s Future applications prediction markets on election day, expectation as to which political Prediction markets are extremely thereby measuring the effects of party is likely to do a better job in useful for estimating the market’s different candidates on the economy.19 lowering unemployment. expectation of the probability of a
It seems quite possible to design prediction market contracts so that they bring out the connection between an event and other variables. For instance, Justin Wolfers and Eric Zitzewitz reported on experimental markets run on Tradesports in the months prior to the 2004 presidential election.20 In one example, they ran markets linked to whether George W. Bush would be re-elected, whether Osama bin Laden would be captured prior to the election, and whether both events would occur. These markets suggested a 91 per cent chance of Bush being re-elected if Osama had been found but a 67 per cent unconditional probability.
In the Australian context, it is possible to imagine using these contingent markets to determine
13 the melbourne review Vol 3 Number 1 May 2007
particular event occurring. Markets on markets will be underprovided. In the Putting Crowd Wisdom to Work’, election outcomes serve a useful public area of economic derivatives, it might Prediction Markets Summit, purpose in providing an up-to-the- well be in the interests of government New York, NY, 3 February, and minute barometer of the probability departments to fund the creation a blog entry at http://googleblog. that a given party or candidate will be of markets that aggregate traders’ blogspot.com/2005/09/putting- elected. For firms whose business is expectations of future economic crowd-wisdom-to-work.html. exposed to a large degree of political releases. Indeed, the Reserve Bank 2 risk, our advice is to follow the betting of Australia already closely tracks For a comprehensive review markets, not the opinion polls or the the difference in yields on inflation- which inspires much of this televised talking heads. indexed bonds and non-indexed bonds essay, see J. Wolfers and E. — effectively a prediction market on Zitzewitz 2004, ‘Prediction Contingent markets serve a related future inflation. Markets’, Journal of Economic goal — helping to answer the Perspectives, vol.18 no. 2, question, ‘If event A occurs, then how More generally, the highly restrictive pp.107–126. will it affect outcome X?’. Carefully stance that governments have designing contingent markets is taken towards regulating prediction 3 J. Wolfers and E. Zitzewitz 2006, trickier than simple prediction markets does not appear to take any ‘Interpreting Prediction Market markets but the benefits from these account of the informational benefits Prices as Probabilities,’ National markets are plausibly larger too. of such markets. Would there be Bureau of Economic Research a public benefit from liberalising Working Paper no. 12200. Prediction markets doubtless have gambling regulations to give their limitations but they may be 4 R. Forsythe, F. Nelson, G.R. prediction markets a freer hand? useful as a supplement to the other Neumann, and J. Wright 1992, You bet. n relatively primitive mechanisms for ‘Anatomy of an Experimental predicting the future like opinion ENDNOTES Political Stock Market,’ American surveys, politically appointed panels Economic Review, vol. 82, no. 1 For a discussion of the Google of experts, hiring consultants or 5, pp. 1142–61; D. Shaw and B. prediction markets, see the holding committee meetings. We are Roberts 2000, ‘Campaign Events, presentation by B. Cowgill 2006, already seeing increasing interest in the Media and the Prospect of ‘Google Prediction Markets: these markets in the private sector, Victory: The 1992 and 1996 US with the experiments at Hewlett Presidential Elections’, British Packard, Microsoft and Google Journal of Political Science, vol. now being supplemented with new 30, pp. 259–289. markets on pharmaceuticals and the 5 J. Berg, R. Forsythe, F. Nelson likely success of future technologies and T. Rietz 2001, ‘Results from a on NewsFutures. Dozen Years of Election Futures With the exception of a short-lived Markets Research’, forthcoming attempt by a US defence agency in C. Plott and V. Smith (Eds) to create information markets, Handbook of Experimental Economic Results. governments have shied away from directly operating them. This may 6 In 2001, ACNielsen and Newspoll be a sensible policy stance — yet correctly forecast a Coalition politicians should also recognise that, win, while Morgan forecast a if the valuable information generated Labor win. In 2004, ACNielsen by trade in these markets is not fully and Galaxy correctly forecast internalised into the profits earned a Coalition win, while Morgan by these private firms, prediction forecast a Labor win, and
14 the melbourne review Vol 3the Number melbourne 1 May 2007 review
Newspoll’s final poll had the A Field Experiment with Andrew Leigh parties 50–50. Racetrack Betting’, Journal of Andrew Leigh is an economist in the Political Economy, vol. 106, no. 3, 7 J. Wolfers and A. Leigh 2002, Research School of Social Sciences at pp. 457–482. ‘Three Tools for Forecasting the Australian National University. Federal Elections: Lessons from 15 P. Rhode and K. Strumpf 2006, Email: [email protected]. 2001’, Australian Journal of ‘Manipulating political stock Website: http://econrsss.anu.edu. Political Science, vol. 37, no. 2, markets: a field experiment au/~aleigh/ pp. 223–240. and a century of observational Justin Wolfers 8 A. Leigh and J. Wolfers 2006, data’, mimeo., University of ‘Competing Approaches to North Carolina. Justin Wolfers is Assistant Professor Forecasting Elections: Economic of Business and Public Policy at 16 E. Servan-Schreiber, J. Wolfers, Models, Opinion Polling and the Wharton School, University Prediction Markets’, Economic D. Pennock and B. Galebach of Pennsylvania, Faculty Research Record, vol. 82, no. 258, pp. 325–340. 2004, ‘Prediction Markets: Does Fellow with the National Bureau Money Matter?’, Electronic of Economic Research, Research 9 D. Pennock, S. Lawrence, C.L. Markets, vol. 14, no. 3. Affiliate with the Centre for Economic Giles, and F.A. Nielsen 2001, ‘The Policy Research, a Visiting Scholar 17 Real Power of Artificial Markets’, A. Leigh, J. Wolfers and E. with San Francisco Federal Reserve Science, vol. 291, pp. 987–988. Zitzewitz 2003, ‘What do Financial and a Research Fellow with Institute Markets Think of War in Iraq?’, 10 K-Y. Chen and C. Plott 2002, for the Study of Labor (IZA). NBER Working Paper no. 9587. ‘Information Aggregation Email: [email protected]. National Bureau of Economic Mechanisms: Concept, Design Web: www.nber.org/~jwolfers and Implementation for a Sales Research, Cambridge, MA. Forecasting Problem’, CalTech 18 J. Slemrod and T. Greimel 1999, Social Science Working Paper ‘Did Steve Forbes Scare the no. 1131. Municipal Bond Market?’, Journal 11 G. Ortner 1998, ‘Forecasting of Public Economics, vol. 74, no. 1, Markets — An Industrial pp. 81–96. Application’, mimeo., Technical 19 E. Snowberg, J. Wolfers and E. University of Vienna. Zitzewitz 2007, ‘Partisan Impacts 12 R. Gürkaynak and J. Wolfers, 2005, on the Economy: Evidence from ‘Macroeconomic Derivatives: An Prediction Markets and Close Initial Analysis of Market-Based Elections’, Quarterly Journal of Macro Forecasts, Uncertainty Economics, vol. 122, no. 2. and Risk’, NBER International Macroeconomics Annual. 20 J. Wolfers and E. Zitzewitz 2004, ‘Experimental Political Betting 13 P. Rhode and K. Strumpf 2006, Markets and the 2004 Election’, ‘Manipulating political stock The Economists’ Voice, vol. 1, no. markets: a field experiment and a century of observational 2, article 1. data’, mimeo., University of 21 R. Hanson 2004, ‘Shall We Vote on North Carolina. Values, But Bet on Beliefs’, mimeo., 14 C. Camerer 1998, ‘Can Asset George Mason University (available Markets be Manipulated? at: http://hanson.gmu.edu)
15