TRENDS & CONTROVERSIES

Building intelligent systems By Marti A. Hearst University of California, Berkeley one e-citizen at a time [email protected]

For decades, the AI community attempted to build intelligent systems by writing software that mimics human thought processes. More recently, researchers have advocated solving com- plex problems by building networks of interacting autonomous agents. But some researchers are now proposing that the key to building intelligent systems is leveraging the power of individual human thought. The idea is that now that networked information systems are widely available, gaping hole in our individual memories. I they can be used to link together the mental efforts of individuals in innovative ways. am talking in general about speculative mar- In this installment of “Trends and Controversies,” we discuss two variations on this idea. kets, and in particular about decision mar- First, Robin Hanson describes his idea of decision markets, also known as idea futures, in which kets. Decision markets might allow us to matters of public debate are resolved via a mechanism like a commodities market. more accurately estimate the consequences This idea rests on the assumption that the answers to pressing social questions (such as, of important decisions, by helping us to bet- “Does violence in video games cause violence in children?”) are not generally agreed on be- ter share relevant information. cause those in the best position to know the answer can’t or won’t share this information. The Consider, for example, a clearly impor- proposed solution is to create a market in which getting the answer right or wrong has financial tant policy question such as, consequences. The hypothesis is that a market like this will tend to draw out the voices of those who are in good positions to know the correct answer. Just as online auctions let buyers and How would crime rates change if more citi- sellers find acceptable prices for their merchandise, an idea market allows knowledge-seekers to zens could legally carry hidden guns? find a well-motivated answer for their question. The market might also compel people to invest in research to find out the correct answer. Many observers say hidden guns obvi- The reason this idea might work is that those who are not well-informed will quickly lose ously increase crime, while many others their investments and will be motivated to stop participating in the market, while those who have strongly disagree (see John Lott’s More more accurate information stand to gain more from the market and will direct its outcome. Han- Guns, Less Crime, Univ. of Chicago son cites research in which an idea-based market was better at predicting election results than Press). polls and expert forecasts. I suspect that the existence of such diver- In the second essay, David Stork presents what he calls the Open Mind Initiative. This is a gent opinions reflects the fact that we suf- broader formulation of an idea that subsumes decision markets. Stork argues that the newly fer from a serious failure to share informa- networked society lets us capture and pool the results of human mental cycles that would oth- erwise be lost if simply expended in individual pursuits. For example, rather than simply play- tion. The so-called Information Revolution ing a video game, why not play one that has a side-effect of producing training data for a has greatly improved our ability to find out machine-learning algorithm? Rather than simply finding and bookmarking a useful Web page, what others have said. However, it has done why not add it to an online Web directory? Many piecewise variations of these ideas are cur- much less to improve our ability to find out rently being bantered about; Stork puts these within a framework that lets us think about them what other people know. We can now find a more generally. He also draws parallels and contrasts between Open Mind and the Open blizzard of words on a topic such as the Source software development effort. interplay between guns and crime, but we These essays are complementary to those that appeared in the January/February 1999 know that most of those words are written “Trends and Controversies” on the unforseen social consequences of networked information by people with axes to grind. The real systems. The earlier essays described how networked information systems are changing society; problem is not finding more words, but this issue addresses how society might help build intelligent systems. —Marti Hearst judging who really knows about the topic and whether these experts are saying what they know. We are in many ways bit-full, yet information-poor. Decision markets The human part of any large intelligent I suggest that speculative markets are a system is by far the most intelligent part. neglected way to help us find out what peo- Robin D. Hanson, George Mason University As long as this remains true, the biggest ple know. Such markets pool the informa- Engineers’ love of technology often gets system advancements will come from aids tion that is known to diverse individuals in the way of their being useful. Consider that fill big holes in human abilities, rather into a common resource, and have many Post-it Notes or, better yet, plain paper note- than from artifacts that stretch engineers’ advantages over standard institutions for pads. These probably seemed like trivial abilities. information aggregation, such as news ideas, but they turned out to be terribly use- I mention all this because I want you to media, peer review, trials, and opinion ful. Why? Because the marvel that is the consider a simple, not very technically chal- polls. Speculative markets are decentral- human brain has a horrible short-term mem- lenging idea—one that might nevertheless ized and relatively egalitarian, and can ory, which means that dumb-as-dirt memory fill a gaping hole in our collective intelli- offer direct, concise, timely, and precise aids can make people substantially smarter. gence, similar to the way notepads fill a estimates in answer to questions we pose.

16 IEEE INTELLIGENT SYSTEMS Robin D. Hanson is a Robert Wood Johnson Foundation Scholar in Health Policy Research at the University of California at Berkeley. This fall he will be an assistant professor of economics at George Mason University. He spent nine years as a researcher in AI and Bayesian statistics, at Lockheed These estimates are self-consistent across a and then at NASA. His research interests include explaining various puz- zling human behaviors in terms of information asymmetries. He has a BS in wide range of issues and respond quickly physics from the University of California at Irvine, an MS in physics and an to new information. They also seem to be MA in philosophy of science from the University of Chicago, and a PhD cheap and relatively accurate. in social science from the California Institute of Technology. Contact him In particular, for questions such as hid- at 140 Warren Hall, School of Public Health, UC Berkeley, 94720-7360; den guns and crime, I suggest we consider [email protected]; http://hanson.berkeley.edu. decision markets, which are speculative David G. Stork is the chief scientist at Ricoh Silicon Valley and a consult- markets focusing on particular decisions. ing associate professor of electrical engineering at Stanford University. A graduate of MIT and the University of Maryland, he has written and edited How decision markets work several books, including HAL’s Legacy and (with R.O. Duda and P.E. Hart) the forthcoming second edition of Pattern Classification. His central inter- Imagine that we created markets where ests are in pattern recognition in machines and humans. Contact him at people could bet on future crime rates, con- Ricoh Silicon Valley, 2882 Sand Hill Rd., #15, Menlo Park, CA 94025- ditional on allowing or not allowing more 7022; [email protected]. hidden guns. That is, if the market prices predicted that murder rates would be 10% higher should a certain hidden-gun bill pass, then anyone who thought this estimate too high could easily identify a particular prof- Next, you must choose some particular conditional on the bill not passing. More- itable trade. If you made this trade and the trusted third party who will finally declare over, comparing these two estimates tells market estimate then fell to 5%, for exam- a murder rate M within [0,1] and determine you whether, and by how much, specula- ple, you could undo this trade for a profit. whether bill B passed. (This party might be tors expect this bill to increase or decrease Imagine further that if market prices said a jury randomly drawn from some pool.) the murder rate. that crime rates would be 10% higher given You must also either pick a date by which Finally, you have to decide how much to more hidden guns, most nonexperts would these judges are to decide, tie the judging subsidize this market. If interest in your accept this estimate as our “best answer” or to some other event like the release of mur- topic is strong enough, simply creating as a neutral “consensus.” In particular, a der statistics, or grant the judges discretion these markets might induce people to trade state legislature might accept this estimate to choose this date for themselves. in them. Failing that, sufficient interest when considering whether or not to pass Third, you must choose what asset A you might be induced if someone committed to this hidden-gun bill. will bet. If the bet is going to last any sub- make a policy choice based on the market I call such a set of markets a decision stantial time, asset A ought to give a rea- estimate. A state legislature, for example, market. In this situation, advocates for each sonable rate of return, to induce speculators might commit to pass the bill or not de- side of an issue would be forced to influ- to invest in it. You might pick a safe gov- pending on the market estimate of their ence speculators if they wanted to influence ernment bond, or you might pick a broad- effect on the murder rate. general opinion. Speculators, in turn, would index stock mutual fund. You can also safely and directly subsi- have a clear incentive to be careful and Also, given M, B, and this asset A, you dize a market to induce more participation. honest in contributing what they know and authorize some financial institution to Doing this in effect creates an information in judging what advocates know. This is make exchanges between units of A and the prize offered to those who first make the because speculators must “put their money following set of four assets: market price better reflect relevant infor- where their mouth is.” mation. (In econo-speak, one way to do Six steps are required to create a deci- 1. M units of A if B passes, this is to create a market maker whose bid sion market to help us better share informa- 2. M units of A if B does not pass, and ask prices are monotonic functions of tion on a topic such as the effect of hidden 3. 1 – M units of A if B passes, its assets held.) guns on crime. 4. 1 – M units of A if B does not pass. In addition to estimating the effect of First, you must state your claim clearly. hidden guns on crime, decision markets For example, you might focus on a particu- Note that because M + (1 – M) = 1, and be- might give us estimates on lar bill B before your state legislature, cause B either will or won’t pass, this finan- which would allow more citizens to carry cial institution takes no risk from these • Murder rates—with or without capital hidden guns. You might decide to focus on exchanges; each set of four assets will be punishment? your state’s murder rate, using some stan- worth exactly one unit of A in the end. • Average mortality rates—with or with- dard government statistic M as your official Fifth, you create markets in which peo- out national health insurance? measure of it. You should choose a lowest ple can trade various combinations of these • Health-care spending—with expanded and highest relevant murder rate, and scale assets with each other. In particular, if peo- or curtailed use of health maintenace M so that M = 0 at the lowest rate and M = ple trade asset 1 for the bundle of assets 1 organizations? 1 at the highest rate. (You might choose, and 3, the market price (asset ratio in • Employment change—raise minimum for example, the lowest rate to be zero mur- trades) is an estimate of the murder rate wage or rescind NAFTA? ders and the highest rate to be the popula- conditional on the bill passing. Similarly, • Global sea-level and temperature tion size, which is the highest conceivable the price in trades of asset 2 for the bundle changes—impose or not impose a car- murder rate.) of 2 and 4 is an estimate of the murder rate bon dioxide tax?

MAY/JUNE 1999 17 • Military casualties—with a ready use them to form consensus Republican or a Democratic on topics such as the correlation president? between hidden guns and crime? • Stock prices—with a Republi- Until the last few centuries, the can- or Democrat-controlled cost of simply handling trades was US Congress? enough to sharply limit the number • World per-capita food con- of speculative markets that could be sumption—raise or lower aver- made widely available. Yet today age tariffs? ebay. com routinely sells $10 items • Student test scores—with or by having a handful of people bid a without school choice or few times each over a period of a voucher reform? week. Moreover, play-money Web • Future national economic growth betting games have shown that just —raise or lower interest rates, a handful of people, each making a or with or without an education few small trades over several years, subsidy? can create reasonable estimates on a wide variety of questions. (See Science fiction writers have pos- www.hsx.com, myhand.com, and ited even more ubiquitous betting especially www.ideosphere.com.) markets (see John Brunner’s Shock- More important, most speculative wave Rider, Del Rey Books, and Marc Stei- those election markets found that while markets are now illegal. The short history of gler’s Earthweb). In general, decision mar- most traders tended to suffer from cogni- financial market regulation is that every- kets can estimate the net effect of any policy tive biases such as expecting others to thing was once illegal, until limited exemp- choice of interest on any outcome of interest, agree with them, the most active traders tions were granted for specific purposes. as long as there is a decent chance that, after were not biased this way—and active Betting on cards was a foolish waste of the fact, we can reasonable verify what out- traders set the prices. Speculative markets money; only fools would invest in a business come happened and what policy was chosen. thus seem to induce the real experts to self- they did not closely monitor; and it was the select and participate more. Lab experi- height of folly to let people bet on the death How well do markets work? ments also indicate that speculative mar- of others. So casinos, stocks, and insurance By its nature, a betting-market estimate kets tend to aggregate information when were all banned. 4 is decentralized, direct, concise, timely, traders are experienced with their roles and Gradually, exceptions were granted for precise, self-consistent, and responds know the payoffs for other roles.3 what came to be seen as worthy purposes, quickly to new information. It is also egali- Older economics writings sometimes such as teaching people about horses tarian, if everyone is allowed to participate. give the impression that speculative mar- (horseracing) or raising state revenue (lot- But how clear, cheap, and accurate are such kets will not function unless you have thou- teries). Stocks were allowed for the pur- market estimates? sands of traders frequently trading millions pose of capitalizing firms, and insurance On accuracy, decades of research on the of dollars worth of goods. Recent Web was allowed to let individuals hedge risks. efficiency of financial markets have found markets, however, show clearly that mar- More recently, commodity futures and little price-relevant information that is not kets can be much smaller and slower than financial derivatives were allowed to let reflected in market prices. Any inefficien- this. Furthermore, a subsidized market can firms hedge more risks. All these areas are cies seem to be weak and to go away with function over any time period with only highly regulated, however, in part to pre- publicity, because they represent a profit one trader. If an information prize is vent limited exemptions from devolving opportunity. If you think the current price offered, and only one person is induced to into general gambling. is too low, you expect to profit by buying learn enough to only once correct the initial Accepted functions of markets now now and selling later, and buying now will market price to something else, the market include entertainment, capitalization, and raise the price, partially correcting the error has still served a valuable information role. hedging, but not information aggregation. you perceived. With subsidies, the key question is not Thus while it is widely recognized that Speculative markets have done well in whether we can create speculative markets, markets created for other purposes accom- direct tests against standard information- or whether such markets can induce people plish IA, we’re prevented from creating a aggregation institutions. For example, to learn and reveal information. The key market whose primary function is IA. So orange juice futures prices have been shown questions are whether the information we cannot create a market whose legal to improve on government weather fore- gained is worth the costs paid and whether price would inform raging policy debates, casts.1 Also, markets where traders can bet a similar benefit could have come cheaper such as the interplay between crime rates on election results predict vote totals better via some other institution. and hidden guns. than opinion polls. 2 Okay, betting markets are mainly illegal. How do markets do so well? After all, What’s the holdup? But if economists have data suggesting that aren’t they made of the same fallible If speculative markets are so great at in- that speculative markets do well at IA, why humans as other institutions? A study of formation aggregation, why don’t we al- aren’t lots of economists pushing the idea

18 IEEE INTELLIGENT SYSTEMS of better IA via more markets? employee bonuses depend on, and a contin- The Open Mind Initiative Actually, it’s worse than you think; econ- gent bonus is pretty close to a bet. So several David G. Stork, Ricoh Silicon Valley omists also have sophisticated theory that companies, including Hewlett-Packard and suggests that IA should not be that hard on Siemens, have begun experimenting with After decades of research in pattern factual topics like the effect of hidden guns real-money internal speculative markets for recognition and components of intelligent on crime. Rational agents should not even estimating things such as future sales. systems, the AI community has shifted its be able to agree on which one of them thinks Corporations also need to make deci- focus from fundamental concepts and math- hidden guns cause more crime.5 (I won’t sions, and often have problems inducing ematical techniques to large-scale data ac- say more, as the editor wisely advises relevant parties to reveal information about quisition and knowledge engineering. In against using more econo-speak.) Eco- the consequences of those decisions. Fur- contest after contest in academic and com- nomic theory thus really does suggest we thermore, companies have a good rough- mercial realms, the best systems for optical humans have a gaping hole in our social and-ready measure of “good for the com- character, speech and face recognition, and intelligence. pany”— the stock price. Thus, you could so on are the ones trained with the most So why aren’t economists pushing IA create decision markets that estimate data. Collecting very large, high-quality markets? One answer is that economists are whether any particular decision, such as datasets is evidently vital to progress in just spread too thin. Economic theory sug- introducing a new product, is better or these and several other areas. Such data is gests many policy improvements over the worse than some alternative for the stock informal—known by everyone who can status quo, and there are few economists price. Alternatively, you might predict the read, speak, or hear, or has a commonsense that anyone else will listen to. These few sales of some product contingent on some understanding of the world. economists thus have to choose their bat- important product-design decision. Consider, too, the qualified but increas- tles carefully. Just as notepads fill a gaping hole in our ingly compelling success of the Open The relevant theory for IA is also recent, individual cognitive abilities, speculative Source methodology—which promotes and most economists don’t yet know about markets might fill a gaping hole in our col- software reliability and quality by support- it. Worse, the IA functions of markets lective ability to share information. Eco- ing independent peer review and rapid evo- seem too complex to model in much gen- nomic theory suggests that IA should not lution of freely distributed source code. erality. When systems become too com- be that hard, at least for factual policy Linux, SendMail, Apache, the Mozilla ver- plex to model in detail, engineers usually questions like the effect of hidden guns on sion of the Netscape Web browser, and resort to building and testing theory- crime rates. Speculative markets seem to other high-quality software testify to the inspired prototypes. However, economic- work well at such tasks. Let us thus de- viability of this collaborative approach to theorist types who understand this area are velop prototypes to explore this potential, software engineering. reluctant to move that far away from the- in the hopes of someday lifting current Together, these developments suggest a ory. (Capitalization and hedging functions legal barriers to widespread use of more new approach to building components of of markets are easier to model, and econo- effective institutions for IA. intelligent systems—the Open Mind Initia- mists do use theory to design market pro- For more information on this topic, see tive. This initiative relies on three types of totypes for these functions.) http://hanson.berkeley.edu/ideafutures.html. participants: It thus seems to fall to a few economics- savvy and engineering-minded folks like • domain experts who contribute libraries me to think of using prototypes to explore of algorithms, the idea of using more speculative markets • tool developers who contribute and for IA. refine the enabling software, and References • lay e-citizens who contribute data via A promising direction: internal the Internet. 1. R. Roll, “Orange Juice and Weather,” Amer- corporate markets ican Economic Rev., Vol. 74, No. 5, Dec. On the types of topics to which they 1984, pp. 861–880. Users with an interest and expertise in a have been applied so far, IA markets have particular domain, such as speech, vision, looked promising. There remain, however, 2. R. Foresythe et al., “Anatomy of an Experi- mental Political Stock Market,” American many legitimate concerns. For example, Economic Rev., Vol. 82, No. 5, Dec. 1992, does the existence of speculative markets pp. 1142–1161. discourage communication via other chan- Coming Next nels, and is this a net benefit or loss? 3. R. Foresythe and R. Lundholm, “Informa- Tests of prototypes might help us answer tion Aggregation in an Experimental Mar- ket,” Econometrica, Vol. 58, No. 2, Mar. Issue such questions. But how can we test proto- 1990, pp. 309–347. types, if IA markets are generally illegal? Well, there is one plausible loophole (be- 4. I.N. Rose, Gambling and the Law, Gam- sides offshore gambling), which I have bling Times Inc., Hollywood, Calif., 1986. Quantum Computing saved for those of you who are still reading 5. R. Hanson, Four Puzzles in Information & and AI this far: internal corporate markets. Corpo- Politics, PhD thesis, California Inst. of rations have great leeway in what they make Technology, Pasadena, Calif., 1997.

MAY/JUNE 1999 19 Open Mind host

Internet

motivations of e-citizens deserve special consideration. E-citizens seek benefit from the resulting Open Mind software, including software that would be very difficult to de- velop in other ways, such as commonsense E-citizens knowledge. E-citizens would enjoy game interfaces and seek the public recognition of Figure 1. In an Open Mind project on OCR: handwritten characters are presented to e-citizens whose judgements their contributions. There could be financial (here, 4 vs. 9) are entered for instance by means of buttons. Such responses are returned to the Open Mind host and incentives such as lotteries, discounts, or used to train the classifier. frequent-flier awards provided by corpora- tions seeking new customers. language, or commonsense, serve as re- It provides an interface and protocol design Others have discussed components of viewers or moderators. for Open Mind and its use in different con- Consider the development of an optical texts—online interactive data acquisition character-recognition system through Open • efficiently extracting the maximum and voting, collaboration, and machine- Mind (see Figure 1). A host machine pre- information from e-citizens, learning and pattern-recognition algo- sents pixel images of handwritten charac- • detecting and eliminating significant rithms—yet the integration proposed in ters, transformed by distortions, warping, errors and statistical “outliers,” Open Mind seems not to have been dis- line thickening, and so forth, on browsers • repelling hostile attacks, and cussed. A number of existing projects of e-citizens, perhaps in a game interface. • automatically listing contributors would fit under an Open Mind umbrella and E-citizens classify these images by means according to the amount of information would profit from the initiative. One exam- of a button response; the responses are they contributed. ple is Newhoo, where nonspecialist e-citi- aggregated, screened for significant out- zens propose keyword and index informa- liers at the host machine, and used as train- The Open Mind Initiative differs from tion about Web pages. Their contributions ing data to improve the classifier. the Free Software Foundations and the are reviewed by volunteer referee and edi- To explore the novel infrastructure re- Open Source organization in a number of tors (currently 10,000) and made available quired by Open Mind, Stanford graduate ways. First, while Open Source draws its to all. We can imagine Open Mind projects student Chuck Lam and I have developed a support almost entirely from a hacker cul- in numerous pattern-recognition domains or Java- and Web-based version of Animals, ture (for example, roughly 105 program- knowledge engineering to improve navigat- an elementary interactive children’s pro- mers have contributed to Linux), Open ing news groups, Web sites, or FAQs. Later, gram for classifying animals, dating from Mind relies heavily on an e-citizen and these systems can be integrated, for in- the late 1970s. The child (e-citizen) thinks business culture (109 nonprogrammers on stance, to incorporate natural language or of an animal. Using the child’s responses to the Web). While most of the work in Open commonsense constraints in speech recog- a series of questions (for example, two- Source is directly on the final released nition, OCR, or web searching. legged or four-legged?), the program source code, most of the effort in Open Physics has had its atom smashers, tries to determine this animal’s identity. If Mind focuses on the tools, infrastructure, microbiology its Human Genome Project, the program guesses wrong, the child must and data gathering. An expert arbitrates and aeronautics and astronautics its space enter a question that distinguishes her ani- final decisions in Open Source; in Open missions. Now is the time for computer mal from the program’s guess. After a Mind, much information is accepted or science and cognitive science to have number of children have played this guess- rejected automatically by the infrastructure their big science—one that harvests infor- ing game, Animals has learned a simple software. Table 1 summarizes some of mal knowledge from a large number of tree-based classifier for animals. While not these differences. e-citizens for building useful software for a deep or particularly useful core program, While domain experts and infrastructure next-generation systems. Given the con- Animals provides an excellent platform for developers are likely to have the same mo- junction of several forces—the need for solving important problems in Open Mind. tivations as contributors to Open Source, the natural human-machine interfaces and improved Web searching, the existence of Table 1. Comparisons between Open Source and Open Mind. good learning algorithms and Web infra- structure, and the demonstrated success of OPEN SOURCE OPEN MIND the Open Source methodology—the time is right for the Open Mind Initiative. No e-citizens E-citizens crucial Expert knowledge Informal knowledge Machine learning irrelevant Machine learning essential Reference Web useful but optional Web essential Most work is directly on the end-user Most work is on infrastructure, not on the 1. D.G. Stork, “Character and Document software end-user software Recognition in the Open Mind Initiative,” Hacker culture (<105) E-citizen/business culture (< 109) Proc. Int’l Conf. Document Analysis and Separate functions contributed Single functional goal (for example, recognition Recognition (ICDAR ‘99), to be published (device drivers in Linux) rate in OCR) by IEEE Computer Society Press, Los Alamitos, Calif., 1999.

20 IEEE INTELLIGENT SYSTEMS