Games and the power of capturing player data: Using human computation to investigate belief creation in networks Peter M. Landwehr, Marc Spraragen, & Kathleen M. Carley {plandweh,kathleen.carley}@cs.cmu.edu, [email protected] Center for Computational Analysis of Social and Organizational Systems http://www.casos.cs.cmu.edu/ A quick outline 1. Background on scientific discovery games & our concept 2. A network-based modeling system (this is where belief networks come in) & a model of the Sudan 3. Converting that model to a game 4. Current status February 2011 Peter Landwehr 2 Games employing crowdsourcing February 2011 Peter Landwehr 3 Games employing crowdsourcing February 2011 Peter Landwehr 4 Games employing crowdsourcing Solutions created by users February 2011 Peter Landwehr 5 Games employing crowdsourcing Solutions created by users February 2011 Peter Landwehr 6 Games employing crowdsourcing Solutions created by users Solutions discovered by users in a detailed, complete model February 2011 Peter Landwehr 7 Games employing crowdsourcing Solutions created by users Solutions discovered by users in a detailed, complete model What about for social models? February 2011 Peter Landwehr 8 Crowdsourced games for science Solutions created by users Solutions discovered by users in a detailed, complete model Solutions discovered by users Unknown In an uncertain model Social Game February 2011 Peter Landwehr 9 Crowdsourced games for science Solutions created by users Solutions discovered by users in a detailed, complete model Solutions discovered by users As-yet-unnamed In an uncertain model Sudan Game February 2011 Peter Landwehr 10 Two classic examples/failures SimCity: Flint, Michigan SimHealth • Initial setup modeled • A model of the „90s after Flint, Michigan health care debate • Increase the population • Manipulable premises from 10,000 to 21,000 in x “Liberty vs. Equality” and 5 years. “Community vs. x Confined model Efficiency” assumptions x incorrect information x “Select „No disasters‟ at x oversimplified to experts, the disasters menu.” overly complex to novices February 2011 Peter Landwehr 11 Principles of Scientific Discovery Games • “[V]isualizations & graphics [must]… reflect and illuminate the natural rules of the system[,] manage and hide the complexity of the system[, &] be approachable by players” • “Interactions [must]… respect the constraints of the system[, be] sufficient to explore the space of solutions enough to... solve the problem[, & be] intuitive and fun as possible.” • “The goal must... direct players toward the solution to the problem and encourag[e] players to explore the space.” • “If players are to be successful, it is necessary to teach players the system and how the gameplay, visualizations, interactions, and [goal] work.” February 2011 Peter Landwehr 12 Principles of Scientific Discovery Games • “[V]isualizations & graphics [must]… reflect and illuminate the natural rules of the system[,] manage and hide the complexity of the system[, &] be approachable by players” • “Interactions [must]… respect the constraints of the system[, be] sufficient to explore the space of solutions enough to... solve the problem[, & be] intuitive and fun as possible.” • “The goal must... direct players toward the solution to the problem and encourag[e] players to explore the space.” • “If players are to be successful, it is necessary to teach players the system and how the gameplay, visualizations, interactions, and [goal] work.” February 2011 Peter Landwehr 13 Principles of Scientific Discovery Games • “[V]isualizations & graphics [must]… reflect and illuminate the natural rules of the system[,] manage and hide the complexity of the system[, &] be approachable by players” • “Interactions [must]… respect the constraints of the system[, be] sufficient to explore the space of solutions enough to... solve the problem[, & be] intuitive and fun as possible.” • “The goal must... direct players toward the solution to the problem and encourag[e] players to explore the space.” • “If players are to be successful, it is necessary to teach players the system and how the gameplay, visualizations, interactions, and [goal] work.” Can a model be too complex to learn? For the moment, our answer is “yes” February 2011 Peter Landwehr 14 Construct & Belief Networks • Construct is an agent- based model in which A A agents exchange 1 2 knowledge that can be tied to beliefs F1 F2 F3 F4 • Construct represents all relationships in a -1 +1 -1 +1 network format B1 B2 February 2011 Peter Landwehr 15 Construct & Belief Networks • Construct is an agent- based model in which A A agents exchange 1 2 knowledge that can be tied to beliefs F1 F2 F3 F4 • Construct represents all relationships in a -1 +1 -1 +1 network format B1 B2 February 2011 Peter Landwehr 16 Construct & Belief Networks • Construct is an agent- based model in which A A agents exchange 1 2 knowledge that can be tied to beliefs F1 F2 F3 F4 • Construct represents all relationships in a -1 +1 -1 +1 network format B1 B2 February 2011 Peter Landwehr 17 The Sudan Model (1 of 2) • A model of pre-election Sudan in which policymakers try to create intertribal stability among different Sudanese tribes by inculcating common tribal beliefs. – Tribes – Intervention agents: tribal leaders & national leaders • Every time period, tribe members interact • Two forms of intervention – Speeches to the agent population, providing new information – Forums between different tribal leaders • As tribe members‟ beliefs become similar, tribal relationships become similar. February 2011 Peter Landwehr 18 The Sudan Model (2 of 2) • Information in native Construct Code – Interventions and the Intervention agents • Information pre-defined in networks – Relationships between agents derived from Sudan News data – Knowledge and belief ties • In process: migrating data to a single format • Known simplifications: assuming only two critical forms of interaction, and dramatically reducing the number of tribes in Sudan. 100 Timeperiods of execution with up to 20 interventions. February 2011 Peter Landwehr 19 The Sudan Model Game (1 of 3) • An interactive, graphical front end for the model. • Players analyze current tribal knowledge and beliefs, then choose a desired interventions • Intervention choices are passed to Construct, which resumes its state. • Construct passes new output back to the game. • Players win by getting the average homophily between different tribes‟ beliefs above <a critical value> Player Player Player 5 5 5 intervention intervention intervention February 2011 Peter Landwehr 20 The Sudan Model Game (2 of 3) • We will eventually analyze games in which the win conditions are met, looking at trends in player choices. • These chains of decisions will eventually be evaluated as possible methods that could have been and/or can be used in Sudan to limit tribal hostility. February 2011 Peter Landwehr 21 The Sudan Model Game (3 of 3) • Assets! • A game engine! February 2011 Peter Landwehr 22 Current status (1 of 2) February 2011 Peter Landwehr 23 Current status (2 of 2) • 19 “Speech” Interventions, 30 Simulation runs, 50 time periods • Looking for a statistically significant difference in the beliefs held or total facts known by agents in a target group compared to the agents in a control group. • Checked after 5, 10, 25, and 50 time periods. • No differences found. February 2011 Peter Landwehr 24 Upcoming… • Testing for significance after multiple interactions and fixing the model to see these effects • Tuning of both the model and the game to fill out that precise win condition value. • Live testing of the game with humans. • Further integration of the network format into Construct February 2011 Peter Landwehr 25 Further out… • Increasing the complexity of the model • Using the same game/model interface for other scenarios unrelated to Sudan • Testing the most effective combinations of interventions developed by people against those developed by AI & brute force simulation. February 2011 Peter Landwehr 26 Thanks! Center for Computational Analysis of Social and Organizational Systems http://www.casos.cs.cmu.edu/ References (1 of 2) • Andy007, “GameFAQs: SimCity 2000 (PC) Scenario Guide,” GameFAQs, 2000. [Online]. Available: http://www.gamefaqs.com/pc/198648-simcity- 2000/faqs/7283. [Accessed: 24-Jan-2011]. • S. Cooper, A. Treuille, J. Barbero, A. Leaver-Fay, K. Tuite, F. Khatib, A. C. Snyder, M. Beenen, D. Salesin, D. Baker, and Z. Popović. 2010. The challenge of designing scientific discovery games. In Proceedings of the Fifth International Conference on the Foundations of Digital Games (FDG '10). ACM, New York, NY, USA. • International Crisis Group, Sudan: Defining the North-South Border. Juba/Khartoum/Nairobi/Brussels: International Crisis Group, 2010. • International Crisis Group, Sudan: Regional Perspectives on the Prospect of Southern Independence. Nairobi/Brussels: International Crisis Group, 2010. • International Crisis Group, Sudan: Breaking The Abyei Deadlock. Juba/Khartoum/Nairobi/Brussels: International Crisis Group, 2007. February 2011 Peter Landwehr 28 References (2 of 2) • C. Pearce, “Sims, BattleBots, Cellular Automata, God and Go: A Conversation with Will Wright,” Game Studies, vol. 1, no. 2, Sep. 2001. • P. Starr. March 21, 1994. “Seductions of Sim: Policy as a Simulation Game.” In The American Spectator. http://www.prospect.org/cs/articles?article=seductions_of_sim • L. von Ahn and L. Dabbish. 2004. Labeling images with a computer game. In Proceedings of the SIGCHI conference on