Computational Models for Belief Revision, Group Decision-Making and Cultural Shifts MURI ANNUAL PERFORMANCE REPORT 2008-09

AFOSR Grant No. A9550-05-1-0321 (Start Date: 1 May 05) PRINCIPAL INVESTIGATOR: Professor Whitman Richards, 617-253-5776, [email protected] LEAD INSTITUTION: Massachusetts Institute of Technology AFOSR PROGRAM MANAGER: Dr. Terence Lyons, 703-696-9542, [email protected]

NOTE: This performance report covers progress and changes that have occurred in the Option Period from 1 Nov 08 - 31 Oct 09.

PROGRAM OBJECTIVE: The primary objective of this MURI is (1) to explore how beliefs support and lead to certain actions in one culture but not another, and (2) to develop computational models that further our understanding of the relation between beliefs, decisions, and actions. A key requirement for such models is to distinguish the different roles played by sacred values versus instrumental or secular values, which differ widely across cultures. The goal of these models is to provide formal explanations for how the beliefs of individuals affect group and individual actions, and how groups evolve. Such models are an important step toward understanding and predicting the dynamics and actions of groups. They are fundamental to an understanding of how actions of a group may be altered by belief revision, by either internal or external pressures (including force). They also will be needed for strategic reasoning in negotiations, where beliefs in different cultures may lead to what appears to be irrational proposals, yet are seen as rational in that competitive culture. This MURI initiative supports DoD’s need to understand how terrorist cells are created and how they evolve, thus providing models for the underlying seeds, which can be more easily identified early in the development process. In turn, the models may suggest strategies that will help revise or manipulate belief structures to reduce the likelihood of militant cells forming within a culture. The research is also very relevant to negotiations between states having different cultures.

CURRENT MURI CONSORTIUM RESEARCH TEAM MEMBERS *: • Massachusetts Institute of Technology (PI) Whitman Richards, Prof. of Cognitive Science, Computer Science and Artificial Intelligence Laboratory (CSAIL), Joshua Tenenbaum, Assoc. Prof. of Computational Cognitive Science, Patrick Winston, Prof. of Computer Science. CSAIL • *Harvard University, Avi Pfeffer, Assoc. Prof. of Computer Science • University of Michigan, Scott Page, Prof. of Economics & Complex Systems, Jenna Bednar, Assoc. Prof. of Political Science • Northwestern University, Kenneth Forbus, Prof. of Computer Science, Douglas Medin, Prof. of Psychology • CUNY, John Jay Center for Terrorism & Univ. Mich., Scott Atran, Prof. of Anthropology, Psychology and Public Policy • Georgia Tech, Jeff Shamma,, Prof. Electrical & Computer Engineering • North Carolina State, Jon Doyle, Institute Prof. of Computer Science • Georgetown, John Mikhail, Assoc. Prof. Law & Philosophy (Ethics) • *ARTIS Research, R. Davis, CEO

Consultants: Robert Axelrod, Prof. of Political Science and Public Policy, University of Michigan Marc Sageman, MD, PhD; Sageman Consulting LLC; Univ. Penn. Adjunct Prof., Dept. Psychiatry; Rajesh Kasturirangan, PhD., National Inst. Adv. Studies, Bangalore; J. J. Koenderink, Tech Univ. Delft (formerly Prof. of Physics, Helmholtz Institute, Utrecht)

*NOTE: Last June, Prof. Pfeffer joined Charles River Analytics, but still retains a secondary affiliation with Harvard. ARTIS was granted a subcontract in June 09 for “Case-Based Reasoning in Conflict Management”. This subcontract extended the MURI team to include other young Investigators advised by Axelrod & Atran, among others.

1 FINANCIAL: (as of 1 Nov 09): 3-YEAR BASE PERIOD – FY05: $669,604 (5 months) FY06: $1,139,405 FY07: $1,183,188 & FY07: 71,181 (additional funding); FY08: $1,269,564 ; FY09: $1,276,255; FY10: [$604,484 (7 months) not yet authorized as of 14 Dec 09.]

To-Date (14 Dec 2009) Base and Option Period funds of $5,310,323* have been expended and are “off the government’s book.” This figure takes into account invoices from the subawardees and consultants through the period 31 Oct 09, leaving an unexpended balance of about $904,000. The current commitments to the set of present PI’s for the period 1 Nov 09 through 14 May 10 are estimated to be $835,000. This will leave an unexpended balance of $69,000 at the end date of 5/14/10. We have requested a no-cost extension of the MURI to 31 Jul 10, using this balance for summer support of PI’s and students. *Based on our current projections, and based on the fact that we are still waiting on invoices from subawards.

SCIENTIFIC APPROACH: As mentioned, a major goal of the MURI is to develop computational models that further our understanding of the relation between beliefs, decisions and actions in different cultures. Our aim is to provide predictive, rather than just descriptive models. For example, rather than simply describing the structure of a militant or non-violent network (Atran et al), we have made strides in understanding the processes that underlie their formation and evolution (Richards et al.) Understanding the origin and nature of shared and revised beliefs among group members is critical (Page, Bednar, Doyle) Unfortunately, data are scarce, and hence an important component of our effort is the reliable documentation of the formation of groups such as those engaged in the acts at Madrid, London, Bali, etc. The research led by Scott Atran under our MURI currently provides the most detailed and reliable set of data available, and this work continues with more details and studies of a dozen or so such networks. Additional data continue to be collected in non-violent settings (Medin et al.), such as the Wisconsin Menomenee, the Guatemalan Ladinos, and Amish and Muslim communities. These studies of non-violent communities allow closer examination of moral issues and sacred vs. secular values – the latter being critical in both belief revision and negotiations across cultures (see below). Because the origins of many beliefs rest in the traditions of a culture, we are formalizing how beliefs, actions and values are expressed in stories within a culture (Finlayson, Forbus). The framework for representing stories will support rapid, automatic analysis of the semantic interpretation of text, such as excerpts from news sources. Implications for future trends can be explored through analogies with traditional stories from the relevant culture. Finally, individuals or groups striving for social change need to have strategies for actions. We are developing and testing new frameworks for understanding strategic reasoning, subject to beliefs, which dictate in part which moves are considered rational (Pfeffer, Gal). These complement the more traditional “game-theoretic” approaches, and appear more plausible in many real world, multi-agent scenarios.

IMPORTANT TRNSITION EVENTS

• As in previous years, Atran et al continue to be sought after for briefings to various Government agencies. In March 2009, Washington, DC - George Mitchell acknowledged receipt of a summary of Atran’s work on sacred values with respect to the Israel-Palestine conflict and wrote: “Your views will receive my careful consideration as I embark on this assignment” (letter dated March 18, 2009 written from the office of George Mitchell at the State Department). In March 2009, Foreign Minister Tzipi Livni and other members of Knesset, were briefed on research on sacred values in the Israel. In July 2009 there was a briefing in Ankara at a Turkish National Police workshop (for possible extensions of our research into that country) and the U.S. ambassador to Turkey. Briefings also provided by the Honorable Juan Zarate and the Lord John Alderdice. Finally, in October 2009, Washington DC. – there was an Interagency/interdepartmental workshop with briefings for ONR, AFOSR, DHS, NSF. A summary of findings up to 2007 can be found http://www.edge.org/3rd_culture/Atran07/index.html.) • Finlayson’s Story Workbench for the semi-automatic annotations of text is still under beta testing, but with additional users. Its Java wordnet interface subroutine JWI now has increased form 1000 to over 3000 downloads. • Users of Gal’s multi-agent Colored Trails platform for studying strategic reasoning has now expanded from six to over a dozen universities (including MURI recipients Univ. Maryland & Carnegie Mellon.) A few users are international and include the European Space Agency, Bar Ilan University, the American University, Beirut, and Utrecht,

Other Scientific Accomplishments in ’08 – ‘09 (please see last year’s annual report for Option Period review for complete listings for previous years)

2 • Atran, Medin & Ginges have completed an important study showing negotiations that trade sacred for secular assets may create impediments to any further productive negotiations. Hence culturally-sensitive negotiations can not be treated in the same manner as one would by an economist concerned only with monetary assets. This work provides one of the first clear findings in a Middle East context. Resolution of quarrels arising from conflicting scared values may require concessions that acknowledge the opposition’s core concerns (Atran, Axelrod, Davis.) • Atran & Axelrod have begun the development of an approach to framing (and reframing) of contentious values to promote negotiations in seemingly intractable conflicts and to understand competitive framing in pursuit of political goals. • Axelrod (with Davis and Atran) have adapted a case-based reasoning approach to analyze historical events for “framing”, using the 1979 Iraq hostage crisis as an illustration. • Medin’s group contiues the studies of Hindu-Muslim conflict over sacred land in Northern India. Some of he results mirror those of Ginges et al (2007), where moral absolutists were again more accepting of a symbolic tradeoff than a taboo tradeoff. Also shown is a link between group identity and sacred values. A surprising finding was that regarding the issue of sacred sites, the Hindus were vehemently opposed to tradeoffs involving monetary incentives relative to symbolic tradeoffs, but Muslims seemed to prefer them (Sachdeva & Medin, 2009.) • Field studies in Wisconsin that were conducted among Amish, Native Americans and Caucasians revealed that many participants felt that certain moral decisions should NOT be based on assessing costs and benefits, but rather in doing the RIGHT thing (or doing God’s will.) Yet many also expressed concern about long term consequences. This suggests that sacred values may NOT motivate a lack of concern with utility, and raises the possibility that some of these participants perceive or seek utility in ways that differ from researcher’s preconceptions. • Work on mental models continues in Guatemala. Because there is fast socioeconomic and ecological change, these studies are revealing dynamic aspects of beliefs and knowledge about the environment (LeGuen et al, 2010.) Specifically, the new generation of Itza’ and Ladinos show dramatic differences compared to their parents. The changes demonstrate a complex interaction between demographic, economic and environmental factors on the one hand, and cultural values and beliefs on the other. • Joseph’s studies in the Bridgeview Muslim community in Illinois have been focusing on the psychosocial adaptation of American Muslim youths. (See Joseph refs.) • Mikhail has assembled a unique database of legal codes from 163 jurisdictions around the world. For the first time, researchers can now describe, analyze, and make generalizations about the precise scope and function of one important legal issue -- homicide prohibition in human societies. • Kasturirangan (with Raghavan) has developed a theoretical framework that links tacit beliefs (see Gal refs.) and entrenched narratives. The idea is that many tacit beliefs – those acquired effortlessly without explicit teaching – become entrenched by repetition. Examples would be religious rituals, food norms, and some (culturally- dependent) social protocols. To illustrate the framework, the authors show how strategies used in the 1965 war between India and Pakistan was influenced by entrenched beliefs about the martial capacity of Hindus led to and entrenched narrative about the superiority of Pakistani forces in a potential conflict. This misconception was influential in Pakistani decisions regarding the conflict. • Doyle continues the development of a theoretical model for belief change – one that allows the assessment of the character and difficulty of belief change. The model formalizes the ways in which choices to believe or reject new information can depend on preferences based on the types of sources of belief (religious, familial, social, default, etc.) • Page & Bednar continue to develop the concept of institutional path dependence and the relation between beliefs, framings, values and behaviors. For example, when we see an individual or a group respond in some way to an institutional intervention, how do we dissect the extent to which those responses are due to beliefs, or to cognitive frames, to values, or to behavioral repertoires? • Gal, Pfeffer, Kasturirangan & Richards have developed a model of belief and action propagation in graphs, describing how people make decisions based on knowledge that is inferred by observing the actions of others (See Gal et al.) This kind of knowledge is called “tacit.” • The Colored Trails platform has been upgraded by Gal to allow developers and collaborators to access a project’s source code and documentation for a single location, allowing interested parties to easily monitor changes. This platform is now being used to study bargaining behavior in social networks (MIT-Media Lab & Harvard), to explore the generosity, reliability and trust in negotiation in culturally mixed networks (Harvard & Bar Ilan Universities), to study social dilemmas in settings where participants need to form teams in order to complete tasks with other people as well as computer agents (Utrcht University).

3 • Pfeffer has developed the first computer system that presents advice to human decision makers when they are reasoning in sequential games (See Antros et al.) The advice is based on a theory of reasoning in games. Players using this system played better than those without it. • Pfeffer has also continued work on the representation, reasoning, and learning of games in networks. Unlike most research, the emphasis is on multiplayer games, especially those who interact in some sort of social network. Two representations for such games are action graph games and graphical games (See Jiang et al.) • Shamma has been exploring the utility of feedback control in shaping asymptotic collective behavior in complex adaptive systems. Part of this effort focuses on the distributed reinforcement of efficient outcomes in multi-agent coordination problems. These studies are motivated by research on social network formation, and the notion of efficient networks, where agents have discretion over which links to form. • Shamma’s group is also investigating the utility of robust control over preferences in social networks. • The model for small group evolution developed by Richards & Wormald has now been extended to reveal the “fine structure” of a network, namely the “motifs” that are the elemental subgraphs. With MacIndoe, over two dozen large networks have been explored. • Baker (Tenenbaum’s group) continues to explore the relation between beliefs and actions using a Bayesian Framework. Here the emphasis is on modeling an individual’s ability to understand another’s plans for actions. • Forbus’ Qualitative Concept Map system (QCM) has been used to build models of transcript data, providing a format useful for analogical reasoning by machine. The Explanation Agent NLU system also continues to be developed – a critical tool to reasoning about moral decisions that involve alternate possible worlds and counterfactuals, as well as logical and numerical quantification. • Forbus’s group has been modeling the role of cultural narrative in understanding moral situations. They have showed how access to different moral stories can influence the use of canonical moral narratives. The experiments involve Iranian and American participants. • Dehghani and Forbus continue to develop their MoralDM computational model of moral decision-making. The model has now differentiated between variants of three stories, matching the results of the Iranian participants. For each variant the model had to judge whether the protagonist should take the sacrificial choice or the choice that maximized personal utility. • Finlayson has almost completed work on Analogical Story Merging that derives a morphology from a set of stories. It incorporates Forbus & Gentner’s Structure Mapping Engine (also developed on this MURI), as well as Bayesian model merging. • Koenderink & Richards have designed the first multi-level cellular automata suitable for exploring the effect of different “rules” that describe behaviors of (1) individuals (who vote according to their preferences), (2) parties (to which the individuals belong) and (3) a government (that enforces social norms or institutional constraints.)

Student and Postodoctoral support over portions of the base period include: MIT: 4 Phd, 2 MS Graduate Research Assistants and 1 PostDoc; UMich: 5 PhD; Harvard: 2 PhD and 2 PostDoc; John Jay: 2 PhD , Northwestern: 5 PhD and 2 PostDoc.

WORKSHOPS 1. Advancing Computational Models of Narrative (Richards, Finlayson, & Winston) Oct 12-14 ’09 (Report forthcoming) 2. Patterns of Influence (postponed to Fall 2010 (W. Richards organizer) 3. Violent & Non-violent Group Formation (optional)

PUBLICATIONS and PRESENTATIONS: (08-09 only) In 2008-09, over 50 papers have appeared in refereed journals or refereed proceedings, or are under review, and over 20 presentations at national and international meetings. There are about 15 working papers in progress,

Books Atran, S. Talking to the Enemy: The Dreams, Delusions and Science of Sacred Conflicts, New York: HarperCollins (Ecco press), Paris: Editions Denoel, forthcoming.

Atran, S. & Medin, D. (2008). The Native Mind and the Cultural Construction of Nature. Cambridge, MA: MIT Press.

4 Papers Atran, S. (January 2009) Who Becomes a Terrorist Today? Perspectives on Terrorism, 2 (5). (http://www.artisresearch.com/articles/Atran_2008_Who_becomes_a_terrorist_today.pdf)

Atran, S. (2009) A Question of honor: Why the Taliban fight and what to do about it. (under review, based on our sacred values research).

Atran, S. (August 2008) “Band of Brothers”: Civil society and the making of a terrorist. The International Journal of Not-for-Profit Law, 10 (4),. (http://www.icnl.org/knowledge/ijnl/vol10iss4/art_3.htm)

Atran, S. & Axelrod, R. (2008). Reframing sacred values. Negotiation Journal, 24(3), 221-246, available at, http://sitemaker.umich.edu/satran/files/negj0708.pdf

Atran, S. & Henrich, J. The Evolution of Religion. PNAS (Proceedings of the National Academy of Sciences), forthcoming.

Atran, S., Magouirk, J. & Ginges, J. (2008). The radical madrassah factor. CTC Sentinel (Combating Terrorism Center, West Point) 1(3) online, available at, http://www.ctc.usma.edu/sentinel/default.asp Baker, C.L., Saxe, R., & Tenenbaum, J.B. (2009). Action Understanding as Inverse Planning. Cognition, 113, 329- 349.

Bartels, D., Bauman, C. , Skitka, L.J. & D. L. Medin (Eds.), Psychology of Learning and Motivation, Vol. 50: Moral Judgment and Decision Making. San Diego, CA: Academic Press.

Bartels, D. M. & Medin, D. L. (2009). Are morally-motivated decision makers insensitive to the consequences of their choices? Psychological Science Bednar, J., Bramson, A., Jones-Rooy ,A. & S E Page "Emergent Cultural Signatures and Persistent Diversity: A Model of Conformity and Consistency." (5/14/08; under review)

Bednar,J., Chen, Y., Liu,X. & S Page. "Behavioral Spillovers in Multiple Games: An Experimental Study." (1/15/09; under review) Books Bennis, W. M., Medin, D. L., & Bartels, D. M.(in press). The costs and benefits of calculation and moral rules. Perspectives on Psychological Science.

Broniatowski, D. and Kasturirangan, R. (2009). A Graph-Theoretic Analysis of Artifacts from Three Religions’ Mystical Traditions. Submitted to The Journal of Cognition and Culture.

Chasparis, G. and Shamma, J., “Distributed Dynamic Reinforcement of Efficient Outcomes in Multiagent Coordination and Network Formation,” (submitted for publication)

Chasparis, G. and Shamma, J., “Control of preferences in social networks under duopoly and network uncertainty.” (work in progress) Dehghani, M., Iliev, R., Sachdeva, S., Atran, S., Ginges J. & Medin, D. (2009) Emerging sacred values: The Iranian nuclear program (under review).

Dehghani, M., Forbus, K., Tomai, E., Klenk, M. (in press). An Integrated Reasoning Approach to Moral Decision- Making. In Anderson, M., Anderson, S (Ed.) Machine Ethics. Cambridge University Press

Doyle, J. Mechanics and mental change, to appear. Andrew Wicker, Influence Models in Social Networks, NCSU Computer Science Ph.D. proposal, in preparation.

Finlayson, M.A. Deriving Narrative Morphologies via Analogical Story Merging, in New Frontiers in Analogy Research (Proceedings of the 2nd International Conference on Analogy), edited by Boicho Kokinov, Keith Holyoak,

5 and Dedre Gentner, New Bulgarian University Press, Sofia, pp. 127-136. http://www.mit.edu/~markaf/doc/finlayson.kokinov.nbupress.2009.137.pdf Fischhoff, B., Atran, S. & Sageman, M. (2008). M. Mutually Assured Support: A security doctrine for terrorist nuclear weapons threats. Annals of the American Academy of Political and Social Science (Special Issue: Terrorism Briefing for the New President) 618, 160-167, available at, http://sitemaker.umich.edu/satran/files/fischhoff_atran_sageman_published_mas.pdf

Gal, Y & A. Pfeffer Networks of Influence Diagrams: Reasoning About Agents' Beliefs and Decision-Making Processes Journal of Artificial Intelligence 33:109-147, 2008. Ginges, J. & Atran, S. (in press). What motivates participation in violent political action: selective incentives or parochial altruism? Annals of the New York Academy of Sciences.

Ginges, J. & Atran, S. (in press). Flexibility and resistance to compromise over sacred values: An Indonesian case study, In D. Medin (ed.)

Ginges, J. & Atran, S. (2008). Humiliation and the inertia effect: Implications for understanding violence and compromise in intractable intergroup conflicts. Journal of Cognition and Culture, 8, 281-294

Ginges, J. & Atran, S. (2009). Noninstrumental Reasoning over Sacred Values: An Indonesian Case Study. In D. Bartels, C. Bauman, L. Skitka, & D. Medin (eds.): The Psychology of Learning and Motivation. Vol 50, Chpt 6: 193- 206.

Ginges, J., Hansen, I.G. & Norenzayan, A. (2009). Religion and popular support for suicide attacks. Psychological Science, 20, 224-230. (http://idisk.mac.com/jeremyginges1-Public/suicideattacks.pdf)

Haidt, Jonathan, Jesse Graham, and Craig M. Joseph (2008). “Above and Below Left-Right: Ideological Narratives and Moral Foundations.” Psychological Inquiry 19(4) (special issue on political ideology), November 2008.

Iliev, R., Sachdeva, S., Bartels, D. M., Joseph, C. M., Suzuki, S. & Medin, D. L. (forthcoming). “Attending to Moral Values.” In D. M. Bartels, C. W. Bauman, L. J. Skitka, & D. L. Medin (Eds.) Moral Judgment and Decision Making: The Psychology of Learning and Motivation, Vol. 50. San Diego: Elsevier. Jiang, A.X., Leyton-Brown, K. & A. Pfeffer. Temporal Action-Graph Games:A New Representation for Dynamic Games. Uncertainty in ArtificialIntelligence (UAI), 2009. Available from http://www.cs.mcgill.ca/~uai2009/proceedings.html. Joseph, Craig M. (in press). “Emotions, Intuitions, and Virtues: Ethics and Human (Second) Nature.” In Verena Mayer and Mikko Salmela (Eds.), Emotions, Ethics, and Authenticity. Amsterdam: John Benjamins Publishing Company.

Joseph, C. & B. Riedel (2008) Islamic Schools, assimilation, and the concept of Muslim Amereican character. In: Being and Belonging: Muslims in the United States since 1911” K. Ewing (ed). Russell Sage Fdn Press, NY Kamar, E., Gal, Y., & B. Grosz. Modeling User Perception of Interaction Opportunities for Effective Teamwork. Symposium on Social Intelligence and Networking. IEEE Conference on Social Computing. August 2009, Vancouver, British Columbia.

Kemp, C. & J. Tennenbaum (2008) The discovery of structural form. Proc. Nat. Acad. Sciences 105, 10687-10692.

Kasturirangan, R. and Raghavan, S. (2009). Tacit Beliefs and Entrenched Narratives: A Cognitive Study of India- Pakistan Relations. Submitted to Language and Cognition. LeGuen, O., Iliev, R., Lois, X., Atran S., & Medin, D. (provisionally accepted) A Garden Experiment Revisited: Inter- generational Changes in the Sacred and the Profane in Petén, Guatemala. Current Anthropology

Magourik, J., Atran, S. & M. Sageman (2008) Connecting terrorist networks. Studies in Conflict and Terrorism, 31, 1- 16.

6 Page, S. E. & R. Golman “General Blotto: Games of Allocative Strategic Mismatch” Public Choice Vol 138, pp 279 - 310, 2009. Page, S.E. & & R. Golman “Basins of Attraction and Equilibrium Selection Under Different Learning Rules” Evolutionary Economics , in press.

Page, S E. & L. Hong “Interpreted and Generated Signals” forthcoming Journal of Economic Theory

Page, S. E. & R. Golman “Individual and Cultural Learning in Stag Hunt Games With Multiple Actions” forthcoming in Journal of Economic Behavior and Organizations

Richards, W. and O. MacIndoe. The fine structure of social networks. (forthcoming MIT-CSAIL Tech Report.)

Sachdeva, S., Iliev, R., & Medin, D. L. (2009). Sinning Saints and Saintly Sinners: The Paradox of Moral Self- Regulation. Psychological Science, 20(4), 523–528.

Shweder, Richard A., Jonathan Haidt, Randall A. Horton, and Craig Joseph (2008). “The Cultural Psychology of the Emotions: Ancient and Renewed.” In Michael Lewis, Jeannette M. Haviland-Jones, and Lisa Feldman Barrett (Eds.), The Handbook of Emotions (3rd edition). Guilford, CT: Guilford Press. Ullman, T.D., Baker, C.L., Macindoe, O., Evans, O., Goodman, N.D., & Tenenbaum, J.B. (2010). Help or hinder: Bayesian models of social goal inference. In Advances in Neural Information Processing Systems (Vol. 22). Cambridge, MA: MIT Press.

Conference Proceedings & Presentations (08 – 09 only)

Antos, D. & A. Pfeffer (2008) Identifying reasoning patterns in games. 23rd Conf on Uncertainty in AI. Antos, A & A. Pfeffer. Using Reasoning Patterns to Help Humans Solve Complex Games. International Joint Conference on Artificial Intelligence (IJCAI), 2009. Available from http://ijcai.org/papers09/contents.php.

Baker, C.L., Goodman, N.D., & Tenenbaum, J.B. (2008). Theory-based Social Goal Inference. In Proceedings of the Thirtieth Annual Conference of the Cognitive Science Society (pp. 1447-1452).

Chasparis, G. and Shamma, J., “Efficient Network Formation by Distributed Reinforcement,” 47th IEEE Conference on Decision and Control, Cancun, Mexico, Dec 8-11, 2008. Dehghani, M., Iliev, R., Kaufmann, S. (2009). Causal Explanations in Counterfactual Reasoning. Proceedings of the 31th Annual Conference of the Cognitive Science Society (CogSci09).

Dehghani, M., Gentner, D., Forbus, K., Ekhtiari, H., Sachdeva, S. (2009). Analogy and Moral Decision Making. Proceedings of the 2nd International Analogy Conference. Sofia, Bulgaria.

Dehghani, M., Forbus, K. (2009). QCM: A QP-Based Concept Map System. In Proceedings of the 23nd International Workshop on Qualitative Reasoning (QR09).

Dehghani, M., Sachdeva, S., Ekhtiari, H., Gentner D., Forbus, K. (2009). The Role of Cultural Narratives in Moral Decision Making. In Proceedings of the 31th Annual Conference of the Cognitive Science Society (CogSci09) Doyle, J. Cognitive Mechanics: Natural Intelligence Beyond Biology and Computation, Proceedings of the AAAI Fall Symposium on Naturally-Inspired Artificial Intelligence, Arlington, Virginia, November 7-9, 2008, pp. 35-37, AAAI Technical Report FS-08-06. Ficici, S.G. and Pfeffer, A. (2008) Modeling how Humans Reason about Others with Partial Information, Seventh International Conference on Autonomous Agents and Multiagent Systens (AAMAS).

Ficici, S.G. and Pfeffer, A. (2008) Simultaneously Modeling Humans' Preferences and their Beliefs about Others' Preferences, Seventh International Conference on Autonomous Agents and Multiagent Systens (AAMAS).

7 Finlayson, M. A. (2008) Collecting Semantics in the Wild: the Story Workbench. Workshop on “Naturally Inspired AI”, Wash. DC Nov 08; Tech Report FS-08-06, AAAI Press., Menlo Park CA

Finlayson 2009 "Deriving Narrative Morphologies via Analogical Story Merging" at the MIT Workshop onComputational Models of Narrative, Beverly, MA, 9 October 2009

Finlayson 2009 "Deriving Narrative Morphologies via Analogical Story Merging" at the Second International Conference on Analogy, Sofia, Bulgaria, 25 July 2009

Finlayson 2009 "Story Basis Sets and Intermediate Features" at the DARPA Workshop on EN-Mem Systems, Washington DC, 7 February 2009.

Forbus, K. , Klenk, M. & T. Hinrichs. (2008) Companion Cognitive Systems: design goals and some lessons learned. Workshop on “Naturally Inspired AI”, Wash. DC Nov 08; Tech Report FS-08-06, AAAI Press., Menlo Park CA Gal, Y., D'souza, S., Pasquier,P., Rahwan, I. & S. Abdallah. The Effects of Goal Revelation on Computer-Mediated Negotiation. Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci), Amsterdam, the Netherlands, July 2009.

Gal, Y., Kasturirangen, R., Pfeffer, A. & W. Richards. How Tacit Knowledge Guides Action. Symposium on Social Intelligence and Networking. IEEE Conference on Social Computing. August 2009, Vancouver, British Columbia.

Gal, Y. a., Kasturirangan, R., Pfeffer, A., & W. Richards (2009). A Model of Tacit Knowledge and Action. IEEE Computer Society, Los Alamitos, CA, USA. Joseph, C. “Cultural Psychology and the Virtues: Some Theoretical Considerations and Empirical Findings.” Paper presented at the 34th Annual Conference of the Association for Moral Education: “Faith, Democracy and Values: The Challenge of Moral Formation in Families, Schools, and Societies,” University of Notre Dame, November 13 – 16, 2008.

Joseph, C. (2008) Islam, Virtual Theory and the Cognitive Science of Ethics. Ann. Mtg. Amer. Acad, of Religion, Chicao Il (1-3Nov08.)

Joseph, C. Images of the Ideal President and of Actual Presidential Candidates: Evidence from the 2008 Election.Paper presented at the 2008 Annual Meeting of the International Society for Political Psychology, Dublin, Ireland, July 14 – 17, 2009. Kamar, E., Gal, Y., & B. Grosz. Modeling User Perception of Interaction Opportunities for Effective Teamwork. Symposium on Social Intelligence and Networking. IEEE Conference on Social Computing. August 2009, Vancouver, British Columbia.

Koenderink, J. J. & W. Richards (2009) Multi-level cellular automata and social dynamics. Proc. IEEE Int. Conf on Social Computing (SocialCom09), Vancouver, B.C. Also presented at the Annual meeting of the Cog/ Sci Soc., Amsterdam.

Richards, W. & N. Wormald (2009) A representation for small group evolution. Proc. IEEE Int. Conf. on Social Computing (SocialCom09). Vancouver, B.C.

Sachdeva, S. and Medin, D.L. (2009), Group Identity Salience in Sacred Value Based Cultural Conflict: An Examination of Hindu-Muslim Identities in the Kashmir and Babri Mosque Issues. Proceedings of the 31st Annual Meeting of the Cognitive Science Society, CogSci2009. Hillsdale, NJ: Earlbaum.

Tomai, E. and Forbus, K. (2009). EA NLU: Practical Language Understanding for Cognitive Modeling. Proceedings of the 22nd International Florida Artificial Intelligence Research Society Conference. Sanibel Island, Florida.

------For a complete listing of publications, see: http://groups.csail.mit.edu/belief-dynamics

8