Maintaining trust when agents can engage in self-deception Andres´ Babinoa,b,1,2, Hernan´ A. Maksec,d,1, Rafael DiTellae,f,g,1, and Mariano Sigmanh,1 aDepartamento de F´ısica J.J. Giambiagi, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, Buenos Aires 1428, Argentina; bInstituto de F´ısica de Buenos Aires, Consejo Nacional de Investigaciones Cient´ıficas y Tecnicas´ (CONICET), Buenos Aires 1428, Argentina; cLevich Institute, City College of New York, New York, NY 10031; dPhysics Department, City College of New York, New York, NY 10031; ePolitical Economy Group, National Bureau of Economic Research, Cambridge, MA 02138; fSocial Interactions, Identity and Well-being Program, Canadian Institute for Advanced Research, Toronto, ON M5G 1M1, Canada; gGovernment and the International Economy Unit, Harvard Business School, Boston, MA 02163; and hLaboratorio de Neurociencia, CONICET, Universidad Torcuato Di Tella, C1428BIJ Buenos Aires, Argentina Edited by Albert-Laszl´ o´ Barabasi,´ Northeastern University, Boston, MA, and accepted by Editorial Board Member David A. Weitz July 17, 2018 (received for review February 28, 2018) The coexistence of cooperation and selfish instincts is a remark- of Projection by which our actions affect how we think of oth- able characteristic of humans. Psychological research has unveiled ers (11, 12) is at the same time intuitive and paradoxical. From the cognitive mechanisms behind self-deception. Two important a rational perspective, beliefs about others should be based on findings are that a higher ambiguity about others’ social pref- what they have done, not on what we have done to them. How- erences leads to a higher likelihood of acting selfishly and that ever, it has been observed that subjects in economic games not agents acting selfishly will increase their belief that others are only take into account the previous actions of other players, but also selfish. In this work, we posit a mathematical model of also their past actions (13, 14). Additionally, people’s beliefs also these mechanisms and explain their impact on the undermin- depend on their own previous actions (10). ing of a global cooperative society. We simulate the behavior of Here, we use the colloquial term Paranoia to refer to P2 (the agents playing a prisoner’s dilemma game in a random network idea that if there is ambiguity about how another person may of contacts. We endow each agent with these two self-deception act, an agent will sample the distribution biased for the worse mechanisms which bias her toward thinking that the other agent outcomes). Closely related to P2 is the mechanism of “catego- will defect. We study behavior when a fraction of agents with the rization” and “malleability” (15). For example, stealing a pen is “always defect” strategy is introduced in the network. Depending more malleable than stealing the money needed to buy the pen. on the magnitude of the biases the players could start a cascade of Similarly, the distribution of beliefs on the moral judgment of the defection or isolate the defectors. We find that there are thresh- malleable case (stealing the pen) is ambiguous, and hence people olds above which the system approaches a state of complete may use this ambiguity in their favor to act more selfishly. distrust. The aim of this work is twofold: first, to provide a mathe- matical description of these self-deception mechanisms (Para- behavioral economics j cognitive neuroscience j corruption j noia and Projection); and second, based on this mathematical cooperation j self-deception description, to investigate the impact that they may have on the evolution of trust among the agents of a society. ndividuals often deviate from the behavior that maximizes The Model Itheir material reward (1, 2). For example, in the ultimatum We study a set of 105 interacting agents that play a modified Pris- game, people prefer to reject profitable offers that they consider oner’s Dilemma (SI Appendix, section 1.1) game against each unfair (3). This behavior, and other phenomena such as fairness other in a static random network. The main difference from or cooperation (2, 4), can be accounted for within a rational other similar approaches investigating networks and evolution of model that includes broader objectives or “social preferences” (altruism, fairness concerns, etc.) as part of the function which agents seek to optimize. Significance Naturally, agents seek to reduce the problems that arise when material rewards collide with social preferences. For example, “He who wants to kill his dog accuses it of having rabies,” believing that others are altruistic may make it more difficult for the French proverb says. The fact that we alter our beliefs an agent to act selfishly which, in turn, may reduce its monetary about others to act selfishly and, at the same time, keep a payoff. A way of solving this tension is to develop a self-serving positive self-view has been widely studied by behavioral sci- bias: that is, to believe that others are not altruistic to “justify” ences. Here, we propose a mathematical description of two of a selfish act. Cognitive dissonance theory (5, 6) aims to explain these mechanisms of altering beliefs and study a simulation of the emergence of belief with self-serving biases. The idea is a society of agents provided with these biases. We find that that dissonance (contradiction) between cognitions is psychologi- there are sets of parameters that make societies propagate cally uncomfortable, and so it triggers mechanisms of dissonance defection actions and others that protect them from spreading reduction—and one way of doing so is by altering beliefs (7, 8). malicious behavior. Self-deception mechanisms have been broadly studied in eco- Author contributions: A.B., H.A.M., R.D., and M.S. designed research; A.B. performed nomics (2, 9). Recently, using an experimental design called research; A.B. analyzed data; and A.B., H.A.M., R.D., and M.S. wrote the paper. “The Corruption Game,” we demonstrated two of these prin- The authors declare no conflict of interest. ciples (10): This article is a PNAS Direct Submission. A.-L.B. is a guest editor invited by the Editorial Principle 1 (P1) Selfish action alters beliefs about others’ social- Board. preferences. Published under the PNAS license. P2 Ambiguity regarding others’ social preferences 1 A.B., H.A.M., R.D., and M.S. contributed equally to this work.y increases the likelihood of acting selfishly. 2 To whom correspondence should be addressed. Email: [email protected] This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. We use the term Projection to refer to P1, which is a trait that 1073/pnas.1803438115/-/DCSupplemental. describes how people blame others for their actions. The notion Published online August 13, 2018. 8728–8733 j PNAS j August 28, 2018 j vol. 115 j no. 35 www.pnas.org/cgi/doi/10.1073/pnas.1803438115 Downloaded by guest on September 28, 2021 Z 1 cooperation (or corruption or reputation) (16–19) is that here, θ^= θ Betaθ(a, b) dθ, we used a Bayesian updating rule and inference process (similar 0 to ref. 20). This rule was necessary to generate a mathematical model of cognitive biases Paranoia and Projection and study their where a and b are, respectively, the numbers of observed defec- impact on the propagation of strategies. ^ For clarity, we divide the strategy of the agents into three tions and cooperations. As a increases, respect to b, θ approaches stages (Fig. 1): observation, inference, and decision. Observa- 1, indicating that the agent believes that the other agent is likely tion is the process of accumulation of information about other to choose to defect. In this model, each agent has a specific belief agent’s actions. The inference process uses observed informa- distribution for each other agent she interacts with. tion and combines it with priors to generate—using a Bayesian Paranoia and Projection have a different effect on the infer- model—a belief about other agents’ behavior. This stage is mod- ence process. Broadly, Projection changes the beta distribution eled as a beta-binomial process (SI Appendix, section 1.3). The (as if own actions were fragments of observed actions), and output of the inference process is the expected reward for each Paranoia results in sampling unevenly (focusing on the worse possible action. Finally, in the decision stage, the agent chooses outcomes) of the beta distribution. the option that maximizes her expected reward. Projection is a trait that describes how people blame oth- Under these settings, the agents in the network will end up ers for their actions. Although an ideal observer constructs this defecting or cooperating with each other depending on the initial distribution only from priors and observations, to model this conditions. Our primary goal is to investigate how incorporat- characteristic, each time an agent defects, she modifies her beta ing the cognitive biases described in the introduction (Paranoia distribution of beliefs. With Projection the actions of the agent and Projection) affect the evolution of cooperation or defection impact on the resulting Beta distribution, which she then uses to in the network. In the next subsection, we explain how these estimate the probability of defection or cooperation. cognitive biases can be incorporated into a Bayesian inference Specifically, this is done by changing, whenever the agent process. defects, the a parameter (which measures the number of The essential step in the inferential process in our model is observed defections) of the Beta distribution. How much a is the estimation of an agent’s probability of defection, θ, in a changed each time the agent defects is scaled by the parame- given interaction—or equivalently, the probability of coopera- ter Projection in such a way that if Projection = 1 defecting on Beta tion pc = 1 − θ.