You Need to Recognize Ambiguity to Avoid It

Chew Soo Hong,∗ Mark Ratchford† and Jacob S. Sagi‡

Abstract

Experimental investigations of Ellsberg’s (1961) thought experiments yield varying results for the incidence of ambiguity averse choice. Moreover, while intuition might suggest that more “Bayesian” subjects are also more sophisticated than their counterparts, this has not been substantiated to our knowledge. We present two key findings: First, the framing of choices featuring ambiguity significantly affects the frequency of ambiguity averse choice. Second, subjects who consistently assign subjective probabilities to drawing from a deck of cards with unknown composition (“probability-minded” or more “Bayesian” subjects), if anything, appear less sophisticated than those who deem such probabilities to be indeterminate (“ambiguity-minded” subjects). In our study we recast the two Ellsberg urns using equivalent payoff-matrices. Only 13% of our 2506 subjects appeared to fully comprehend this equivalence, and about 1/3 of this subset were probability-minded. Degree of comprehension was significantly and positively associated with a preference for a known versus an unknown bet, suggesting that ambiguity aversion is not driven by lack of sophistication. Among high comprehension subjects, the overall incidence of ambiguity averse choice was consistent with standard experiments and mostly attributable to ambiguity-minded subjects. In particular, 88% of ambiguity-minded subjects opted for the ambiguity averse choice as compared with 28% of probability-minded subjects. Also contrary to intuition, the ambiguity-minded subjects were younger, more educated, more analytic in their approach to problem solving, and more reflective about their choices.

Keywords: Ellsberg Paradox, Uncertainty, Ambiguity, Behavioral

JEL Classiﬁcation number: D03, G02

∗Department of Economics and Department of Finance, National University of Singapore; Department of Eco- nomics, Hong Kong University of Science and Technology. Email: [email protected] †Owen Graduate School of Management, Vanderbilt University. Email: [email protected] ‡Kenan-Flagler Business School, UNC at Chapel Hill. Email: [email protected] 1 Introduction

In his A Treatise on Probability, John Maynard Keynes (1921) stated,

If two probabilities are equal in degree, ought we, in choosing our course of action, to prefer that one which is based on a greater body of knowledge?

Years later, in his doctoral dissertation, Daniel Ellsberg acknowledged Keynes’ contribution. Ellsberg (1961) discussed a thought experiment involving two urns in which the first urn contained 100 red and black balls, exactly 50 of which were red while the second urn contained 100 red and black balls in an unknown ratio. Ellsberg hypothesized that individuals would be indifferent between betting on “red” or “black” in the same urn but might have a strict preference for betting on either color from one urn versus the other. In particular, consistent with Keynes’ conjecture, Ellsberg asserted that a conservative individual might strictly prefer placing a bet on the first urn, where the distribution of balls is unambiguous — a phenomenon later dubbed ambiguity aversion.

Since Ellsberg (1961), a large body of literature has developed in the social sciences theoretically and experimentally exploring the implications of ambiguity aversion. Ellsberg’s two-urn thought experiment has been used to motivate elaborate models for decision-making in the presence of unquantiﬁed uncertainty, as well as explain various aggregate phenomenon.1 Yet, while the evidence for the presence of ambiguity aversion in the two-urn experiment appears to be robust, the same is not true for other experiments featuring choice in the presence of ambiguity. For instance, the rate of consistent ambiguity aversion exhibited in tests of Ellsberg’s (1961) so-called single-urn experiment is far below that reported in two-urn experiments.2 In other words, it might be the case that ambiguity aversion is not robust to framing. Relatedly, it could also be that ambiguity aversion is largely conﬁned to na¨ıve or unsophisticated subjects — a view that might be espoused by proponents of a Bayesian decision making paradigm.

In this study, we investigate the sensitivity of subjects to the framing of the two-urn choice problem. Our primary purpose is to answer the following two questions about “high comprehension” subjects — those who understand the equivalence between the standard

1For a review of contemporary models of ambiguity aversion see Etner, Jeleva, and Tallon (2012). For applications, see references in Ju and Miao (2012). 2In the single-urn experiment, subjects are told that an urn contains 90 balls, exactly 30 of which are red. The remaining balls are either yellow or black in an unknown ratio. Ambiguity aversion is indicated by a preference for betting on red rather than black, while simultaneously preferring to bet that the ball drawn will be black or yellow rather than bet that it will be red or yellow.

1 description of the Ellsberg Paradox and a more complex variant of it:

1. Is ambiguity aversion as prevalent among high comprehension subjects as it is in the typical two-urn experiment?

2. Among the high comprehension subjects, can individuals who exhibit Bayesian tendencies along the lines described by Savage (1954) be described as more sophisticated (i.e., better educated and more thoughtful)?

Both questions are important for economists seeking direction on how to approach the phenomenon of ambiguity aversion. There are, after all, some behaviors exhibited by subjects that economists do not generally ﬁnd worthy of modeling. For instance, evidence for the violation of stochastic dominance and the reduction of compound lotteries also exists (see Kahneman and Tversky, 1979; Tversky and Kahneman, 1986).3 If ambiguity neutrality is the hallmark of sophisticated and thoughtful subjects, while ambiguity non-neutrality is more prevalent among subjects susceptible to framing inconsistencies, then perhaps ambiguity aversion is better viewed as a psychological bias.4 Likewise, investigating whether “sophisticated” subjects are ambiguity averse is germane to understanding the role that ambiguity aversion could potentially have in competitive markets.5 If better education and greater propensity for elaboration are synonymous with ambiguity neutrality, then ambiguity might only play a small role in the setting of prices.6

In our study, to more easily implement the resolution of the bets, we use decks of cards with known and unknown color compositions rather than urns. Moreover, the payoffs to the known versus unknown Ellsberg-type bets are depicted as payoff matrices, illustrated for example in Figure 2. Assuming that subjects understand the payoff matrix and that their preferences are unaltered by the frame, a preference for “K” over “U” is tantamount to a preference for drawing from the known versus unknown urn in the classical 2-urn experiment.

Our experimental results are surprising along several dimensions. Among 2506 subjects taken from the general population, about 13% exhibited “high comprehension” by responding correctly

3In fact, Halevy (2007) shows that violations of the reduction of compound lotteries are correlated with non- neutrality towards ambiguity. 4Charness, Karni, and Levin (2012) cast further doubt on the robustness of ambiguity aversion. In their study, ambiguity-averse subjects could generally be “taught” to become ambiguity-neutral by other ambiguity-neutral subjects. 5Some progress has been made in this direction. Bossaerts, Ghirardato, Guarnaschelli, and Zame (2010) ﬁnd evidence that ambiguity aﬀects prices in the setting of a competitive experimental market modeled after the single- urn problem. 6By “elaboration” we mean cognitive processes such as evaluation, recall, critical judgment, and inferential judgment.

2 and consistently to questions that tested their understanding of the equivalence between the classic Ellsberg choices and those presented via the payoff matrices. Despite the high bar for selection into the “high comprehension” group, preference for the ambiguity averse choice was the same as that typically reported in the literature in the classical 2-urn experiment. Thus, ambiguity aversion is significant among the most thoughtful and educated segment of the general population. We also divided the “high comprehension” subjects into two groups: Those that assigned a likelihood even in the case where likelihoods were not known (consistent with the Savage, 1954, prescription), and those that felt that a likelihood cannot be assigned in such a situation. We label the first group of subjects “probability-minded” and the second “ambiguity-minded”. About 1/3 of the high comprehension subjects were probability-minded. Our next finding was that ambiguity-minded subjects overwhelmingly (88%) opted for a known bet (the ambiguity averse choice) rather than an unknown bet paying slightly more, while this was only true of less than a third (28%) of the probability-minded subjects. Although the preference for the ambiguity averse choice, aggregated over the two groups, is well within experimental findings for the two-urn problem, the individual group statistics lie on opposite extremes: In no other two-urn study we examined is the frequency of ambiguity averse choice as low as in the probability-minded group or as high as in the ambiguity-minded group (see Table 1). This suggests that the population at large could be modeled as consisting of a minority of “Bayesian” types and a majority of “ambiguity averse” types. Both qualitative and psychometric measures suggest that ambiguity-minded subjects appear to be more thoughtful in deliberating over the choice problem than probability-minded subjects. This runs counter to the hypothesis that subjects who are more educated, intelligent, and/or otherwise thoughtful tend more towards ambiguity neutrality.

More generally our study suggests that, while ambiguity aversion is important, task complexity may be an even more important determinant of behavior in the overall population. Among all but the high comprehension subjects, there was a marked decline in ambiguity averse choice relative to existing studies. In other words, ambiguity appears to be less likely to factor into the prediction of choice behavior unless subjects are aware of it. In particular, this provides one answer to the puzzling disparity between ambiguity aversion displayed in the two-urn versus the single-urn choice problems posed by Ellsberg (1961). The latter features a single urn that contains exactly 30 red balls and an unknown proportions of black and yellow balls. Under the hypothesis of ambiguity aversion, subjects prefer betting on “red” rather than “black” while simultaneously preferring to bet on “black or yellow” rather than “red or yellow”. However, this is not always empirically borne out. Table 1 reports the proportion of subjects exhibiting ambiguity aversion in experiments featuring the two-urn problem versus those featuring the one-urn problem. While results vary within each group of experiments, what is striking is the substantially lower average

3 and higher dispersion of ambiguity aversion displayed in single-urn experiments.

A possible reason for the disparity in the ﬁndings reported in Table 1 is that subjects ﬁnd it harder to recognize the ambiguous nature of drawing “red or yellow” in single-urn experiments. One hint of this is the marked increase in the rate of exhibited ambiguity aversion in Slovic and Tversky (1974) where the authors presented arguments for and against ambiguity aversion prior to choice. Another hint comes from the anomalously large rate of exhibited ambiguity aversion demonstrated in the single-urn experiment of Fox and Tversky (1995) who elicit certainty equivalents rather than solicit choice preferences.7 Extrapolating from our study, one would predict that the frequency of ambiguity aversion displayed in the single-urn problem would more closely match that of the two-urn problem among subjects who fully comprehend the ambiguous nature of “red or yellow” versus the unambiguous “black or yellow”.

2 Experiment

Our study was conducted over three experimental sessions and completed by 2506 adult subjects. The ﬁrst session was run using the eLab subject pool at Vanderbilt University’s Owen Graduate School of Management. The second and third sessions were conducted through large commercial survey panel providers. Table 2 reports the aggregated demographics of the three groups. The subjects were predominantly Caucasian, middle-aged residents of English-speaking countries with some college education. A majority, although not overwhelming, were female.

The studies in the first and second experimental sessions were identical with the exception that subjects in the eLab pool were offered the incentive of a chance to “play” their chosen bet for actual stakes while subjects in the second experiment were not. The third study was incentivized and also included a battery of questions useful in constructing psychometric scales. These items appeared at the end of the study and should not have affected the subjects’ responses vis àvis the other studies.8

In all three sessions, subjects were paid a ﬂat participation fee. Subjects in the incentivized sessions who completed the study were entered into one of several drawings for the chance to receive the actual monetary outcome of the choice they made in this experiment. Everyone who completed the study had the same chance of being selected. Random prize drawings were conducted live via Internet webcasts (a link was provided to subjects and the video of the

7Indeed, Trautmann, Vieider, and Wakker (2011) find significant preference reversals — a dissonance between elicited value versus elicited choice — in the context of ambiguity aversion. 8The additional questions were posed to subjects from the first two sessions in separate follow-up studies.

4 drawing was kept for several weeks).

2.1 Design

The design of this study was intended to elicit the relationship between subjects’ choice behavior in a 2-by-2 payoff-matrix version of Ellsberg’s two-urn problem and their comprehension of the choice task. The row and column of the outcome of a matrix bet was determined by drawing independently from each of two decks of cards: one containing 10 red and 10 black cards, and the other containing 20 red and black cards in an unknown proportion. Subjects first answered five questions. Responses were used to assess how they assigned likelihoods to drawing a red card from each of the two decks and whether they understood the concept of a payoff-matrix bet after it was explained to them. In addition, responses were used to test whether subjects recognized the equivalence between a bet on a deck and an equivalent bet framed in terms of a payoff matrix.

The ﬁve questions, denoted as Qi and described immediately below, were followed by an actual choice task between two matrix-payoﬀ bets.

Assessing the likelihood of drawing red in the two decks. After the presentation of basic instructions, each session began with the following representation of the Ellsberg 2-urn setting:9

There are two decks of cards, each containing exactly 20 cards: Deck #1 contains exactly 10 black cards and 10 red cards. Deck #2 contains an unknown mix of black and red cards (could be all black, all red, an even mix, or any combination in between).

A card is randomly drawn from each deck: One card is drawn from Deck #1 (which contains exactly 20 cards, 10 of which are black and 10 of which are red), and One card is drawn from Deck #2 (which contains an unknown mix of 20 red and black cards).

We used cards rather than urns for ease and transparency of subsequent implementation. Subjects were next asked to answer two multiple choice questions designed to assess their comprehension of the setting:

Qk In Deck #1, what is the percentage chance that a red card is drawn? (i.e. what are the odds that a red card is randomly selected from Deck #1?)

9A full description of the study, as well as a description of how it varied between the three sessions, is available in an Internet Appendix: http://public.kenan-ﬂagler.unc.edu/faculty/sagij/E2cFrameExperimentScreenShots.pdf.

5 Qu In Deck #2, what is the percentage chance that a red card is drawn? (i.e. what are the odds that a red card is randomly selected from Deck #2?)

For each question, they were asked to select one of the following possible responses: 0% chance, 25% chance, 50% chance, 75% chance, 100% chance, and “Cannot be determined”. While “50% chance” is the only correct answer to Qk, there are two reasonable responses to Qu. A subject could reason that, since there is no rationale for suspecting that red cards are more prevalent than black cards in the second deck, a legitimate response is “50% chance”. Such reasoning is consistent with a Bayesian approach to forming a subjective prior. On the other hand, answering

“Cannot be determined” also seems to be a legitimate response to Qu given that the frequency-based probability of drawing red is not known.

In our analysis of their choice behavior, we classiﬁed subjects as probability minded if they selected 50% as the chance of drawing red in their response to Qu and as ambiguity minded if they selected “Cannot be determined” instead. We deem the remaining responses to Qu (i.e., 0%, 25%, 75%, or 100%) as less satisfactory. In doing so, we are taking a stance that the principle of insuﬃcient reason is intuitively applicable in forming a prior over the number of red cards in deck #2 — in other words, one cannot provide a compelling reason to justify an asymmetric prior in a situation where all information is patently symmetric. We return to this when we analyze the data and provide evidence substantiating the view that subjects who entered less satisfactory responses to Qu were generally not engaged in the study.

Comprehension of payoff matrix bets. The notion of a payoff matrix, contingent on drawing from the two decks, was next presented and explained to the subjects. Each subject was shown the payoff matrices in Figure 1 and then received a detailed explanation of what the outcome would be in each state if the payoff matrix B was chosen.10

To test subjects’ comprehension, they were presented with the following question and asked to select as a response one of $1, $2, $3, and $4 (the correct response is $2).

Qcomp Suppose you select Option A.... Given that you select Option A (shown above), what would the payoﬀ be if Black is drawn from Deck #1 and Red is drawn from Deck #2?

10 For instance, “Suppose you select Option B. Then a card will be drawn from each of the two decks. If the card from Deck #1 is Black and the card from Deck #2 is Black then you win $3 (the amount in the top-left box).” The other three possibilities were similarly described.

Below is a hypothetical choice. Suppose you are asked to choose between the following two payoff tables:

Deck #2: 20 cards Deck #2: 20 cards of which an of which an Option A unknown number OR Option B unknown number are black and/or are black and/or red red

Black Red Black Red

Win $1 Win $2 Win $3 Win $1 Deck #1: 20 Black Deck #1: 20 Black cards of cards of which 10 which 10 are black are black and 10 are and 10 are Win $3 Win $4 Win $4 Win $2 Red red Red red

The payoff from your choice will be determined by drawing two cards: One card from Deck 1 containing 10 black and 10 red cards One card from Deck 2 containing 20 cards with an unknown mix of red and/or black cards.

Suppose you select Option B: Fig. 1: The payoﬀ-matrix representation of drawing from two decksDeck #2: of 20 cards, cards with known and of which an unknown distributions, respectively. Option B unknown number are black and/or red

Frame Consistency. The next two questions in the study allowedBlack us toRed infer whether subjects recognized the equivalence between the classical 2-urn Ellsberg problem and its representation Win $3 Win $1 Deck #1: 20 Black using payoff matrices. Specifically, subjects were shown,cards one of at a time, the payoff matrices in which 10 are black Figure 2, and asked to assess the likelihood of winningand in10 are each case. Win $4 Win $2 red Red

Deck #2: 20 cards Deck #2: 20 cards of which an of which an K unknown number U unknown number are black and/or are black and/or red red

Black Red 3 Black Red

Win Win Win Win Deck #1: 20 Black $0 $0 Deck #1: 20 Black $0 $95 cards of cards of which 10 which 10 are black are black and 10 are Win Win and 10 are Win Win red Red $95 $95 red Red $0 $95

Fig. 2: The Ellsberg 2-urn choice problem depicted using payoﬀ-matrices. Rather than urns, we

used decks of cards with known and unknown color compositions in the experiment.

The ﬁrst payoﬀ matrix shown was U, after which subjects were asked,

0 Q Given the above scenario, what is the probability (what are the odds) that you will win $95? u

The payoff from your choice will be determined by drawing two cards: As with Qu, the possible responses were: 0% chance, 25% chance, 50% chance, 75% chance, 100% 1. One card from Deck 1 containing 10 black and 10 red cards 2. One card from Deck 2 containing 20 cards with an unknown mix of red and/or chance, and “Cannot be determined”.black cards. Subsequently, the subjects were shown the payoﬀ matrix 0 K, and then asked the same question (denoted Qk). Subjects who recognized the equivalence, Given the above scenario, what is the probability (what are the odds) that you will win $95? 7

0 and were engaged in the task, were expected to give the same answer to Qu as they did to Qu, 0 and the same answer to Qk as they did to Qk. We refer to such subjects as frame-consistent.

0 0 Matrix-Based Choice Task. After responding to Qk and Qu, subjects were presented with the following:

We will now ask you to make a choice between two payoﬀ tables. There is no right or wrong answer. Please make your decision by carefully considering the information presented in each choice scenario.

This choice task is depicted in Figure 3: Situation #1

Deck #1 contains exactly 20 cards with 10 red cards and 10 black cards Deck #2 contains exactly 20 cards with an unknown number of red and black cards (could be all black, all red, an even mix, or any combination in between)

Please select the bet below that you most prefer (Option A or Option B)

Deck #2: 20 cards Deck #2: 20 cards of which an of which an Option A unknown number OR Option B unknown number are black and/or are black and/or red red

Black Red Black Red

Win Win Win Win Deck #1: 20 Black $0 $101 Deck #1: 20 Black $0 $0 cards of cards of which 10 which 10 are black are black Win Win and 10 are Win Win and 10 are Red red Red red $0 $100 $100 $100

(Main Choice)

1 = Option A 2 = Option B

Please briefly indicate why you selected this option:

0 Fig. 3: The choice problem presented to each subject after (s)he answered Qk, Qu, Qcomp, Qu, 0 and Qk. 8

If the top-right cell in Option A was $100, then the choice between Options A and B would be isomorphic to the choice of drawing from the unknown versus the known urn in the classic 2-urn experiment. The additional $1 was added to break indiﬀerence. In other words, a Bayesian whose preferences satisfy ﬁrst-degree dominance should strictly prefer Option “A” over Option “B”.

8 After this task was completed, in another departure from the classic experiment, we oﬀered subjects a chance to provide a written explanation for their choice.

At the end of the study we collected demographic data. Information was obtained about subjects’ age, gender, race, education, household income, and country of residence. In the third experimental session, we included a battery of items from the “Need for Cognition” (NFC) psychometric scale (see Cacioppo, Petty, and Feng, 1984). We also administered this scale in a follow-up study with the subjects of the ﬁrst two experiments.11 The NFC scale has been used extensively in the social psychology literature to assess the degree to which subjects tend to be analytical in their approach to problem-solving (see Petty, Bri˜nol,Loersch, and McCaslin, 2009). We decided to obtain this measure for our subjects in order to have an independent, though self-reported, measure of their analytic tendencies.

2.2 Results

We present the ﬁndings from our study ﬁrst in terms of subjects’ performance in the comprehension test (Tables 3 and 4) followed by an analysis of their choice behavior (Tables 5 and 6).

Engagement and Comprehension. Table 3 summarizes subjects’ comprehension of the basic setup as assessed by responses to Qk, Qu and Qcomp. Panel A documents that a small minority

(3.7%) of subjects fail to answer both Qk and Qu satisfactorily. Across the subgroups listed in the ﬁrst column, the median education level is a 2-year degree (Associate’s) or “some college”.

The clearest pattern appears to be that those answering Qk and/or Qu inadmissibly spent considerably less time completing the study, suggesting that typical engagement levels may have been low for these individuals. While subjects that provided an inadmissible answer to one of Qk or Qu tended to have lower education levels, this is not uniformly true. Specifically, subjects that answered both Qk and Qu inadmissibly had the same level of education as those that answered both correctly. The difference between these two subgroups is the median time spent on the study, which was the most pronounced difference for this statistic between any two subgroups in Panel A. Thus, while education and therefore comprehension may have played a role in whether or not subjects answered Qk and Qu admissibly, it appears that effort or engagement was as important if not more so.

Panel B of Table 3 reports on the attributes of subjects who selected the correct response of $2

11Approximately 65% of the subjects in the ﬁrst two experiments completed the follow-up study.

9 for Qcomp. While 79% of subjects provided admissible answers to the first two questions, only about half of the subjects, 54%, who completed the study answered Qcomp correctly (and some subjects might have guessed the answer). The dramatic contrast suggests that comprehension played a more important role in Qcomp than it did in Qk or Qu. This view is reinforced by the fact that the number of subjects who answered Qcomp correctly was below 60% even when conditioning on those subgroups that answered one or both of Qk and Qu admissibly. Moreover, the median time to completion of subjects who answered Qcomp correctly (590 seconds) was 55 seconds longer than those who didn’t, and this difference is significantly (p < 0.001) lower than the 148 second difference between subjects who answered both Qk and Qu admissibly, and those that answered one of them inadmissibly. Finally, the lowest median time to completion was exhibited by subjects that answered both Qk and Qu inadmissibly, and was statistically the same whether or not the subjects answered Qcomp correctly (p > 0.50). From these considerations we surmise that Qk and Qu primarily measure subjects’ engagement level while Qcomp is additionally related to their ability to understand the setup.

Frame Consistency. Panel A of Table 4 reports on the attributes of subjects, according to whether or not they were frame-consistent. The total number of frame-consistent subjects (i.e.,

23.9%) is substantially less than the number of subjects who answered Qcomp correctly (i.e., 53.7%), and the latter number is in turn significantly smaller than the number of subjects who gave admissible responses to both Qk and Qu (i.e., 79.1%). This suggests that these tasks are progressively more difficult for the subjects.12 Based on this, we assign each subject a “task score” (T-Score). Subjects who, through their answers, exhibited an understanding of the deck likelihoods and the payoff matrices, and are also frame consistent, are assigned a T-Score of “3”.

For the remaining subjects, those who answered Qk, Qu and Qcomp admissibly but were not frame-consistent received a T-Score of “2”. This group appeared to be relatively well-educated and devoted above-average eﬀort to the study (as indicated by median time logged — see Panel

B). Subjects who admissibly answered Qk and Qu, but did not correctly answer the matrix payoﬀ comprehension question received a T-Score of “1”. The remaining subjects who did not satisfactorily answer the questions about the deck likelihoods, Qk and Qu, received a T-Score of “0”.

We’ve already argued that T-Score 0 subjects could be plausibly categorized as inattentive. A

12It seems intuitive that frame consistency would be the most challenging task. At the very least, it requires subjects to understand the payoff matrix representation and to be engaged to the same extent as they were in answering Qk and Qu. In that sense, it subsumes the previous two tasks in difficulty. Beyond that, a subject would additionally have to be aware that a payoff matrix featuring constant payoffs across the rows (or columns) is tantamount to a draw from a single deck.

10 concern is that, rather than being a proxy for comprehension for remaining subjects, the T-Score instead measured subjects’ level of attentiveness to the tasks. One of our goals is to determine whether ambiguity attitudes varied across subjects that did or did not perceive the frame equivalence. It is therefore important to understand whether T-Score 2 subjects failed to be frame consistent because they became inattentive or because the frame-consistency task was too complex despite their efforts. Under the former hypothesis, the subjects chose not to expend effort on the task and thus their answers were random. To help validate the T-Score as a measure of comprehension, especially when differentiating between T-Score 2 and T-Score 3 subjects, we examine four measures of subjects’ attentiveness: time to complete the study (“Duration”), subjects’ Need for Cognition scale measure (“NFC Score”), length of verbal explanation of decision in the choice task (“Comment Length”), and inconsistent responses to demographic questions (“Demographic Inconsistency”). Comment Length corresponds to the number of letters a subject used in explaining his or her decision in the matrix-based choice task after making the choice.13 Demographic Inconsistency measured the number of inconsistent responses to the demographic questions concerning birth year, race, and gender (three attributes that are extremely unlikely to change with time).14 All measures save for Duration are based on questions answered after the frame-consistency task and we assume that attentiveness in a later task implies attentiveness in an earlier one (i.e., subjects are not likely to disengage and then re-engage soon after). No single measure perfectly captures attentiveness and some, like the NFC scale, are also intuitively correlated with thoughtfulness and comprehension. One expects, however, that inattentive subjects would score significantly worse on most of these measures relative to attentive subjects.

Panel B of Table 4 reports statistics, conditional on T-Score subgroup, for the four attentiveness measures. Medians are used in all but the demographic inconsistency measures (the median demographic inconsistency in each subgroup was zero). A “*” signifies that the subgroup’s statistic is different than its predecessor’s statistic with p-value of 5% or less.15 A “**” signifies that the subgroup’s statistic is different than its predecessor’s statistic with p-value of 1% or less. The “†” signifies that the subgroup’s statistic is not different than its predecessor’s statistic with p-value of 5% or less, but is different from the statistic of its predecessor’s predecessor with

13Using the number of words instead leads to no qualitative difference. 14As explained earlier, we conducted a follow up study several months apart with some participants in the first two sessions. In each follow-up we collected the same demographic data as in the original study. Given an inconsistent demographic responses, we are unable to tell whether inaccurate replies were made in the study featuring the matrix choice. Thus, the presence of an inconsistency suggests that a subject was inattentive in at least one of the two studies but we cannot tell which for certain. 15For instance, the median comment length of T-Score 1 subjects was 31 letters, which was statistically different from the median of 24 letters used by T-Score 0 subjects.

11 p-value of 1% or less. The results strongly suggest that inattention is a primary driver behind the difference between T-Score 0, 1, and 2 subjects. By contrast, the preponderance of evidence suggests that T-Score 2 subjects were as attentive as T-Score 3 subjects, but that the frame-consistency task was prohibitively complex despite their efforts. Thus, while one may be justified in categorizing Task Score 0 or 1 subjects as inattentive, T-Score 2 subjects merit being regarded as attentive as T-Score 3 subjects yet exhibiting lower comprehension. Henceforth, we refer to the T-Score 3 subjects as high comprehension, the T-Score 2 subjects as low comprehension, and the T-Score 1 and 0 subjects as Inattentive.

To further probe the differences between high comprehension and low comprehension subjects in a multi-variate setting, Table 5 reports the results of a Probit regression in which the dependent variable is 1 for the former and 0 for the latter subjects. The single most significant independent variable was education. Other important demographic determinants of comprehension were age, gender, race, and whether the choice problem was incentivized. Younger subjects were more likely to exhibit high comprehension as were those subjects who participated in the incentivized experiments (accounting for about 74% of the subjects). While income and country of residence were not important factors, males were more likely to be in the high comprehension cohort than females and the same was true for Caucasians versus non-Caucasians.16 Table 5 also reports Probit regression results when the NFC scale (ranging from 18 to 90 in our sample) is included among the demographic variables. As might be expected, NFC was a highly significant determinant of the level of comprehension, even after controlling for the other variables including education. In particular, whether the experiment was incentivized or not did not matter once the NFC scale was included as a control.17

2.2.1 Observed Choice Behavior

The analysis in the previous section allows us to classify subjects based on whether they were attentive, and if so whether they appeared to comprehend the equivalence between a “simple” and “complex” way of framing the choices in the Ellsberg 2-urn setup. Panel A of Table 6 reports observed choice behavior (recall Figure 3) and how it related to subjects’ attentiveness and comprehension. Fewer than half the subjects in the inattentive cohorts, those with T-Scores of 0 or 1, opted for the bet on the known deck. In a standard Ellsberg experiment, these results would be reported as a rate of exhibited ambiguity averse choice of less than 50% (column fk). Such a

16Females outnumbered males in the sample and in every T-Score level but the highest. Non-Caucasians were over-represented in the T-Score 0 cohort, perhaps because language barriers reduced the rate of correct answers to the likelihoods of cards in the two decks. 17High NFC subjects tend to “enjoy” solving problems and may require less incentives to complete analytical tasks.

12 rate is lower than any among the 2-urn experiments documented in Table 1 and is signiﬁcantly lower than the aggregate rate over all experiments reported in the same table (t-statistic of -7.65). We assume that the choices by individuals in the inattentive cohorts were either made randomly or via heuristics invoked because subjects did not understand the setup.18 Consequently, we do not interpret fk for the T-Score 0 group as a rate of ambiguity averse choice. Instead, we interpret fk for inattentive T-Score 0 subjects as a baseline rate of preferring the known bet, and use it for measuring the rate of exhibited ambiguity averse choice of the other groups.19

For each of the groups with T-Score levels between 1 and 3, column “Null0” in Panel A of Table 6 reports the t-statistic testing the hypothesis that fk = 45%, the uninformed or “noisy” response exhibited by subjects with T-Score “0”. Not surprisingly, the choices of subjects in the other inattentive cohort, those with T-Score of 1, could not be distinguished from the baseline (t = 1.30). The low comprehension subjects with T-Score 2, who were attentive, understood the payoff matrix and the deck likelihoods but were not frame-consistent, demonstrated a marginally significant degree of ambiguity averse choice (by preferring the known bet) relative to the baseline. By contrast, the high comprehension subjects, with T-Score 3, exhibited a highly significant degree of (relative) ambiguity averse choice. Moreover, for each group we tested fk against the null that it equals 65%, the aggregate rate of ambiguity averse choice in the classical Ellsberg experiment (taken from Table 1). Under this test, the rate of ambiguity averse choice of 68.9% for the high comprehension subjects (T-Score 3 ) could not be distinguished from 65%. By contrast, fk for each of the other groups was significantly below 65%. In other words, high comprehension subjects behave the same as the general population when members of the latter are confronted with a clear choice between an ambiguous and an unambiguous bet. Another implication is that subjects’ “sophistication” is not commensurate with ambiguity neutrality as one might intuit. The results further suggest that in experiments where ambiguity is partially obscured by complexity, as is the case here, one will see a lower overall rate of ambiguity averse choice relative to an experiment where the ambiguity is clearly manifest. This too runs counter to the intuition that ambiguity is driven by complexity.

Is Ambiguity Averse Choice Robust to Framing? To address this question we ﬁrst take the following position on what it means for an observable choice to be robust to framing. Suppose that a given choice problem with two alternatives, A and B, can be presented in various

18An example of a heuristic, perhaps because the subject doesn’t understand the payoﬀ-matrix, is: “I’ll take the option with the largest payoﬀ.” Another might be, “I don’t know what is going on here so I’ll randomize.” Both types of reasonings were stated in the explanatory comments provided by subjects. 19 The results do not change if either one of 50% or fk for the T-Score 1 subjects is used as a baseline of inatten- tiveness instead (or if one employs as a benchmark the fk of the combined T-Scores 0 and 1 cohorts).

13 consequentially equivalent ways. Then we define an alternative, say B, to be the robust choice if and only if it is preferred after a subject weighs all equivalent ways of framing the problem while recognizing their equivalence. In other words, the robust choice is the one that would not be renounced after comprehensive deliberation. In our experiment, the high comprehension subjects were the only ones that appeared to recognize that a bet framed as a drawing from a single deck of cards could be equivalently cast as a payoff matrix from a double drawing. Their preference was, in aggregate, essentially identical to the average choice made by subjects confronted with the “standard frame” of the Ellsberg 2-Urn choice problem in which 65% percent of subjects opt for the ambiguity averse choice of the known bet. Our results suggest that the latter choice behavior is robust because it is the maintained choice behavior even when subjects view a different though consequentially (nearly) equivalent frame.20

Probability minded versus ambiguity minded . As discussed in the preceding Subsection 2.1, subjects are classified as probability minded if they assessed the likelihood of drawing red from the unknown deck in Qu as being 50% and as ambiguity minded if they indicated that such a likelihood cannot be determined. The classification excludes some T-Score 0 subjects (those 21 who assigned a probability other than 50% to Qu). Column 3 in Panel B of Table 6 documents the percentage of ambiguity-minded subjects in each T-Score cohort. Among the T-Score 0 subjects, who we conjecture guessed at answers to Qu, the ratio of ambiguity-minded subjects (55%) cannot be statistically distinguished from 50%. By contrast, the higher T-Score cohorts exhibited a far greater (and highly significant) majority — two thirds or more — of ambiguity-minded subjects.

What is striking is that, save for high comprehension subjects, there was no statistical diﬀerence in fk between probability-minded and ambiguity-minded subjects (see Columns 5,6 and 9). This further reinforces the notion that only high comprehension subjects understood the choice problem well enough to provide answers that diﬀered from “noise”. Even more striking is the fact that the greatest contrast in choice behavior between any two groups was found between high comprehension probability-minded and high comprehension ambiguity-minded subjects. The

20The classical 2-Urn choice problem is not completely equivalent to our matrix payoff choice problem because in one state we slightly increased the payoff of the unknown bet. Presumably, changing the $101 payoff in the unknown bet to $100 to achieve complete consequential equivalence would not reduce the number of high comprehension subjects who opt for the known bet. In that sense, our inference of the robustness of the ambiguity averse choice is conservative. 21Further analysis of the choice of subjects with T-Score 0 revealed no significant difference among subgroups.

Speciﬁcally, we looked at subjects who answered Qk less satisfactorily versus those that answered Qk less satisfactorily; we also looked at the subgroup of T-Score 0 subjects who used the heuristic of providing the same answer to

Qk and Qu, and those who also gave the same answer to Qcomp.

14 latter overwhelmingly chose the unknown bet (fk = 28%) while the vast majority of the former opted for the known bet (fk = 88%). In fact, the rate of ambiguity averse choice among the high comprehension probability minded subjects is signiﬁcantly lower than exhibited by any cohort in any experiment we reviewed, while the opposite is true in the case of high comprehension ambiguity minded subjects.

Next, we focus on high comprehension subjects who are probability minded versus those who are ambiguity minded. Panel A of Table 7 compares the demographic statistics of these two groups and reveals a pattern which runs counter to the intuition that probability-minded subjects may better approximate the Bayesian ideal of neo-classical economics. Ambiguity-minded subjects were generally younger, more educated, and score higher on the NFC scale than those high comprehension subjects who are probability minded. A significant difference in gender was also observed. Because the demographic variables can be correlated, to disentangle their effects we also ran a multi-variate Probit regression which is reporeted in Panel B of Table 7. In the latter, gender was no longer significant after controlling for the joint effects of age, education, and need for cognition (which remain significant). The analysis suggests that high comprehension ambiguity-minded subjects rank as the most thoughtful and educated cohort in our sample. This is further corroborated by a comprehensive content analysis of the rationale given by subjects for their choices (see Appendix A) where, by and large, ambiguity-minded subjects appear to exhibit more reflective and sophisticated reasoning than those who were probability-minded.

According to the answers provided by the high comprehension probability-minded subjects, Option A stochastically dominated Option B. It would therefore appear natural for them to prefer the unknown but dominating Option A to the known and dominated Option B. While this is compatible with 82% of the observed behavior, there remains the possibility, as suggested by Keynes (1921), that some probability minded subjects may have an innate preference for betting on probabilities about which they have more knowledge22. This explanation can potentially account for the 28% of the probability-minded subjects who chose Option B.

2.3 Further discussion

Potential implications for other studies. In the single-urn problem, ambiguity averse choice is demonstrated by a particular pattern over two choice questions. Subjects ﬁrst have to prefer betting on Red than on Black, and then prefer to bet on “Black or Yellow” rather than “Red or Yellow”. The ﬁrst choice seems essentially the same as betting on the known versus unknown urn in the classical Elllsberg 2-urn choice problem. Thus one may expect a 65% rate of

22See Chew and Sagi (2008); Abdellaoui, Baillon, Placido, and Wakker (2011)

15 ambiguity-aversion based on the first question alone. Indeed, this is supported by the data in Binmore, Stewart, and Voorhoeve (2012) who find substantial ambiguity averse choice (about 65% of subjects) based only on the first question (top of their Figure 7). However, an overall preference for the known bet can only be reliably inferred if subjects answer the second question in a manner that is consistent with ambiguity aversion. It is here that subjects are required to essentially exhibit frame-consistency. Applying the lesson of our study to the single-urn case, one may suspect that because of task complexity many subjects systematically fail to see the analogy between the known bets, Red and “Black or Yellow” (or, equivalently, the unknown bets Black and “Red or Yellow”). This failure may lead them to make inconsistent choices across the two choice comparisons and exhibit the low rates of ambiguity averse choice as documented by Binmore, Stewart, and Voorhoeve (2012) and Charness, Karni, and Levin (2012).

The implications of our study for market experiments, such as that conducted by Bossaerts, Ghirardato, Guarnaschelli, and Zame (2010), is that the most attentive and sophisticated subjects would be the ones who recognize the ambiguity and are also ambiguity averse. Even if they constitute a minority, one may expect that ambiguity-minded and high-comprehension subjects would have a signiﬁcant price impact, consistent with the ﬁndings of Bossaerts, Ghirardato, Guarnaschelli, and Zame (2010).

Suspicion. Several papers in the literature raise the concern that subjects may be more apt to choose a bet with “known” probabilities over one with “unknown” probabilities should they suspect that the experiment might not be fair (e.g., the experimenters wish to avoid having to pay).23 While we cannot completely rule out a possible role for suspicion in subjects’ decisions, there is little corroborative evidence that it did. There was no sign in the brief essay answers that suspicion was an issue.24 Moreover, among high comprehension subjects who were probability-minded, there was an overwhelming preference for the unknown bet, far more than in standard 2-urn experiments. Thus, for suspicion to have a role, there would need to be a strong relationship between a predisposition towards suspicion and ambiguity-mindedness in the sample accompanied by a near absolute reluctance among such subjects to comment about their suspicions — something that we have no reason to believe or conjecture. In particular, the articulate nature of the responses by high comprehension, ambiguity-minded subjects suggests quite the opposite (see Appendix A).

23These include Schneeweiss (1968); Brewer (1965); Kadane (1992) and Pulford (2009). 24Among the 2453 subjects that entered something in the comment ﬁeld after choosing payoﬀ matrix “A” or “B”, we found only one instance where a subject indicated suspicion about the experiment. This subject chose “A” (the unknown bet — which would contradict suspicion as a choice motive) and yet wrote, “Because in deck 1 you have a 50/50 chance because there are 10 black & 10 red. Is there a trick because option a as [sic] a win of $101.”

16 3 Concluding remarks

There are several key findings to take away from our study. First, framing matters and is a first-order determinant in choice problems featuring ambiguity. In particular, the unusually low degree of preference for a known versus an unknown bet appears linked to the fact that only a small portion of the subject pool (about 13%) displayed an understanding of how payoffs with known or unknown probabilities could be equivalently presented. In any complex situation involving ambiguity, one may therefore expect to see substantially less evident ambiguity aversion than in the classic 2-urn experiment. To the extent that complexity is rife, one may not observe much ambiguity aversion in practice.

A second key ﬁnding is that (the small minority of) individuals who are suﬃciently sophisticated to appreciate the presence of ambiguity in complex situations choose similarly to typical decision makers who are facing a clear-cut problem between ambiguous and unambiguous bets. By focusing on attentive, educated, and analytic individuals, one does not exclude those who are non-Bayesians or ambiguity averse. In other words, the phenomenon ambiguity aversion appears to be robust to framing.

Our final set of key findings is that among the group of attentive and high comprehending decision makers who are capable of seeing through frames, the majority are non-Bayesians. Surprisingly, the non-Bayesians tend to be more educated and analytic than subjects who most resemble Bayesian decision-makers. Thus, complexity may have the effect of limiting the impact of ambiguity aversion to the decisions of a minority (roughly 8% in our sample) of relatively sophisticated decision-makers.

17 A Appendix A: Content Analysis

Here we provide anecdotal evidence from content analysis as a means to infer respondents’ psychological process in selecting one of the two given options. Speciﬁcally, we measure the frequency with which terms related to particular lines of reasoning appear in post-selection open-ended responses across all three experimental iterations (2506 original subjects). Considering only high comprehension subjects (i.e., were assigned a Task Score of “3”), we separately analyze the textual responses of those who selected the ambiguity-neutral (or seeking) Option A and those who chose the ambiguity-averse Option B.

Across the three studies, only 25 high comprehension and ambiguity-minded individuals selected Option A. Twenty-four of these 25 provided open-ended responses as to their reasoning. These responses were coded into four groups: (1) “Bayesians”, which includes all who mentioned their expected odds of winning to be 50/50 and/or alluded to the chance to attain a greater monetary reward; (2) “Ambiguity seeking”, which includes those who alluded to preference for the chance to have greater than 50/50 odds; (3) “Contradictory”, where responses appear to indicate ambiguity aversion yet Option A was selected; (4) “Indeterminate,” meaning the selection logic was unclear. For example, the response “Option B gives me an EV of $50. Option A’s EV ranges from $0 to $100.50...which is actually $50.25. Therefore option A is better” was coded as Bayesian, “the thought of the unknown is intriguing” was coded as Ambiguity seeking, and “at least this way I know that Deck #1 provides a 100% chance of winning. Since Deck #2 is completely unknown, I’d rather bank on Deck #1” was considered Contradictory. In total, 13 responses (54.2%) were coded as Bayesian, ﬁve as ambiguity seeking, four as contradictory, and two as indeterminate.

Next, we considered the 192 high comprehension subjects who were ambiguity-minded, and chose Option B. All but one of these 192 provided a textual reason for their choice via the open-ended response item. These were coded into three groups: (1) “Ambiguity averse,” which includes those who mention a preference for known 50% versus unknown odds of winning; (2) “Indeterminate”, meaning the selection logic was unclear; (3) “Guess”, wherein the respondent indicated they made their choice at random (e.g., “don’t know just a hunch”). Examples of “Ambiguity averse” responses include “I have a 50% chance of selecting a red card from deck 1. It is unknown if any red cards are in deck two. Thus, I would take my chance on getting a red card from deck 1”, “50/50 chance with B; unknown chance with A”, and “I am happy with a 50% chance to win $100. I don’t know what my odds would be to win $100/$101 in Option A”. In total, 181 responses (94.3%) were coded as ambiguity averse, seven as indeterminate, and six as guesses. Hence the vast majority of respondents appear to exhibit at least some degree of ambiguity aversion as evidenced by their textual responses.

18 Considering only high comprehension subjects who were probabilistically-minded, we again separately analyze those who selected Option A and Option B. In the former category, 75 of the 76 respondents who selected Option A provided a textual reason for their choice via the open-ended item. These were coded into ﬁve groups: (1) “Bayesians”, which includes all who mentioned their expected odds of winning to be 50/50 and/or alluded to the chance to attain a greater monetary reward; (2) “Ambiguity seeking”, which includes those who indicated they have better odds or a better chance to win with Option A; (3) “Contradictory”, where responses appear to indicate a misunderstanding of the choice scenario; (4) “Indeterminate”, meaning the selection logic was unclear; (5) “Guess”, wherein the respondent indicated they made their choice at random. In total, 36 responses (48%) were coded as Bayesian, 10 as ambiguity seeking, 10 as contradictory, 9 as indeterminate, and 10 as guesses.

Finally, we consider high comprehension and probabilistically-minded subjects who chose Option B. In this category, 27 of 30 who selected Option B provided a textual reason for their choice via the open-ended item. These were coded into three groups: (1) “Ambiguity averse”, which includes those who mention a preference for known 50% versus unknown odds of winning; (2) “Indeterminate”, meaning the selection logic was unclear; (3) “Guess”, wherein the respondent indicated they made their choice at random. In total, 19 responses (70.4%) were coded as ambiguity averse, ﬁve as indeterminate, and three as guesses.

As evidenced by the number of characters written by high comprehension subjects, ambiguity-minded respondents on average wrote more than twice as much as probabilistically-minded respondents. Ambiguity-minded subjects who chose Option A wrote an average of 131.0 characters in their textual response, while those who chose Option B wrote an average of 121.5 characters. By comparison, probabilistically-minded subjects who selected Option A wrote an average of 61.9 characters, while those who selected Option B wrote 48.8 characters on average. Based on the amount of text as well as its content, we conclude that the ambiguity-minded subjects appear to exhibit more sophisticated reasoning than those who were probabilistically-minded.

19 References

Abdellaoui, M., A. Baillon, L. Placido, and P. P. Wakker, 2011, “The Rich Domain of Uncertainty: Source Functions and Their Experimental Implementation,” American Economic Review, 101(2), 695–723.

Ahn, D., S. Choi, D. Gale, and S. Kariv, 2011, “Estimating Ambiguity Aversion in a Portfolio Choice Experiment,” Working Paper.

Becker, S. W., and F. O. Brownson, 1964, “What Price Ambiguity? or the Role of Ambiguity in Decision-Making,” Journal of Political Economy, 72(1), 62–73.

Binmore, K., L. Stewart, and A. Voorhoeve, 2012, “How much ambiguity aversion?,” Journal of Risk and Uncertainty, 45(3), 215–238.

Bossaerts, P., P. Ghirardato, S. Guarnaschelli, and W. R. Zame, 2010, “Ambiguity in Asset Markets: Theory and Experiment,” Review of Financial Studies, 23(4), 1325–1359.

Brewer, K., 1965, “Decisions under Uncertainty: Comment,” Quarterly Journal of Economics, 79, 657–653.

Cacioppo, J. T., R. E. Petty, and K. C. Feng, 1984, “The Eﬃcient Assessent of Need for Cognition,” Journal of Personality Assessment, 48(3), 306–307.

Charness, G., E. Karni, and D. Levin, 2012, “Ambiguity Attitudes: An Experimental Investigation,” Working Paper.

Chew, S. H., and J. S. Sagi, 2008, “Small worlds: Modeling attitudes toward sources of uncertainty,” Journal of Economic Theory, 139(1), 1–24.

Cohen, M., J.-Y. Jaﬀray, and T. Said, 1985, “Individual Behavior Under Risk and Under Uncertainty: An Experimental Study,” Theory and Decision, 18, 202–228.

Cohen, M., J.-M. Tallon, and J.-C. Vergnaud, 2011, “An experimental investigation of imprecision attitude and its relation with risk attitude and impatience,” Theory and decision, 71(1), 81–109.

Curley, S. P., and J. F. Yates, 1989, “An Empirical Evaluation of Descriptive Models of Ambiguity Reactions in Choice Situations,” Journal of Mathematical Psychology, 33, 397–427.

Curley, S. P., J. F. Yates, and R. A. Abrams, 1986, “Psychological sources of ambiguity avoidance,” Organizational Behavior and Human Decision Processes, 38(2), 230–256.

Einhorn, H. J., and R. M. Hogarth, 1986, “Decision Making under Ambiguity,” The Journal of Business, 59(4), S225–s250.

20 Ellsberg, D., 1961, “First order versus second order risk aversion,” Quarterly Journal of Economics, 75(4), 643–669.

Etner, J., M. Jeleva, and J.-M. Tallon, 2012, “Decision Theory Under Ambiguity,” Journal of Economic Surveys, 26(2), 234–270.

Fox, C. R., and A. Tversky, 1995, “Ambiguity Aversion and Comparative Ignorance,” The Quarterly Journal of Economics, 110(3), 585–603.

Halevy, Y., 2007, “Ellsberg Revisited: An Experimental Study,” Econometrica, 75(2), 503–536.

Hogarth, R. M., and H. J. Einhorn, 1990, “Venture Theory: A Model of Decision Weights,” Management Science, 36(7), 780–803.

Ju, N., and J. Miao, 2012, “Ambiguity, Learning, and Asset Returns,” Econometrica, 80(2), 559–591.

Kadane, J., 1992, “Healthy Scepticism as an Expected-Utility Explanation of the Phenomena of Allais and Ellsberg,” in Decision-Making under Risk and Uncertainty: New Models and Empirical Findings. Kluwer, Dordrecht.

Kahneman, D., and A. Tversky, 1979, “Prospect theory: An analysis of decisions under risk,” Econometrica, pp. 263–291.

Keynes, J. M., 1921, A Treatise on Probability. Macmillan And Co.

K¨uhberger, A., and J. Perner, 2003, “The Role of Competition and Knowledge in the Ellsberg Task,” Journal of Behavioral Decision Making, 16, 181–191.

Liu, H.-H., and A. M. Colman, 2009, “Ambiguity aversion in the long run: Repeated decisions under risk and uncertainty,” Journal of Economic Psychology, 30(3), 277–284.

MacCrimmon, K. R., and S. Larsson, 1979, “Utility theory: Axioms versus paradoxes,” Expected utility hypotheses and the Allais paradox, 21.

Petty, R., P. Bri˜nol,C. Loersch, and M. McCaslin, 2009, “The need for cognition,” in Handbook of individual diﬀerences in social behavior, ed. by M. Leary, and R. Hoyle. Guilford Press, New York, NY, chap. 21, pp. 318–329.

Pulford, B. D., 2009, “Is luck on my side? Optimism, pessimism, and ambiguity aversion,” Is Luck on My Side? Optimism, Pessimism, and Ambiguity Aversion, 62(6), 1079 – 1087.

Savage, L. J., 1954, The Foundations of Statistics. John Wiley, New York, NY.

21 Schneeweiss, H., 1968, “Spieltheoretische Analyse des Ellsberg-Paradaxons,” Zeitschrift fur die Gesamte Staatswissenschaft, 124, 249–255.

Slovic, P., and A. Tversky, 1974, “Who Accepts Savages’s Axiom?,” Behavioral Science, 19(6), 368–373.

Trautmann, S. T., F. M. Vieider, and P. P. Wakker, 2011, “Preference Reversals for Ambiguity Aversion,” Management Science, 57(7), 1320–1333.

Tversky, A., and D. Kahneman, 1986, “Rational choice and the framing of decisions,” Journal of business, pp. 251–278.

22 Table 1: Experimental frequency of ambiguity aversion in the literature investigating Ellsberg’s 2- urn and 1-urn thought experiment. Only studies that explicitly reported frequencies of ambiguity averse choices are included.

% Ambiguity Averse Reference 2-Urns 1-Urn Subjects Notes Becker and Brownson (1964) 47.1 34 Table 2 Slovic and Tversky (1974) 58.6 29 Table 1 MacCrimmon and Larsson (1979) 78.9 19 Figure 7 Cohen, Jaﬀray, and Said (1985) 59.0 134 Section 5.2 Curley, Yates, and Abrams (1986) 65.4 26 Table 1 Einhorn and Hogarth (1986) 47.4 133 Table 1 Curley and Yates (1989) 70.2 31 Figure 6 Hogarth and Einhorn (1990) 77.9 195 Table 4 Fox and Tversky (1995) 62.0 162 p. 592 K¨uhberger and Perner (2003) 75.0 61 Figure 1 Pulford (2009) 73.2 112 Table 1 Liu and Colman (2009) 68.1 69 Section 2.4 Ahn, Choi, Gale, and Kariv (2011) 21.5 144 Table 1 Trautmann, Vieider, and Wakker (2011) 62.7 59 Table 2 Cohen, Tallon, and Vergnaud (2011) 37.5 104 p. 102 Binmore, Stewart, and Voorhoeve (2012) 24.3 942 Table 1 Charness, Karni, and Levin (2012) 8.0 272 Section 2.3 27.1 1672 Total 65.6 854

23 Table 2: Demographic data for the 2506 subjects that completed the study. “Income” represents the combined household income. “Age” is in years.

Education Percent Age Percentile Country Percent Less than high school 1.9 21 p1 United States of America 74.0 High school / GED 21.5 26 p5 Canada 21.7 Some college 25.1 29 p10 Australia 0.9 Associate’s degree 13.2 37 p25 United Kingdom 0.8 Bachelor’s degree 25.3 50 p50 Trinidad and Tobago 0.6 Master’s degree 9.8 60 p75 New Zealand 0.5 Ph.D. 1.5 67 p90 Jamaica 0.5 MD, JD, etc. 1.8 73 p95 Ireland 0.2 82 p99 Other 0.8

Gender Percent Income Percent Male 42.3 <$20k 11.2 Female 57.7 $20k- $29,999 11.1 $30k - $39,999 12.1 $40k - $49,999 11.8 Race Percent $50k - $59,999 11.7 White 83.6 $60k - $69,999 8.2 Asian 5.1 $70k - $79,999 8.9 African-American 5.0 $80k - $89,999 5.1 Hispanic 2.4 $90k - $99,999 5.0 Other 2.0 $100k - $109,999 4.3 Bi-racial 0.9 $110k - $119,999 1.7 Native American 0.8 $120k - $129,999 1.9 Paciﬁc Islander 0.2 $130k - $139,999 0.8 $140k - $149,999 1.7 $150k + 4.5

24 Table 3: Analysis of basic comprehension. “Median duration” corresponds to the time in seconds logged by subjects in the experiment. A “Median edu(cation)” level of “3” (resp. “4”) means that the highest level of education attained by the subject is “Some college” (resp. “Associates degree”).

Panel A: Median Median N % All Subjects duration education All subjects 2506 100.0 562 4

Admissible reponse to:

Qk 2262 90.3 580 4

Qu 2134 85.2 572 4

Qk and Qu 1982 79.1 587 4

Inadmissible reponse to:

Qk 244 9.7 368 3

Qu 372 14.8 471 3

Qk and Qu 92 3.7 348 4

Qk or Qu 524 20.9 439 3

Panel B: Matrix Comprehension Yes No Median Median Median Median N % Cohort duration edu N % Cohort duration edu All subjects 1346 53.7 590 4 1160 46.3 535 3

Admissible reponse to:

Qk 1253 55.4 610 4 1009 44.6 546 3

Qu 1203 56.4 599 4 931 43.6 543 3

Qk and Qu 1147 57.9 615 4 835 42.1 553 3

Inadmissible reponse to:

Qk 93 38.1 345 3 151 61.9 380 3

Qu 143 38.4 470 3 229 61.6 472 3

Qk & Qu 37 40.2 345 3 55 59.8 366 4

25 Table 4: Definition and analysis of subjects’ Task Score (T-Score). In Panel A a T-Score of 0 is assigned to subjects that did not provide satisfactory answers to the deck likelihood questions Qk and Qu. Of the remaining subjects, those that answered Qcomp incorrectly received a T-Score of 1. Of the remaining subjects, those that were (resp. were not) frame consistent were assigned a T-Score of 3 (resp. 2). Panel B reports measures of engagement for the various T-Score cohorts. A ‘median education level of “3” (resp. “4” or “5”) means that the highest level of education attained by the subject is “Some college” (resp. “Associates degree” or “Bachelor’s degree”). “Median duration” corresponds to the time in seconds logged by subjects in the experiment. The NFC score is based on the “Need for Cognition” scale measure (see Cacioppo, Petty, and Feng, 1984). “Comment Length” corresponds to the number of characters a subject used to explain her/his selection in the choice task. A subject provided “demographically inconsistent” information if birth year, gender, or race were not consistent between the original and a follow-up study. A ∗ (resp. ∗∗) indicates that a T-Score subgroup’s statistic is significantly different from that of its predecessor with p-value between 1% and 5% (resp. 1% or less). A “†” signifies that the subgroup’s statistic is not different than its predecessor’s statistic with p-value of 5% or less, but is different from the statistic of its predecessor’s predecessor with p-value of 1% or less.

Panel A: Comprehension of: Median T-Score Likelihood Matrix Frame N % All Subjects education 0 No No No 263 10.5 3 0 No No Yes 62 2.5 3 0 No Yes No 159 6.3 3 0 No Yes Yes 40 1.6 3 1 Yes No No 660 26.3 3 1 Yes No Yes 175 7.0 3 2 Yes Yes No 824 32.9 4 3 Yes Yes Yes 323 12.9 5

Panel B: Median Median Median Mean T-Score Duration NFC Score Comment Length % Demographic Inconsistency 0 439 56 24 9.3 1 553** 59* 31* 5.7 2 617** 64** 57.5** 3.0† 3 593 65† 69* 5.3

26 Table 5: Probit regressions of subject comprehension on demographic variables. Subjects with T-Score of 3 (resp. 2) are considered high (resp. low) comprehension. In the Probit analysis the dependent variable is one if the subject is high comprehension and zero if the subject is low comprehension. Gender is one for males and two for females. Education level is one if the highest attained education is less than high school, two for high school, etc. (as in Table 2. Combined income is similarly scored. The US Indicator is one if country of residence is the United States and zero otherwise. The incentive indicator is one if the elicited choice preference was incentivized and zero otherwise. The Caucasian Indicator is one if the subjected answered “White” for their race and zero otherwise. The NFC score is based on the “Need for Cognition” questionnaire (see Cacioppo, Petty, and Feng, 1984). Coefficients marked with ∗ are significant at the 5% level, while those marked with a ∗∗ are significant at a 1% level or better.

Variable Coeﬀ. S.D. Coeﬀ. S.D. Constant −0.77∗ 0.34 −1.14∗∗ 0.44 Age −0.0124∗∗ 0.0033 −0.0109∗∗ 0.0037 Gender −0.33∗∗ 0.098 −0.307∗∗ 0.109 Education 0.24∗∗ 0.035 0.20∗∗ 0.039 Combined income −0.0125 0.0140 −0.0105 0.0157 US Indicator −0.013 0.111 −0.095 0.123 Incentive Indicator 0.40∗∗ 0.119 0.19 0.143 Caucasian Indicator 0.53∗∗ 0.138 0.36∗ 0.151 NFC score 0.014∗∗ 0.0045 Pseudo R2 0.113 0.084

27 Table 6: The table reports choice preferences for the known versus unknown likelihood bet (i.e., payoﬀ matrix) by subject comprehension. Panel A reports unconditional results. Panel B reports the results conditional on whether subjects answered “50%” when asked for the likelihood of drawing a red card from the deck with unknown distribution (“Probability-minded” subjects), or whether they answered “Cannot be determined” (“Ambiguity minded” subjects). high comprehension subjects correctly answered questions about likelihoods and the payoﬀ matrices, and were also frame-consistent. Low comprehension subjects correctly an-

swered questions about likelihoods and the payoﬀ matrices, but were not frame-consistent. The column “fk” reports the frequency

of ambiguity averse choice for that cohort of subjects. The “Null0” column reports the t-statistic for the hypothesis that the

frequency of ambiguity averse choice is the same as that for the inattentive group with T-Score of “0”. The “Nulllit” column reports the t-statistic for the hypothesis that the frequency of ambiguity averse choice is the same as that of the literature (0.66

based on 854 observations). The “Null: fk amb = fk prb” column reports the t-statistic for the hypothesis that probability minded and ambiguity minded subjects in a T-Score cohort make the ambiguity averse choice with the same frequency.

Panel A: 28

Subject Prefer Prefer Null0 Nulllit

Classiﬁcation T-Score Unknown Known fk (%) t-stat t-stat Inattentive 0 290 234 44.7 -7.65 Inattentive 1 432 403 48.3 1.30 -7.21 Low Comprehension 2 414 410 49.8 1.83 -6.59 High Comprehension 3 101 222 68.7 6.83 1.00

Panel B: Ratio of Probability minded subjects Ambiguity minded subjects

Subject Ambiguity minded Prefer Prefer Prefer Prefer Null: fk amb = fk prb

Classification T-Score subjects Unknown Known fk (%) Unknown Known fk (%) t-stat Inattentive 0 0.55 39 30 43.5 42 41 49.4 0.73 Inattentive 1 0.68 141 127 47.4 291 276 48.7 0.35 Low Comprehension 2 0.87 60 45 42.9 354 365 50.8 1.51 High Comprehension 3 0.67 76 30 28.3 25 192 88.5 10.95 Table 7: A comparison of high comprehension subjects whose choices were consistent with their “mindedness”. Panel A reports univariate measures and their differences between high comprehension subjects that were probability-minded and chose the unknown bet vesus high comprehension subjects that were ambiguity-minded and chose the known bet. Gender is one for males and two for females. Education level is one if the highest attained education is less than high school, two for high school, etc. (as in Table 2. Combined income is similarly scored. The US Indicator is one if country of residence is the United States and zero otherwise. The incentive indicator is one if the elicited choice preference was incentivized and zero otherwise. The Caucasian Indicator is one if the subjected answered “White” for their race and zero otherwise. The NFC score is based on the “Need for Cognition” questionnaire (see Cacioppo, Petty, and Feng, 1984). Panel B documents the results of a Probit regression in which the dependent variable is one if the subject is high comprehension and ambiguity-minded and chose the known bet, and zero if the subject is high comprehension and probability-minded and chose the unknown bet. Coefficients marked with ∗ are significant at the 5% level, while those marked with a ∗∗ are significant at a 1% level or better.

Panel A: Combined Mindedness Statistic Age Gender Education Income Probability mean 50.7 1.6 3.8 5.5 sd 13.8 0.5 1.6 3.7 Ambiguity mean 44.1 1.4 4.7 6.1 sd 14.5 0.5 1.6 4.0 Diﬀerence (t-stat) -3.97 -2.2 4.4 1.2

US Incentive Caucasian NFC Mindedness Statistic Indicator Indicator Indicator Score Probability mean 0.72 0.79 0.89 61.0 sd 0.45 0.41 0.32 12.0 Ambiguity mean 0.69 0.82 0.88 65.0 sd 0.46 0.38 0.33 12.2 Diﬀerence (t-stat) -0.48 0.59 -0.17 2.8

Panel B: Variable Coeﬀ. S.D. Constant 0.28 0.71 Age −0.020∗∗ 0.0062 Gender −0.31 0.18 Education 0.23∗∗ 0.063 Combined income −0.025 0.025 US Indicator −0.25 0.19 Incentive Indicator −0.31 0.25 Caucasian Indicator 0.22 0.25 NFC score 0.015∗ 0.0071 Pseudo R2 0.108