On the Sources of Ordinary Science Knowledge and Extraordinary Science Ignorance

PREPRINT Oxford Handbook on the Science of Science Communication

Dan M. Kahan Yale University It is impossible to make sense of persistent controversy over certain forms of decision-relevant science without understanding what happens in the much greater number of cases in which members of the public converge on the best available evidence without misadventure. In order to live well—or just to live, period—individuals must make use of much more scientific information than any (including a scientist) is in a position to comprehend or verify. They achieve this feat not by acquiring expertise in the myriad forms of science essential to their well-being but rather by becoming experts at recognizing what science knows—at identifying who knows what about what, at distinguishing the currency of genuine scientific understanding from the multiplicity of counterfeit alternatives. A rational form of information processing, their recognition of valid science is guided by recourse to cues that pervade their everyday interactions with other non-experts, whose own behavior convincingly vouches for the reliability of the scientific knowledge on which their own actions depend. Cases of persistent controversy over decision-relevance science don’t stem from defects in public science comprehension; they are not a result of the failure of scientists to clearly communicate their own technical knowledge; nor are they convincingly attributable to orchestrated deception, as treacherous as such behavior genuinely is. Rather such disputes are a consequence of one or another form of disruption to the system of conventions that normally enable individuals to recognize valid science de- spite their inability to understand it. To preempt such disruptions and to repair them when they oc- cur, science must form a complete understanding of the ordinary processes of science recognition, and democratic societies must organize themselves to use what science knows about how ordinary members of the public come to recognize what is known to science.

1. The perils of ignoring the denomi- is scientific consensus by the most reliable measures of such—authoritative pronouncements issued by nator expert panels charged with aggregating and weigh- We live in an age of spectacular public conflict ing the extant evidence for the benefit of decision over policy-relevant facts. Is the earth heating up— makers, individual and collective (e.g., American and if so are humans the cause? Will vaccinating Academy of Pediatrics 2012; Intergovernmental schoolgirls against the human papilloma virus pro- Panel on Climate Change 2014; National Resource mote their health by immunizing them from a can- Council 2004). cer-causing sexually transmitted disease—or put The persistence of deep public division on such their health at risk by lulling them into unprotected facts is rightly understood to pose a significant chal- sex, thereby exposing them to other STDs and teen lenge to the prospects for enlightened self- pregnancy? Do laws allowing private citizens to car- government. Individuals of diverse outlooks can’t ry concealed handguns increase violent crime—or be expected always to agree on the ends of public reduce it by enabling potential victims to defend policymaking. But if, in the face of voluminous themselves and by deterring violent predation more quantities of sound, clearly articulated, and widely generally? disseminated scientific evidence they cannot even Each of these and numerous other factual ques- agree on the facts, how can democratic citizens be tions are subject to intense, sustained political con- expected to identify much less deliberate intelligent- testation. Yet each has also been the subject of ex- ly over the values-tradeoffs that competing policy tensive empirical investigation. Indeed, on each there responses entail? Sources of Ordinary Science Knowledge—and Ignorance 2

Global warming Private gun possession Fracking

Extremely 7 high risk 6

5 r = - 0.65, p < 0.01 r = - 0.55, p < 0.01 r = - 0.50, p < 0.01 4

None at all 0

Very liberal Moderate Very Conservative Very liberal Moderate Very Conservative Very liberal Moderate Very Conservative Strong Democrat Independent Strong Republican Strong Democrat Independent Strong Republican Strong Democrat Independent Strong Republican Artificial food colorings Radio waves from cell phone Genetically modified food

Extremely 7 high risk 6 5 r = - 011, p < 0.05 r = - 0.01, p = 0.56 r = - 0.06, p < 0.01 4 3 2 1

None at all 0

Very liberal Moderate Very Conservative Very16 liberal1 Moderate0 Very1 Conservative16 Very liberal Moderate Very Conservative Strong Democrat Independent Strong Republican Strong Democrat Independent Strong Republican Strong Democrat Independent Strong Republican Residential exposure to magnetic field Use of artificial sweeteners of high-voltage power lines in diet soft drinks Nanotechnology Extremely 7 high risk 6

5 r = - 0.09, p < 0.01 r = - 0.05, p < 0.05 r = 0.04, p = 0.08 4 3 2 1

None at all 0

Figure 1. Risks, polarized and not. Scatterplots relate risk perceptions to political outlooks for members of nationally representative sample (N = 1800), April–May 2014. Source: Kahan (2015b). I will refer to the failure of valid scientific evi- This is a substantial problem in studies that dence to quiet disputes over policy-relevant facts as purport to “explain” controversy over a single issue the Science Communication Problem. The stock of em- (human-caused climate change, for example) by pirical studies examining how this state of affairs examining public opinion on that issue alone. But affects various issues is already ample, and is ex- the problem is barely any smaller for studies that panding at an accelerating pace. The central theme purport to explain the Science Communication of this chapter is that scholarly fixation on such Problem generally by examining only sets of issues controversies itself is inhibiting both scientific in- (climate change plus gun control plus nuclear power vestigation of the source of the Problem and dis- plus gun control plus the HPV vaccine etc.) that covery of effective means for counteracting it. feature it.The simple fact is that the number of issues that actually display the Science Communica- In a narrowly methodological sense, scholarly tion Problem is orders of magnitude smaller than fixation on controversies that feature the Science the number that do not but plausibly could. Diverse Communication Problem involves a form of select- members of the public in the U.S. are not divided ing on the dependent variable. Selecting on the de- over the hazards of chronic exposure to power-line pendent variable occurs when a study design tests a magnetic fields; they aren’t polarized over pasteur- hypothesis with a sample that has been assembled ized milk; they aren’t split on the carcinogenic ef- on the basis of criteria that assume the truth of the fects of artificial sweeteners or food colorings—and hypothesis—or that in any case exclude from the so on and so forth (Kahan 2015b) (Figure 1). A re- sample observations that could well falsify it. searcher who studies only public opinion on global warming or only global warming plus other issues Sources of Ordinary Science Knowledge—and Ignorance 3 that conspicuously feature the Science Communica- more likely to be true) from the merely plausible is tion Problem will never be able to be confident that the principle mission of empirical research (Watts any influence she identifies as affecting those issues 2011). truly is affecting them if she has failed to confirm Four plausible explanations for the Science that those influences are absent in the myriad cases Communication Problem quickly become much less in which diverse members of the public have con- so when we don’t make the error of “ignoring the verged on the best available evidence (or at least denominator” —or systematically averting our em- converged on something). pirical gaze from the almost infinitely large class of But this mistake—one, essentially, of “ignoring cases in which we don’t see public conflict over deci- the denominator” when looking at issues that fea- sion-relevant science but easily could. I’ll call these ture the science of Science Communication Prob- mistakes the four “False Starts,” and give each one lem—shouldn’t be understood in narrow, methodo- its own label: logical terms only. It is in fact symptomatic of I. “The public is irrational”; something much bigger, much more fundamental: a II. “First thing we do—let’s kill all the cultural science of science communication literacy deficit. scientists”; The sorts of explanations for the Science Commu- III. “Welcome to the age of science de- nication Problem that appear plausible when re- nial”; and searchers and others confine their attention solely to IV. “The public is being manipulated.” controversies that feature conspicuous instances of the Science Communication Problem are ones that, 2.1 . The most plausible—and also most com- to be blunt, reflect widespread and long-standing monly asserted—explanation for the Science Com- misunderstandings of the ordinary processes by munication Problem is the public’s limited capacity which the public comes to know what is known by to comprehend science. The public is only modestly science. Forms of science communication that re- science literate. About half, we are regularly remind- flect these misunderstandings are at best ineffectual ed, understands that the earth orbits the sun in a and at worst highly counter-productive. year as opposed to a day (National Science Founda- Or so I will argue. The remainder of this essay tion 2016); less than a quarter knows that nitrogen is will be divided between efforts to illustrate the sorts the most common gas in the earth’s atmosphere of mistakes we are likely to make if we “ignore the (Pew 2013); less than ten percent can make sense of denominator” when investigating the Science a two-by-two contingency table essential to deter- Communication Problem, the focus of Part 2; and, mining the ratio of true- to false-positives when in Part 3, efforts to furnish a glimpse of the insights assessing medical test results (Kahan 2016). So how into the Problem that can be achieved when re- can members of the public possibly be expected to searchers examine the sources of ordinary science understand what scientists are saying when scientists knowledge—that is, the influences that enable the try to explain complex issue like climate change or public to agree about so much of what science nuclear power? knows. Part 4 identifies what juxtaposing ordinary More importantly still, members of the public science knowledge and the Science Communication don’t think the way scientists do. They rely on rapid, Problem implies about the proper focus for the intuitive, affect-driven sources of information pro- emerging field of “science of science communica- cessing to the exclusion of the deliberate, conscious, tion.” analytic ones essential to making appropriate judgments of risk. As a result, they tend to overestimate 2. Four false starts the magnitude of more emotionally charged calami- ties (terrorists attacks, e.g.) and discount actuarially What makes persistent public disagreement more consequential but more temporally or emo- over policy-relevant facts so puzzling is not the tionally remote ones (e.g., the impact of human- scarcity of explanations for this phenomenon; on caused climate change). They also are more likely to the contrary, it is the surfeit of them. The number rely on defective heuristics, such as crediting the of plausible mechanisms that might account for the opinions of their peers, a form of reasoning that can Science Communication Problem far exceeds the trigger self-reinforcing states of polarization (e.g., number that could be true. Separating the true (or Marx, Weber et al. 2007; Sunstein 2005, 2007). Sources of Ordinary Science Knowledge—and Ignorance 4 The “public irrationality thesis” or PIT, let’s call While one can make a compelling normative it, is plausible. But if we don’t ignore the denomina- case for scientists speaking with greater clarity (Ol- tor, we’ll quickly see that it is indeed a false start. son 2009; Dean 2009) or with less certitude about policy outcomes (Lempert, Groves & Fischbach What attending to the denominator allows us to 2013), the idea that how scientists talk is the cause see is how many more instances of the Science of the Science Communication Problem is palpably Communication Problem we’d expect to see if PIT unconvincing (see Atkin and Scheuffele, Chapter were the explanation for it. Why aren’t members of X). Again, all one has to do is look at the myriad the public either politically fragmented over, or uni- science issues that don’t provoke persistent contro- formly anxious about, medical x-rays, for example? versy. How about raw milk (Sci., Media, & Public The answer cannot be that members of the public Res. Group 2016)? Is there some reason to believe are experts on nuclear science; half of them think biologists have been doing a better job explaining (or incorrectly guess) that “atoms” are “smaller than pasteurization than climate scientists have been do- electrons” (National Science Foundation 2016). It is ing explaining the greenhouse effect? What folksy also clear that “radiation” generally makes them idioms or tropes did the former use that were so queasy—an affective response that accounts for effective in quieting political polarization? Or was it widespread fear of nuclear power (e.g., Peters, Bur- that they just were more genuinely neutral on raston & Mertz 2004). whether people should drink their milk straight up from the cow’s udder? Or how about nanotechnology? Although still Here, obviously, I’m relying on a cascade of routinely applied, the label “emerging technology” rhetorical questions in lieu of evidence. But the ab- clearly no longer fits: the last decade has seen the sence of evidence is my evidence. No one has ever steady introduction of nanotechnology consumer thought it worthwhile to construct a statistical mod- goods, which now total some 2000 (Vance, Kuiken el to test whether the difference between public et al. 2015). One might have imagined (or fretted) acceptance of, say, the dangers of ozone depletion, that the “grey goo” doomsday scenario popularized on the one hand, and those of human-caused cli- by K. Eric Drexler’s 1986 book Engines of Creation mate change, on the other, correlates with the clarity would furnish affectively rich soil for growing pub- and policy-neutrality of the National Academy of lic risk perceptions. But environmental activists Sciences’ respective reports on those issues (Na- have been trying to cultivate such fears for over a tional Research Council 1976, 1982, 2008, 2011); or decade (e.g., “Green Goo: Nanotechnology Comes whether the difference between how readily, and Alive” 2003) with zero success. The reason isn’t that how rapidly, states added the adolescent HBV vac- the public has been inoculated with valid scientific cination to their mandatory school-enrollment im- information on nanotechnology, over 75% of munization schedules, on the one hand, and how Americans—a proportion that has not moved in persistently they have resisted adoption of the HPV over a decade—says that it knows little or nothing vaccine, on the other (Kahan 2013), correlates with about it (Liang, Ho et al. 2015). the clarity and policy-neutrality of the American 2.2. A related false start blames scientists. If de- Academy of Pediatrics’ endorsements of both spite the clarity of the evidence, members of the (American Academy of Pediatrics 1992, 2007). public aren’t converging on some policy-relevant Likely no one has because it’s clear to the naked facts, the reason must be that scientists are failing to ear that what these groups of scientists had to say convey the evidence clearly enough (e.g., Brownell, on the uncontested members of these societal-risk Price & Steinman 2013). Or maybe they are speak- pairs (ozone depletion, the HBV vaccine) was no ing out too clearly, crossing the line from factfinder less obscure and no less opinionated than what they to policy advocate in a manner that compromises had to say about the contested ones (climate their credibility (e.g., Tamsin 2013). Or perhaps change, the HPV vaccine). But whatever the source what is compromising their credibility is how cagily of the omission, including only contested cases in they are hiding their advocacy by implausibly assert- the sample and leaving uncontested ones out neces- ing that the facts uniquely determine particular poli- sarily defeats any valid inference about how the cy outcomes (e.g., Fischoff 2007). “obscurity” or “partisanship” of scientists’ own words affects the Science Communication Problem. Sources of Ordinary Science Knowledge—and Ignorance 5

Even if it brings no immediate benefits, Public assessment of amount of scientific research that advances the government spending for scientific frontiers of knowledge is necessary and research: 1981–2014 should be supported by the federal government. 50% 100% too little 40% 75% agree

30% 50% 20%

25% 10% disagree too much

0% 0% '81 '85 '88 '90 '92 '97 '99 '01 '02 '04 '06 '08 '10 '12 '14 '85 '88 '90 '92 '97 '99 '01 '02 '04 '06 '08 '10 '12 '14

Figure 2. Select NSF Indicators’ science attitude items. Source: NSF (2016). 2.3. Another false-start account of the Science caused climate change or human evolution just is Communication Problem attributes it to growing evidence of a deficit in the cultural authority of sci- resistance to the authority of science itself. Along ence. with widespread disbelief in evolution, political con- But in that case, what started out as an explana- flict over global warming or other issues is variously tion for the Science Communication Problem has depicted as evidence of either the “anti-science” transmuted, ironically, into a piece of evidence- sensibilities of a particular segment of the public impervious dogma. Is it possible that some influ- (e.g., Mooney 2012) or of a creeping anti-science ence wholly unrelated to the authority of science strain in American culture generally (e.g., Frank accounts for the peculiarly contested status of this 2013). collection of issues? The question is now ruled out Anyone who manages to divert his gaze from by definitional fiat. the Science Communication Problem is sure to spy 2.4. Advocates seeking to mislead the public on evidence massively out of keeping with this account. issues that feature the Science Communication In its biennial Science Indicators series, for example, Problem are legion. But this does not in itself estab- the National Science Foundation (2016) includes lish that orchestrated misinformation campaigns survey measures that consistently evince effusive cause the problem. Indeed, studies that purport to degrees of confidence in and support for science find causation by focusing on misinformation on (Figure 2). These levels of support do not vary climate change or other contested issues furnish meaningfully across groups defined by their political textbook examples of selecting on the dependent outlooks or degrees of religiosity (Figure 3). Indeed, variable. The excluded observations, again, are other the levels of support are so high that it would be science issues that do not fit the profile associated impossible for them to harbor practically significant with the Science Communication Problem. The levels of variance across groups of any substantial inference that misinformation causes the Problem’s size. For behavioral validation of these sensibilities, signature form of persistent contestation is warrant- all one has to do is look at the care-free confidence ed only if (among other things) examination of rele- individuals evince in science when making decisions vant cases that lack such contestation reveals the both mundane (the ingestion of a pill to preempt absence of comparable sources of misinformation. hair loss) and vital (submission to radiation therapy If, in contrast, we discover that issues unplagued by for cancer). Because this evidence is so obvious, it’s the Science Communication Problem are not free of less likely proponents of the “age of denial” thesis the scourge of misinformation, then we will have don’t see it than that they see it as irrelevant. On reason to doubt that misinformation furnishes a this view, confusion over or rejection of the disposi- satisfactory explanation of why some issues are tive evidence that science has collected on human- characterized by this problem. Sources of Ordinary Science Knowledge—and Ignorance 6 “Would you say that, on balance, the benefits of scientific research have outweighed the harmful results, or have the harmful results of scientific research been greater than its benefits?” 100% 100% 100% 100% Harms greater Harms greater 90% 90% 90% 90%

80% 80% 80% About Equal 80% About equal 70% 70% 70% 70%

60% 60% 60% 60% Benefits greater Benefits greater 50% 50% 50% 50%

40% 40% 40% 40%

30% 30% 30% 30%

20% 20% 20% 20%

10% 10% 10% 10%

0% 0% 0% 0% Very liberal Liberal Moderate Conservative Very Conservative -1.7 -1 0 1 1.7 Democrat Democrat Independent Republican Republican Very low Low Average high Very high Left_right political orientation Religiosity Figure 3. Science attitude by political orientation and religiosity. Source: Original analysis of General Social Survey 2006, 2008, 2010, 2012. N’s = 4653 (left_right), 4607 (religiosity). “Left_right” is political outlook scale formed by aggregation of 7-point liberal- conservative ideology and 7-point partisan identification items (α = 0.65). “Religiosity” is standardized score on scale formed by aggregation of self-reported religiosity, frequency of prayer, and church attendance (α = 0.79). In fact, parties seeking to mislead the public on man, Cuite & Morin 2013). When specifically asked, issues unaffected by the Science Communication bewildered survey respondents say they prefer the Problem are legion, too. Likely the most compelling issue of whether and how to regulate GM foods example involves GM foods, which have been re- simply be left to expert regulators to deal with as peatedly found to be as safe as their conventionally those experts see fit (McFadden & Lusk 2016). grown equivalents (National Research Council Of course, as ample as their efforts to sow con- 2016). Never mind the $25 million that GM labeling fusion have been (e.g. Kloor 2012), supporters of proponents spent merely to strike out on public GM food regulation have no doubt spent less to referenda in three politically liberal states— Oregon, mislead citizens than have, say, opponents of cli- Washington, and California, the last of which de- mate change regulation. But the more apt compari- feated a labeling initiative in a year in which Presi- son is with the efforts of GM food opponents in dent Obama crushed his Republican rival Mitt Europe, where comparable investments have paid Romney by twenty percentage points (Tims 2014; handsome dividends: high levels of anxiety about Chokshi, 2013; Flynn 2012). Forget, too, that the GM technology in popular opinion, and the regula- U.S. Congress has now passed into law a federal tory exclusion of GM products from consumer statute that these groups bitterly fought and that will markets (Sato 2007). Manipulating public opinion is preempt any further attempts by them to mandate not as easy as it looks—or at least not as easy as one on-package labelling (Bjerga & Keane 2016). Those might infer if one makes the mistake of examining results have marked only the final step in the long only cases that feature the Science Communication march of futility for GM foes in trying to arouse Problem and ignore all the rest. public concern. After twenty years of persistent agi- tating, anti-GM groups have not only failed to gen- Nothing in this account by any means suggests erate the level of public anxiety that has historically that efforts to mislead the public are never of con- surrounded issues such as nuclear power, toxic sequence in cases that feature the Science Commu- waste disposal, and air and water pollution in the nication Problem. But obviously there are other U.S.; they have failed to even produce discernable influences that determine when misinformation mat- comprehension that GM foods exist at all. Less than ters and how much. Unless one’s sample includes a 50% of the public realizes that GM foods are al- fair representation of issues that don’t feature the ready on supermarket shelves (in 75% of every food Science Communication Problem, no amount of product on sale there, in fact), and only 25% (in- data collection will help to identify what those influ- credibly!) that they have ever consumed any (Hall- ences are. Sources of Ordinary Science Knowledge—and Ignorance 7 3. Four theses on ordinary science ommendation is sound before you undertake such knowledge treatment. But what will you do? Will you carefully read So far I’ve emphasized how easily one can cred- and evaluate all the studies that inform your physi- it false accounts of the Science Communication cian’s recommendation? If those studies refer, as Problem if one focuses only on instances of it. I they inevitably will, to previous ones the methods of now want to pivot the analysis and offer a glimpse which aren’t reproduced in those papers, will you of how much insight can be gained by studying read those, too? If the studies you read refer to con- what is going on in the much larger class of cases cepts with which you aren’t familiar, or use methods that evade the Problem’s signature form of contes- which you have no current facility, will you enroll in tation. a professional training program to acquire the nec- The exposition will be admittedly skeletal. It, essary knowledge and skills? And once you’ve done too, consists in nothing more than four proposi- that, will you redo the experiments—all of them; tions. These relate to ordinary science knowledge—or not just the ones reported in the papers that support the normal state of collective convergence by mem- the prescribed treatment but also those relied on bers of the public on what is known by science: and extended by them—so you can avoid taking anyone’s word on what the results of such studies “Individuals must accept as known I. actually were as well? more decision relevant science (DRS) than they can possibly understand or Of course not. Because by the time you did verify for themselves”; those things, you’d be dead. To live well—or just to II. “Individuals acquire the insights of live—individuals (including scientists) must accept DRS by reliably recognizing it”; much more DRS than they can ever hope to make III. “Public conflict over DRS is a recog- sense of on their own. nition problem, not a comprehension Science’s way of knowing involves crediting as problem”; and true only inferences rationally drawn from observa- “The recognition problem reflects a IV. tion. This was—still is—a radical alternative to oth- polluted science communication envi- er ways of knowing that feature truths revealed by ronment.” some mystic source to a privileged few, who alone Even when the argument is presented in this enjoy the authority to certify the veracity of such form, however, I expect it to bear sufficient weight insights. That system is what the founders of the to show that adding empirical muscle to it—by Royal Society had in mind when they boldly formu- making ordinary science knowledge the focus of lated their injunction to “take no one’s word for it.” scientific conjecture and refutation (Popper But it remains the case that to get the benefits of 1962a)—is the best way to advance the project to the distinctive, and distinctively penetrating, mode understand and to solve the Science Communica- of ascertaining knowledge they devised, we must tion Problem. take the word of those who know what’s been ascer- tained by those means—while being sure not to take 3.1. The motto of the Royal Society is Nullius in the word of anyone else (Shapin 1994). verba, which translates literally into “take no one’s word for it.” But something—namely, any pretense 3.2. But how exactly does one do that? How of being a helpful guide to getting the benefits of do reasonable, reasoning people who need to use scientific knowledge—is definitely lost in a transla- science for an important decision but who cannot tion that literal. plausibly figure out for themselves what science knows figure out who does know what science If you aren’t vigorously nodding your head, knows? then consider this possibility. You learn next week that you have an endocrinological deficit that can be We can rule out one possibility right away: that effectively treated but only if you submit to a regi- members of the public figure out who genuinely men of daily medications. You certainly will do possesses knowledge of what science knows by enough research to satisfy yourself—to satisfy any evaluating the correctness of what putative experts reasonable person in your situation—that this rec- believe. To do that, members of the public would Sources of Ordinary Science Knowledge—and Ignorance 8 have to become experts in the relevant domain of tions, and so on” (ibid., p. 30). To be sure, powers knowledge themselves. Again, it is obvious that they of critical reasoning play a role. We must calibrate lack both the capacity and time to do that. this facility of recognition by “learning how to criti- cize, how to take and to accept criticism, how to Instead they have to become experts at recogniz- respect truth” (ibid, p. 36), a view Baron (1993) and ing valid sources of science. They become expert at Keil (2003, 2010, 2012) both develop systematically. that, moreover, in the same way they become expert at recognizing anything else: by using a conglomera- But the objects of the resulting power to discern tion of cues, which operate not as necessary and valid science are not the qualities that make it valid: sufficient conditions, but as elements of prototypi- those are simply far too “complex,” far too “diffi- cal representations (e.g., “cat,” “advantageous chess cult for the average person to understand (Baron, position,” “ice cream sandwich,” “expert”) that are 1993, p, 193). What this faculty attends to instead summoned to mind by mental processes, largely are the signifiers of validity implicit in informal, everyday unconscious, that rapidly assimilate the case at hand social processes that vouch for the good sense of rely- to a large inventory of prototypes acquired through ing on the relevant information in making im- experience. In a word (or two words), they use pat- portant decisions (Keil 2010, 2012). Popper charac- tern recognition (Margolis 1993). terizes the aggregation of these processes as “tradition,” which he describes as “by far the most im- This is equivalent to the answer that Popper portant source of our knowledge” (1962b, p. 36). gave (in an essay the title, and much more, of which are the inspiration for this one) in answering the It is worth noting that although Popper here is near-identical question about how we come to referring to the process by which ordinary science know what is known by science. Popper’s target was knowledge is disseminated to nonscientists, there is a cultural trope of sensory empiricism that treated no reason to think that scientists are any less in as “scientific knowledge” only what one has ob- need of their own valid-science recognition capacity, served for oneself. After impaling this view on the or that they acquire or exercise it in a fundamentally speartips of a series of reductios, Popper explains different way. Indeed, there is ample reason to think that most things we know”—i.e., know to be known that it couldn’t possibly differ from the faculty that to science—“we have learnt by example, by being members of the public use to recognize valid sci- told.” In appraising the conformity of any such ence (Shapin 1994) aside from its being more finely piece of information to the qualities that invest it calibrated to the particular insights and methods with the status of scientific knowledge, moreover, needed to be competent in the production of the an individual must rely on “his knowledge of per- same (Margolis 1987, 1996). sons, places, things, linguistic usages, social conven-

There is “solid evidence” of recent global warming due “mostly” to “human activity such as burning fossil fuels.” [agree, disagree]

1 1 Liberal .9 .9 Democrat .8 .8 .7 .7 .6 .6 .5 .5 .4 .4 Conservative .3 .3 Republican .2 .2 .1 .1 Probability of correct response correct of Probability 0 response correct of Probability 0 1st percentile 16th percentile 50th percentile 84th percentile 99th percentile 1st percentile 16th percentile 50th percentile 84th percentile 99th percentile Ordinary Climate Science Intelligence Ordinary Climate Science Intelligence

Figure 4. Belief in human-caused climate change in relation to performance on climate science literacy assessment and political outlooks. Source Kahan (2015a). N = 2000, nationally representative sample. “Ordinary Climate Science Intelligence” is 9- item climate-science literacy test. Scores for “Liberal Democrat” and “Conservative Republican” based on logistic regression model. Colored bars denote 0.95 CIs. Sources of Ordinary Science Knowledge—and Ignorance 9 “How do we gain our knowledge about how to myriad other risk sources about which members of analyze data?” ask Andrew Gelman and Keith the public know just as little and on which they are O’Rourke (2015, pp., 161-2). By “informal heuristic no better equipped to analyze data. The Second reasoning,” they reply, of the sort that enables those Thesis posits that where the Science Communica- immersed in a set of practice to see the correctness tion Problem is absent, members of the public con- of an answer to a problem before, and often with- verge on the best evidence not by comprehending its out ever being able to give a fully cogent account of, validity but rather but by recognizing it. why. If these claims are correct, then we should ex- 3.3. Now we can start to put things together. pect the Science Communication Problem to be one The First False Start, PIT, attributes the Science that affects the capacity of the pubic to recognize Communication Problem to defects in the public’s valid science, not their capacity to comprehend it. capacity to understand science. Challenging this That is a proposition easily tested. position, I noted the absence of contestation over

Global warming Private gun possession Fracking Extremely high 7 risk Liberal Liberal Democrat 6 Democrat Liberal Democrat 5

4 Conservative Republican 3 Conservative Republican 2 Conservative Republican 1 None at all 0 1st pcerntile 50th percentile 99th percentile 1st pcerntile 50th percentile 99th percentile 1st pcerntile 50th percentile 99th percentile Ordinary science intelligence Ordinary science intelligence Ordinary science intelligence User exposure to radio waves from cell Artificial food colorings Genetically modified food phones Extremely high 7 7 risk 6 6 6

Liberal 5 5 5 Liberal Democrat Democrat

4 4 Liberal 3 3 Conservative Democrat 3 Conservative Republican Conservative

2 Republican 2 Republican

1 1 0 None at all 0 0 1st pcerntile 50th percentile 99th percentile 1st pcerntile 50th percentile 99th percentile 1st pcerntile 50th percentile 99th percentile Ordinary science intelligence Ordinary science intelligence Ordinary science intelligence

Residential exposure to magnetic field Use of artificial sweeteners in diet soft drinks Nanotechnology of high-voltage power lines Extremely high 7 risk 6 Liberal 5 Liberal Democrat Democrat 4 Conservative 3 Republican Conservative 2 Conservative Republican Republican Liberal 1 Democrat

None at all 0 1st pcerntile 50th percentile 99th percentile 1st pcerntile 50th percentile 99th percentile 1st pcerntile 50th percentile 99th percentile Ordinary science intelligence Ordinary science intelligence Ordinary science intelligence Figure 5. Impact of increased science comprehension on political polarization of select risk items. Source Kahan (2015a). N = 2000, nationally representative sample. “Ordinary Science Intelligence” is 18-item -science literacy test. Scores for “Liberal Demo- crat” and “Conservative Republican” based on linear regression model. Colored bars denote 0.95 CIs.

Sources of Ordinary Science Knowledge—and Ignorance 10 The tests that can be performed are two, First, consensus is across the run of issues on which con- one can examine whether lower degrees of science tending cultural groups are divided (Kahan, Jenkins- comprehension predict disagreement on issues af- Smith & Braman 2011). This result is contrary, of fected by the Science Communication Problem. course, to the Third False Start, which asserts that we’ve entered an “age of science denial” since it The answer is no. There is zero correlation, in shows that no group understands itself to be “reject- fact, between belief in human-caused climate ing” the weight of scientific opinion on any of the change and how individuals perform on a valid issues featured in the Science Communication Prob- “climate science literacy” test (Kahan 2015a) (Figure lem. That these groups are forming such unreliable 4). Indeed, political polarization doesn’t abate as judgments on what scientists’ consensus is, howev- individuals become more proficient in numeracy, er, suggests that something is disabling their mem- cognitive reflection, and other capacities essential to bers from correctly assessing information about accurate risk perception increase; on the contrary, it what genuine experts believe on these matters. increases (Kahan & Peters 2012; Kahan & Peters et al. 2013; Kahan 2013; Kahan 2015a) (Figure 5). That inference has been confirmed experimen- tally. When evaluating a highly credentialed scientist, The second test examines the relationship be- individuals of opposing cultural identities will con- tween issues that feature the Science Communica- form their assessment of the expertise of that scien- tion Problem and individuals’ apprehension of the tist within the relevant domain to whether he was indicia of valid science. This evidence suggests that depicted as supporting or opposing the position disagreement about what evidence counts as valid that predominates within the subjects’ own groups. and what it implies is precisely what accounts for In other words, rather than treating the view of a the persistent contestation associated with the Sci- particular expert scientist as reason to adjust their ence Communication Problem. assessment of the weight of scientific opinion on Thus, members of the groups that disagree the issue at hand (whether human beings are caus- about key facts on human-caused climate change, ing global warming; whether laws allow carrying of nuclear power, and gun control, for example, disa- concealed weapons in public increases or decreases gree just as sharply about what scientific consensus crime; whether nuclear wastes can be safely deposit- is on those facts. In addition, if we treat, say, the ed in deep geologic isolation), study subjects use National Academy of Sciences’ expert consensus their current position to determine what weight to reports as a benchmark, no group is any more likely give the view of any particular scientist (ibid) than any other to be correct about what scientific (Figure 6).

Figure 6. Biased recognition of scientific expertise. Source: Kahan, Jenkin-Smith, Braman (2011). N = 1336, nationally representative sample. Based on logistic regression model. Slopes indicate probability of recognizing that featured scientist is an expert within climate-change domain conditional on attributed position and subject political outlook. Colored bars reflect 0.95 confidence intervals. Sources of Ordinary Science Knowledge—and Ignorance 11 Note, too, that this style of reasoning dispenses cultural commitments or basic understanding of the with the need for anyone to engage in misinfor- best way to live. The suggestion that direct commu- mation—the fourth False Start—in order to exploit nication with scientists is more consequential re- the Science Communication Problem. No one is flects either the First or Second False Start or both: doing anything to misinform the subjects in this individuals have neither the time nor the capacity to study; the subjects are aggressively misinforming them- extract information directly from scientists. Much selves by selectively crediting or discrediting evidence more accessible, and much more readily subject to on what scientists believe in patterns that cohere meaningful interpretation, are words and actions of with the positions associated with their group iden- other ordinary people, whose use of DRS vouches for tities. their confidence in it as a basis for decision. That is a style of reasoning in the nature of con- Indeed, it vouches as effectively when nothing is firmation bias. When indulged in outside the lab, it said about it as it does when something is. Noth- will result in the pattern of persistent disagreement ing—including a new National Academy of Sciences about scientific consensus that we observe in the expert consensus report (National Research Council world. It is a pattern of information processing, 2016) that few members of the public will ever rec- moreover, that systematically blocks diverse mem- ognize exists—will as effectively assure an ordinary bers of the public from giving effect to cues that person that it is safe to eat GM corn chips as will would otherwise be expected to help them converge watching his best friend and his brother-in-law and in their recognition of who knows what about what. his officemate eating them without giving the matter a second’s thought; seeing what these models of 3.4. The species of pattern recognition that normal behavior do is the “all clear” signal that ob- non-expert members of the public normally use to viates the need for the vast majority of Americans recognize valid science enables them to get the ben- even to bother learning that the corn chips they are efit of substantially more scientific insight than any eating contain GM foods (Hallman, Cuite & Morin could possibly hope to comprehend. The evidence I 2013). Social proof (Aronson 1999) is the dominant described in the last section, however, shows how mode of communicating the reliability of decision- this critical capacity can be disabled. The final “the- relevant science for non-expert members the public. sis of ordinary science knowledge” identifies the source of this disablement: a polluted science communi- Of course, ordinary citizens don’t interact only cation environment. with those with whom they share important cultural commitments. But they interact with them much As noted, Popper (1962b) attributes the acquisi- more than they interact with others, for the simple tion and exercise of the capacity for science- reason that they find their company more congenial recognition to immersion in a set of social processes and more conducive to all manner of profitable in- and conventions. When I refer to the science communi- tercourse. They are also less likely to waste time cation environment, I mean the sum total of the pro- squabbling with these people, and can also read cesses and conventions that enable recognition of them more reliably, distinguishing those who really valid science in this way (Kahan 2015b). Any influ- do know what science knows from those who are ence that impairs or impedes the operation of these only posers. It is perfectly rational, then, for them to social practices will degrade the power of free, rea- consciously seek guidance from those who share soning citizens to recognize valid science, and hence their outlooks, and to form unconscious habits of to fully realize its benefits. As a result, we may un- mind that privilege these individuals as sources of derstand any such influence to be a form of pollution guidance on what science knows (Kahan 2015b). in the science communication environment. This process is admittedly insular, but it clearly The sorts of influences that can generate such works in the main. All of the major cultural groups in disablement are no doubt numerous and diverse. which this process operate are amply stocked both But I will focus on one, which degrades an especial- with members high in science comprehension and ly consequential cue of science validity. with intact social processes for transmitting what Of all the sources of ordinary science they know. No group that lacked these qualities— knowledge, by far the most significant are individu- and that as a result regularly misled its members on als’ interactions with others with whom they a share the content of valid DRS—would last very long! On Sources of Ordinary Science Knowledge—and Ignorance 12 issues that don’t display the profile of the Science son but by conscripting it into the service of advancing Communication Problem, those highest in science their group’s cause in a demeaning form of cultural proficiency do tend to converge on the best available status competition. IPC does not create the role that evidence, and no doubt pull the other members of social influences play in popular recognition of what their groups along in their wake (Figure 5). science knows. Rather it corrupts them, transform- ing the role that spontaneous, everyday social inter- But such a system is vulnerable to a distinctive actions play from an engine of convergence on the pathology: identity-protective cognition (IPC). IPC occurs best available evidence into a relentlessly aggressive when a policy-relevant fact that admits of empirical agent of public dissensus over what scientific con- inquiry becomes entangled in antagonistic social sensus really is. meanings that transform positions on them into badges of identity in, and loyalty to, competing cul- 4. Understanding and protecting the sci- tural groups (Kahan 2010, 2012). The cost under ence communication environment those conditions of forming factually incorrect beliefs on matters such as whether humans are heating up I will conclude by adding to the “four theses of the earth or whether fracking will exhaust or con- ordinary science knowledge” a fifth: taminate drinking water sources is essentially zero: individuals’ personal views and actions are not con- V. Protecting the science communication sequential enough to affect the level of risk they environment from contamination is the face, or the likely adoption of ameliorating (or simp- principal aim of the science of science ly pointless or even perverse) regulatory responses. communication. But given what beliefs on these subjects (correct or The Fifth Thesis places an analytical capstone incorrect) have come to signify about the kind of atop the arc of the argument this essay has present- person one is—about whose side one is on, in what ed. I began by showing that “ignoring the denomi- has become a struggle for status among competing nator”—fixating on the relatively small number of cultural groups—the personal cost of forming the cases that feature persistent public conflict over wrong ones in relation to one’s own cultural identity policy-relevant facts and ignoring the vastly larger could be punishingly high (Kahan, Peters et al. number of ones that feature convergence— 2012). predictably generates insupportable explanations of In such circumstances, individuals can be ex- the Science Communication Problem. Next I exam- pected to use their reason to form and persist in be- ined how attending to the dynamics that account for liefs that reliably vouch for their group identities the normality of public convergence on what is regardless of whether those beliefs are factually accu- known by science generates a much better explana- rate. This conclusion is consistent with numerous tion: the disruption of the everyday social processes studies, observational (Bolsen, Druckman & Cook productive of the cues that certify the validity of 2015, 2013; Gollust, LaRussao et al. 2015; Gollust, decision relevant science. A research program aimed Dempsey, et al. 2010) and experimental (Kahan, at managing those influences—at neutralizing the Braman, et al. 2009, 2010), that link IPC to the Sci- disruptive influence that they exert on the sources ence Communication Problem’s signature forms of of ordinary science knowledge—is ideally calculated polarization. Indeed, individuals who enjoy the high- to test this theory and thus advance knowledge of est level of proficiency will display this form of motivat- how people come to know what is known by sci- ed reasoning to the greatest extent, precisely be- ence. cause they will be the most adept at using their rea- But more fundamentally, the Fifth Thesis is in- soning proficiency to secure the interest that they tended to summon the energy and focus needed for share to form identity-expressive beliefs (Kahan in the science of science communication to perfect press). enlightened self-government. As Popper (1945) also In sum, the antagonistic social meanings that famously made clear, liberal democracy is the politi- trigger IPC are a toxic form of pollution in the sci- cal regime must congenial to the advancement of ence communication environment of culturally plu- scientific knowledge. The privileged status it affords ralistic societies. They disable individuals’ science- individual freedom of thought fuels the engine of recognition capacities, not by diminishing their rea- scientific conjecture and refutation and removes any Sources of Ordinary Science Knowledge—and Ignorance 13 bureaucratic impediment to its relentless conver- Aronson, E. The Social Animal (Worth Publishers, gence on truth. New York, N.Y., 1999). But precisely because the highly diverse citizens Baron, J. Why Teach Thinking? An Essay. Applied of this regime will not submit to a central authority’s Psychology 42, 191-214 (1993). certification of the truth (Nullius in verba!), they face Brownell, S.E., Price, J.V. & Steinman, L. Science a distinctive epistemic challenge. The problem is not communication to the general public: why we the absence of a social system of truth certification; need to teach undergraduate and graduate stu- they have one—“tradition”—without which scien- dents this skill as part of their formal scientific tific “knowledge would be impossible” (Popper training. J. Undergraduate Neuroscience Educ. 12, 1962b, p. 36). But in a pluralistic liberal regime, E6 (2013). there will necessarily be a plurality of certifiers—a plurality of communities, bonded by their shared view Bjerga, A. & Keane, A.G. Obama REady to Sign of the good, whose interactions generate the multiple Food-Label Bill Consumer Groups Dislike. “traditions,” immersion in which calibrates the rea- Bloomberg.com (July 13, 2016), available at soning faculties their members use to recognize val- http://www.bloomberg.com/politics/articles/ id science. Conflicts among these plural communi- 2016-07-13/obama-ready-to-sign-food-label- ties of certification, even if rare, are statistically cer- bill-consumer-groups-find-suspect tain to arise as their members’ independence of Chokshi, N. Big corporate spending pays off in mind and freedom of action continuously enlarge Washington’s genetically modified food fight. the stock of scientific knowledge at their disposal. Wash. Post (Nov. 6, 2013), The threat IPC poses to the science communication https://www.washingtonpost.com/blogs/gov environment is intrinsic to the very conditions that beat/wp/2013/11/06/big-corporate- make liberal democratic societies so distinctively fit spending-pays-off-in-washingtons-genetically- for producing scientific knowledge. modified-food-fight/. But this is not an “inherent contradiction” in Dean, C. Am I making myself clear? : a scientist's guide to the constitution of the Liberal Republic of Science. talking to the public (Harvard University Press, It is just a problem to be solved—with the instru- Cambridge, Mass., 2009). ment best suited for solving any problem that threatens the welfare of its citizens: the power that Doh, J.P. & Guay, T.R. Corporate Social Responsi- science’s distinctive method of disciplined observa- bility, Public Policy, and NGO Activism in tion and inference confers on human reason. Assur- Europe and the United States: An Institution- ing that the citizens of liberal democratic regimes al-Stakeholder Perspective. Journal of Manage- get the full benefit of the knowledge that their way ment Studies 43, 47-73 (2006). of life makes possible is what the Science of Science Drexler, K.E. Engines of creation (Anchor Communication is for (Kahan 2015b). Press/Doubleday, Garden City, N.Y., 1986). References Edwards, T. Climate scientists must not advocate particular policies. The Guardian, July 31 (2013). American Academy of Pediatrics, Committee on Infectious Diseases. Universal hepatitis B im- Fischhoff, B. Nonpersuasive Communication about munization. Pediatrics 8, 795-800 (1992). Matters of Greatest Urgency: Climate Change. Environmental Science & Technology 41, 7204-7208 American Academy of Pediatrics. HPV Vaccine (2007). Does Not Lead to Increased Sexual Activity (2012), http://tinyurl.com/jrjx37o Flynn, D. GM Food Labeling in California Goes Down in Defeat. Food Safety News. (Nov. 7, American Academy of Pediatrics. Prevention of 2012), at Human Papillomavirus Infection: Provisional http://www.foodsafetynews.com/2012/11/bi Recommendations for Immunization of Girls g-setback-for-right-to-know-about-gm-foods- and Women With Quadrivalent Human Papil- prop-37-goes-down-in-crushing- lomavirus Vaccine. Pediatrics 120, 666-668 defeat/#.V1p5L7srJGr. (2007). Sources of Ordinary Science Knowledge—and Ignorance 14 Frank, A. Welcome to the age of denial. N.Y. Times Kahan, D.M. Climate-Science Communication and (Aug. 21, 2013), available at the Measurement Problem. Advances in Political http://www.nytimes.com/2013/08/22/opinio Psychology 36, 1-43 (2015a). n/welcome-to-the-age-of-denial.html?_r=0. Kahan, D.M. Ideology, Motivated Reasoning, and Gelman, A. & O’Rourke, K. Convincing Evidence. Cognitive Reflection. Judgment and Decision in Roles, Trust, and Reputation in Social Media Making 8, 407-424 (2013). Knowledge Markets: Theory and Methods (ed. E. Kahan, D.M. The Politically Motivated Reasoning Bertino & A.S. Matei) 161-165 (Springer In- Paradigm. Emerging Trends in Social & Behavioral ternational Publishing, Cham, 2015). Sciences, (in press), Gollust, S.E., Dempsey, A.F., Lantz, P.M., Ubel, http://papers.ssrn.com/sol3/papers.cfm?abstr P.A. & Fowler, E.F. Controversy undermines act_id=2703011 support for state mandates on the human pap- Kahan, D.M. What is the "science of science com- illomavirus vaccine. Health Affair , 2041- 29 munication"? J. Sci. Comm, 14, 1-12 (2015b). 2046 (2010). Kahan, D.M., Braman, D., Slovic, P., Gastil, J. & Gollust, S.E., LoRusso, S.M., Nagler, R.H. & Cohen, G. Cultural Cognition of the Risks and Fowler, E.F. Understanding the role of the Benefits of Nanotechnology. Nature Nanotech- news media in HPV vaccine uptake in the nology 4, 87-91 (2009). United States: Synthesis and commentary. Human vaccines & immunotherapeutics, 1-5 (2015). Kahan, D.M., Jenkins-Smith, H. & Braman, D. Cul- tural Cognition of Scientific Consensus. J. Risk “Green Goo: Nanotechnology Comes Alive.” Res. 14, 147-174 (2011). ETCgroup.org (Feb. 2003), at http://www.etcgroup.org/content/green-goo- Kahan, D.M., Peters, E., Dawson, E. & Slovic, P. nanotechnology-comes-alive. Motivated Numeracy and Enlightened Self Government. Cultural Cognition Project Working Hallman, W., Cuite, C. & Morin, X. Public Percep- Paper No. 116 (2013). tions of Labeling Genetically Modified Foods. Rutgers School of Environ. Sci. Working Paper Keil, F.C. Folkscience: Coarse interpretations of a 2013-2001, available at complex reality. Trends in cognitive sciences 7, 368- http://humeco.rutgers.edu/documents_PDF/ 373 (2003). news/GMlabelingperceptions.pdf. Keil, F.C. Running on Empty? How Folk Science Kahan, D. Fixing the Communications Failure. Na- Gets By With Less. Current Directions in Psycho- ture 463, 296-297 (2010). logical Science 21, 329-334 (2012). Kahan, D. Why we are poles apart on climate Keil, F.C. The feasibility of folk science. Cognitive change. Nature 488, 255 (2012). science 34, 826-862 (2010). Kahan, D., Braman, D., Cohen, G., Gastil, J. & Kloor, Keith. GMO Opponents Are the Climate Slovic, P. Who Fears the HPV Vaccine, Who Skeptics of the Left. Slate (Sept. 26, 2012), Doesn’t, and Why? An Experimental Study of available at the Mechanisms of Cultural Cognition. Law http://www.slate.com/articles/health_and_sci Human Behav 34, 501-516 (2010). ence/science/2012/09/are_gmo_foods_safe_ oppo- Kahan, D.M. ‘Ordinary science intelligence’: a sci- nents_are_skewing_the_science_to_scare_peo ence-comprehension measure for study of risk ple_.html. and science communication, with notes on evolution and climate change. J. Risk Res, 1-22 Liang, X.X. Value predispositions as perceptual filters: Compar- (2016). ing of public attitudes toward nanotechnology in the United States and Singapore. Public Understanding of Science Kahan, D.M. A Risky Science Communication En- 24, 582-600 (2015). vironment for Vaccines. Science 342, 53-54 Lempert, R.J., Groves, D.G. & Fischbach, J.R. Is it (2013). Ethical to Use a Single Probability Density Sources of Ordinary Science Knowledge—and Ignorance 15 Function? (Santa Monica, CA: RAND Corpo- neered Crops: Experiences and Prospects (National ration, 2013). Academies Press, Washington, D.C., 2016). McFadden, B.R. & Lusk, J.L. What consumers don’t know National Research Council (U.S.). Panel on Atmos- about genetically modified food, and how that affects be- pheric Chemistry. Halocarbons, environmental ef- liefs. The FASEB Journal (2016). fects of chlorofluoromethane release (National Acad- emy of Sciences, Washington, 1976). Lynch, D. & Vogel, D. The regulation of GMOs in National Science Foundation. Science and Engineering Europe and the United States: A case-study of Indicators 2016 (Wash. D.C., 2016). contemporary European regulatory politics. Olson, R. Don't be such a scientist : talking substance in Council on Foreign Relations (2001). an age of style (Island Press, Washington, DC, Margolis, H. Patterns, thinking, and cognition : a theory of 2009). judgment (University of Chicago Press, Chicago, Peters, E.M., Burraston, B. & Mertz, C.K. An Emo- 1987). tion-Based Model of Risk Perception and Margolis, H. Paradigms and Barriers (University of Stigma Susceptibility: Cognitive Appraisals of Chicago Press, Chicago, 1993). Emotion, Affective Reactivity, Worldviews, Margolis, H. Dealing with risk : why the public and the and Risk Perceptions in the Generation of experts disagree on environmental issues (University Technological Stigma. Risk Analysis 24, 1349- of Chicago Press, Chicago, IL, 1996). 1367 (2004). Marx, S.M., Weber, E.U., Orlove, B.S., Leiserowitz, Popper, K.R. Conjectures and refutations; the growth of A., Krantz, D.H., Roncoli, C. & Phillips, J. scientific knowledge (Basic Books, New York,, Communication and mental processes: Expe- 1962a). riential and analytic processing of uncertain Popper, K.R. On the Sources of Knowledge and of climate information. Global Environ Chang 17, Ignorance. in Conjectures and Refutations 3-40 47-58 (2007). (Oxford University Press London, 1962b). Mooney, C. The Republican Brain: The Science of Why Popper, K.R. The open society and its enemies (G. They Deny Science--And Reality (John Wiley & Routledge & sons, London,, 1945). Sons, Hoboken, NJ, 2012). Sato, K. Meanings of genetically modified food and policy National Research Council (U.S.). Board on At- change and persistence: The cases of France, Japan and mospheric Sciences and Climate. & National the United States (Princeton University, 2007). Research Council (U.S.). Committee on Stabi- Science, Media, and the Public Research Group lization Targets for Atmospheric Greenhouse (SCIMEP).. Exploring Public Opinion and Gas Concentrations. Climate stabilization targets : Risk Perceptions of Food in Wisconsin. Uni- emissions, concentrations, and impacts over decades to versity of Wisconsin-Madison. Madison, WI: millennia (National Academies Press, Washing- Department of Life Sciences Communication ton, D.C., 2011). (2016). Available from National Research Council (U.S.). Committee on http://scimep.wisc.edu/projects/reports/ Chemistry and Physics of Ozone Depletion. Causes and effects of stratospheric ozone reduction, an Shapin, S. A social history of truth : civility and science in seventeenth-century England (University of Chica- update : a report (National Academy Press, go Press, Chicago, 1994). Washington, D.C., 1982). Sunstein, C.R. National Research Council (U.S.). Committee on Laws of fear : beyond the precautionary principle (Cambridge University Press, Cam- Ecological impacts of Climate Change. Ecologi- bridge, UK ; New York) (2005). cal impacts of climate change (National Academies Press, Washington, D.C., 2008). Sunstein, C.R. On the Divergent American Reac- tions to Terrorism and Climate Change. National Research Council (U.S.). Committee on Co- Genetically Engineered Crops. Genetically Engi- lumbia Law Rev 107, 503-557 (2007). Sources of Ordinary Science Knowledge—and Ignorance 16 Tims, D.. Measure 92, GMO-labeling initiative, fails narrowly. Oregonian (Nov. 5, 2014), at http://tinyurl.com/o33zs2c. Vance, M.E., Kuiken, T., Vejerano, E.P., McGinnis, S.P., Hochella, M.F., Jr., Rejeski, D. & Hull, M.S. Nanotechnology in the real world: Rede- veloping the nanomaterial consumer products inventory. Beilstein Journal of Nanotechnolo- gy 6, 1769-1780 (2015) Watts, D.J. Everything is Obvious: Once You Know the Answer: How Common Sense Fails (Atlantic Books, 2011).