K.-S. Focsaneanu Dept. of Chemistry and Biomolecular Science 1. to learn more about the types of bias, and how they may influence scientific research 2. to become more self-aware of our own biases 3. take steps to avoiding bias in the future

ADDRESSING BIAS IN SCIENCE 2 1. Science is concerned with understanding how nature and the physical world work. 2. Science can prove anything, solve any problem, or answer any question. 3. Any study done carefully and based on observation is scientific. 4. Science can be done poorly. 5. Anything done scientifically can be relied upon to be accurate and reliable. 6. Different scientists may get different solutions to the same problem. 7. Knowledge of what science is, what it can and cannot do, and how it works, is important for all people.

ADDRESSING BIAS IN SCIENCE 3 • OBJECTIVITY is the key to good science.

• To be objective, experiments need to be designed and conducted in a way that does not introduce bias into the study.

• BIAS: A prejudiced presentation of material, or consistent error in estimating a value

ADDRESSING BIAS IN SCIENCE 4 • Claim: A statement put forth as true in an argument or on an issue. • Reason: A general statement that offers support for a claim. • Evidence: Facts, statistics, and examples used to support reasons.

EVIDENCE REASON CLAIM

ADDRESSING BIAS IN SCIENCE 5 All communication can be subject to bias, from either the author or the audience. • cognitive bias: results primarily from people’s natural, cognitive tendencies or deficiencies • motivational bias: the individual is deliberately trying to introduce bias in order to gain some advantage

While the central goal of scientific communication is for scientists to communicate facts and theories to other scientists and experts, these communications are also shared with non-scientists and thus we need to be aware of how other people will interpret the content and message.

ADDRESSING BIAS IN SCIENCE 6 • Affinity or in-group bias: people extend not only greater trust, but also greater positive regard, cooperation, and empathy to in-group members compared with out-group members. This preference for people like ourselves is largely instinctive and unconscious.

• Affinity bias manifests not only as a preference for in-group members — but it may also manifest as an aversive tendency towards out-group members.

• What are the implications of this effect?

• How can an individual’s experiences disrupt affinity bias?

ADDRESSING BIAS IN SCIENCE 7 • researcher/experimenter • reporting/publication • sampling/selection • omission • procedural • measurement • interviewer/response • philosophical

ADDRESSING BIAS IN SCIENCE 8 Pseudoscientific claims: who is accountable?

OR

ADDRESSING BIAS IN SCIENCE 9 • Confirmation Bias: searching for evidence that accords with or confirms their current hypothesis • weighing evidence according to how neatly a causal explanation fits rather than how much data supports it

SEEK TO TEST CLAIMS, NOT JUST CONFIRM THEM!

ADDRESSING BIAS IN SCIENCE 10 Figure 3: Graph Suggests a Negative Response Figure 4: Graph Suggests No Response 1.155 2

1.15 1.5 1.145

1.14 1 Absorbance Absorbance 1.135 0.5 1.13

1.125 0 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 Concentration (mmol/L) Concentration (mmol/L)

ADDRESSING BIAS IN SCIENCE 11 1. Selection Bias – a statistical sampling bias from selecting groups that are not representative of the population of interest 2. Self-Selection Bias – a statistical sampling bias from participants opting to respond who are not as diverse as the underlying population 3. Non-Response Bias – a statistical sampling bias from differences in characteristics between those responding and those not responding 4. Survivorship Bias – a statistical sampling bias from looking only at a remaining group rather than the full population

ADDRESSING BIAS IN SCIENCE 12 • SAMPLE SIZE: Is the sample big enough to get a good average value? • SELECTION OF SAMPLE: Does the composition of the sample reflect the composition of the population? – location, age, gender, ethnicity, nationality and living environment – lab: sample heterogeneity, viability, appearance… • How to minimize sample selection bias: – Use a RANDOM SAMPLE, so every individual/unit has an equal likelihood of being chosen – Limit the question asked to the specific group sampled

ADDRESSING BIAS IN SCIENCE 13 1. Is the method of data collection chosen in such a way that data collected will best match reality? 2. Are the measurements taken accurately? 3. Are there additions to the environment that may influence results? 4. Has the experiment been designed to isolate the effect of multiple factors?

ADDRESSING BIAS IN SCIENCE 14 ADDRESSING BIAS IN SCIENCE 15 • deliberate malfeasance • conflict of interest • competition with other scientists for limited resources (jobs, grants, etc) • employment/promotion contingent on conducting successful studies • pressure to publish early in order to meet performance indicators and beat rival scientists to the discovery • reputational motivators - the addition of caveats can undermine authority

ADDRESSING BIAS IN SCIENCE 16 O

ONa Hg S

ADDRESSING BIAS IN SCIENCE 17 The Dunning-Kruger Effect Chem. Educ. Res. Pract., 2014, 15, 24

• The fool doth think he is wise, but the wiseman knows himself to be a fool. (As You Like It, Act V:Sc. 1, line 30–31)

• real-world consequence: underestimation and planning fallacy

ADDRESSING BIAS IN SCIENCE 18 In Australia in 2016, 14 428 people died from the three causes shown below. Please indicate how many of these deaths resulted from each cause.

CAUSE OF DEATH NUMBER

Shark attack

Road accidents

Dementia

ADDRESSING BIAS IN SCIENCE 19 • our tendency to revise our memories when we come into possession of new information • consequence: believing that we actually knew what we now know all along—or that the outcome of a now observed event was easily predictable in advance. • how to avoid this: better book-keeping! – record, in advance, predictions, hypotheses and confidence levels – this provides an unbiased baseline against which to compare conclusions once the outcome is known

Including the experimental design and hypotheses ensures that those evaluating a paper know that the conclusions derived are not simply the result of hindsight bias.

ADDRESSING BIAS IN SCIENCE 20 • What are the traits required for someone to become a scientist? • Some stereotypes: – likely to be smarter than average – higher in conscientiousness and need for cognition – detail-oriented and determined – logical reasoning skills

There is no single trait that protects against all biases. Because biases result from a wide variety of different processes, a trait that protects against one type of bias can open a person up to other biases. In short, while we may all be different, we are all likely to be biased in some way.

ADDRESSING BIAS IN SCIENCE 21 • Anchoring bias: relying too heavily on the first piece of information you come across • Blind-spot bias: recognizing bias in others, but failing to recognize it in yourself • Negativity bias: focusing on negative events at the expense of positive or neutral events • Outcome bias: making a decision based on the outcome of a previous event without any regard to other factors involved • Social-desirability/Participant bias: response/behaviour is influenced by preconceived notions about what is believed to be an acceptable response, either consciously or non-consciously

ADDRESSING BIAS IN SCIENCE 22 OH OH OH H HO O OH OH

OH H O C OH OH O HO OH epigallocatechingallate (EGCG) Keith U. Ingoldresvera tvs.rol A.T. Diplock

CH3 OH HO

CH3 H3C O CH3 H CH3 H CH3 CH CH3 3 CH3 BHT α-tocopherol (Vitamin E) In In-H O2 ArOH RH R ROO ROOH + ArO kiinh RH or O2

no reactiion ROOH RH

ADDRESSING BIAS IN SCIENCE 23 OH OH OH H HO O OH OH

OH H O C OH OH O HO OH epigallocatechingallate (EGCG) resveratrol

CH3 OH HO

CH3 H3C O CH3 H CH3 H CH3 CH CH3 3 CH3 BHT α-tocopherol (Vitamin E)

FEBS Letters, 1973, v. 29

ADDRESSING BIAS IN SCIENCE 24 OH OH OH H HO O OH OH

OH H O C OH OH O HO OH epigallocatechingallate (EGCG) resveratrol

CH3 OH HO

CH3 H3C O CH3 H CH3 H CH3 CH CH3 3 CH3 BHT α-tocopherol (Vitamin E) Biology of Vitamin E, 1983, p.45

ADDRESSING BIAS IN SCIENCE 25 OH OH OH H HO O OH OH

OH H O C OH OH O HO OH FRB&M, 1990, v.9, p.205 epigallocatechingallate (EGCG) resveratrol

CH3 OH HO

CH3 H3C O CH3 H CH3 H CH3 CH CH3 3 CH3 BHT α-tocopherol (Vitamin E)

ADDRESSING BIAS IN SCIENCE 26 • The scientific community engages in certain quality control measures to eliminate bias: – Results are verified by independent duplication and publication in a peer-reviewed journal • Independent duplication: Two or more scientists from different institutions investigate the same question separately and get similar results. • Peer-reviewed journal: A journal that publishes articles only after they have been checked for quality by several expert, objective scientists from different institutions.

ADDRESSING BIAS IN SCIENCE 27 For research involving human participants: • ensure and inform the participant about their anonymity • properly motivate the participant • regularly checking for outliers in the data • use of objective, external observations/data collection, e.g. biosensors, eye tracking

ADDRESSING BIAS IN SCIENCE 28 • “Scientifically-proven” – Science does not seek to prove but to disprove • Emotional appeals – Conclusions should be data-based • Strong language – Scientific conclusions should only report what the data supports – Words should be chosen very carefully to avoid exaggeration or claims not supported by data

THE DATA SHOULD BE CONVINCING, NOT THE WORDS USED!

ADDRESSING BIAS IN SCIENCE 29 1. Be aware of your own prejudices – take the Harvard Implicit Bias Test 2. Value quantitative, reproducible data collection, when possible 3. Stay informed, but break out of your ‘bubble’

ADDRESSING BIAS IN SCIENCE 30 1. I want to be up-to-date on news about a place, group of people, issue or aspect of culture that I care about or am personally connected to. 2. I don’t want to seem like I don’t know what is going on in the world because it’s embarrassing. 3. I want to find “news I can use” on a specific topic or issue. 4. I want to be entertained. 5. I feel like following news will help me academically/professionally. 6. I don’t follow a lot of news, actually…

ADDRESSING BIAS IN SCIENCE 31 The New York Times • word choice 3/11/2003 • omissions Iraq forces suspension of U.S. surveillance flights Iraqi fighter jets threatened two American U-2 • limiting debate surveillance planes, forcing them to abort their mission and return to base, senior U.S. officials • story framing said Tuesday. • USA Today biased selection/use of sources 3/11/2003

U.N. Withdraws U-2 Planes

U.N. arms inspectors said Tuesday they had withdrawn two U-2 reconnaissance planes over Iraq for safety reasons after Baghdad complained both aircraft were in the air simultaneously.

ADDRESSING BIAS IN SCIENCE 32 PEW RESEARCH CENTER

ADDRESSING BIAS IN SCIENCE 33 ALLSIDES.COM

ADDRESSING BIAS IN SCIENCE 34 1. Track your news consumption over a 48h period 2. Define your media needs. Does your ‘diet’ supply it? Modify your news diet, as necessary. 3. Evaluate your media sources and reflect on your experience, in a ~500 word essay.

ADDRESSING BIAS IN SCIENCE 35