Survivorship Bias: The Exception is Not the Rule

Survivorship Bias: Lesson Plan

Topic

Survivorship bias is the tendency to focus on the people or things that made it past some selection process, as if they are an accurate representation of the whole population, while accidentally ignoring the people who did not make it past the selection process. A common example of the survivorship bias in action is when children look at the success of their favorite celebrities and think that all it takes is talent and perseverance to do well in Hollywood. However, children are forgetting to take into account the numerous wannabe actors and singers who had plenty of talent and determination but failed to get noticed by entertainment companies.

Possible subjects/classes Time needed

● Psychology ● Economics 30-45 minutes ● History ● Statistics

Video link:

https://academy4sc.org/topic/survivorship-bias-the-exception-is-not-the-rule/

Objective: What will students know/be able to do at the end of class?

Students will be able to: ● Define survivorship bias ● Summarize the problem that Abraham Wald saw with the engineering of the World War II aircrafts ● Identify the potential consequences of falling prey to the survivorship bias

Key Concepts & Vocabulary

{what keywords do students need to know to fully understand the video/lesson?}

Materials Needed

{anything that the teacher needs to prepare for the lesson, other than the video}

Before you watch Survivorship Bias: The Exception is Not the Rule

Pose this real-life problem to students: During World War II, one of the chief concerns of the United States was finding a way to maximize the survival chances of American bomber airplane pilots. Military engineers wanted to pinpoint the parts of the plane that most needed extra armor.

To help solve the problem, the military examined planes that safely returned from combat and provided data of where the planes were most frequently damaged. The wings, the body, and the area around the tail gunner had the most bullet holes. Show students the picture below to demonstrate the data:

Ask the students: based on this information, where do you think they should put the extra armor on the planes? (Note: Students will likely say the wings, the body, and around the tail gunner.)

Point out to the students that the only planes that were available for study were the ones that survived battle - they shouldn’t forget that they’re lacking crucial damage data from the planes that were shot down. Mention that the wings, the body, and around the tail gunner are actually the spots that need armor the least because all the planes available for study had survived in spite of the bullet holes in these areas. Tell the students that if they originally answered that the wings, the body, and around the tail gunner were the places that needed more armor, they’ve fallen into something called the survivorship bias.

While you watch

1. Define survivorship bias 2. Why is it important to be aware of survivorship bias? 3. Name one everyday example of survivorship bias mentioned in the video Survivorship Bias: The Exception is Not the Rule

After you watch/discussion questions

1. How might the survivorship bias be potentially dangerous? 2. What’s an example of survivorship bias that you or a close friend have experienced? 3. Let’s say a scientist experienced survivorship bias while collecting data. What would the consequences of that be?

Activity Ideas

1. Students design their own experiment/question to ask other students in school and see how many fall prey to survivorship bias. 2. Students choose a dream career and create an explanation for their parents as to why it is not an unrealistic dream, avoiding making arguments based on survivorship bias. They can practice their argument in pairs or groups in the class. 3. Split students up into groups and assign each group one of the scenarios from the attached worksheet. Have students contemplate their given scenario and present to the class why survivorship bias was an issue and how it could be avoided.

Sources/places to learn more

4. Liang, Bing. “Hedge Funds: The Living and the Dead.” Journal of Financial and Quantitative Analysis, vol. 35, no. 3, Sept. 2000, pp. 309–26. Cambridge Core, doi:10.2307/2676206 . 5. Mind, Language of. “Survivorship Bias or Why ‘Follow Your Heart’ Is Sometimes Terrible Advice.” Medium , 17 Oct. 2018, https://medium.com/the-polymath-project/survivorship-bias-or-why-follow-your-h eart-is-sometimes-terrible-advice-a73ac570bc3. 6. Shermer, Michael. “How the Survivor Bias Distorts Reality.” Scientific American, doi:10.1038/scientificamerican0914-94 . Accessed 8 Aug. 2019.