The Role of Analytics in Evidence Based Decision Making

THE ROLE OF ANALYTICS IN EVIDENCE- BASED DECISION MAKING

STEVE BENNETT, PH.D.

Expected Utility Theory: Decision alternative A: 80% chance of having a 1,000 AED outcome Decision alternative B: 40% chance of having a 2,000 AED outcome

Expected Value [A]: 80% X 1,000 = 800 AED Expected Value [B]: 40% X 2,000 = 800 AED

Choose A or B: A: 80% to win 10,000 AED B: 7,000 AED for certain

Choose A or B: A: 80% to lose 10,000 AED (20% chance of losing nothing) B: lose 7,000 AED for certain

Copyright © 2015, SAS Institute Inc. All rights reserved. BIASES IN JUDGMENT Availability Bias Overconfidence Anchoring Bias Confirmation Bias Conservatism Bias Information Bias Outcome Bias Survivorship Bias

Copyright © 2015, SAS Institute Inc. All rights reserved. Copyright © 2015, SAS Institute Inc. All rights reserved. Analytics is the scientific process of transforming data into insights for decision making.

Copyright © 2015, SAS Institute Inc. All rights reserved. Copyright © 2015, SAS Institute Inc. All rights reserved. Copyright © 2015, SAS Institute Inc. All rights reserved. Copyright © 2015, SAS Institute Inc. All rights reserved. “The U.S. government has access to a vast amount of information…. the storehouse is immense…But the U.S. government has a weak system for processing and using what it has.”

“The system was blinking red during the summer of 2001 – but no one connected the case in his or her inbox to the threat reports …no one looked at the bigger picture, no analysis foresaw the lightning that could connect the thundercloud to the ground.”

Open-source news/media 911 EMS

Infected Event mitigation due to action facilitated by early warning

16 Time Poison Control Social Media ED Hospital/Lab

Open-source news/media 911 EMS

Infected Event mitigation due to action facilitated by early warning

17 Time 18 19 20 Street-level view of EMS activity in Charleston County, SC

SMARTT data (hospital bed availability by department)

Box chart of Charleston County EMS run counts per day of the week Pins represent (over 365 days) EMS runs

Charleston County EMS TAP trends by syndrome over 21 time Charleston County GI TAP alert on 6/30/2013 22 Social Media Analytics

23 Social Media Analytics

• ~85% correct • No apparent decrease over time • Not much difference over labelers )

t Random Forest Class (Yp) Y -1 0 1 -1 0 191 97 0 0 17086 1554 1 0 1696 3376 Human Class ( Class Human

Pr{Yp= 1|Yt = 0} = 0.083 Type 1 Error (false positive) Person Tweets Labelled Classifier Performance 6000 83.78% Pr{Yp= 0|Yt = 1} = 0.334 Type 2 Error (false negative) AR DM 6000 86.53% Pr{Yt=1} = 0.214 Proportion Sick EH 6000 84.92% Proportion Not_Sick Pr{Yt=0} = 0.786 PS 6000 85.80%

Prob Error = Pr{Yt=1}Pr{Yp=0|Yt=1} + Pr{Yt=0} Pr{Yp=1|Yt=0} = 0.136 Be an integrator – gather and align the data and information that you have

Be curious – seek new data sources both directly and indirectly related to your decision space

Be open – accept that conventional wisdom might be challenged when evidence and insights begin to emerge Be patient – Analytics as a component of evidence- based decision making requires organizational and