University of Nebraska at Omaha DigitalCommons@UNO Information Systems and Quantitative Analysis Department of Information Systems and Faculty Publications Quantitative Analysis 2017 Mitigation of Cognitive Bias with a Serious Game: Two Experiments Testing Feedback Timing and Source Norah E. Dunbar Matthew L. Jensen Elena Bessarabova Yu-Hao Lee Scott N. Wilson See next page for additional authors Follow this and additional works at: https://digitalcommons.unomaha.edu/isqafacpub Part of the Computer Sciences Commons Authors Norah E. Dunbar, Matthew L. Jensen, Elena Bessarabova, Yu-Hao Lee, Scott N. Wilson, Javier Elizondo, Bradley J. Adame, Joseph Valacich, Sara Straub, Judee K. Burgoon, Brianna Lane, Cameron W. Piercy, David Wilson, Shawn King, Cindy Vincent, and Ryan Schuetzler International Journal of Game-Based Learning Volume 7 • Issue 4 • October-December 2017 Mitigation of Cognitive Bias with a Serious Game: Two Experiments Testing Feedback Timing and Source Norah E. Dunbar, University of California Santa Barbara, Department of Communication, Santa Barbara, California Matthew L. Jensen, University of Oklahoma, Price College of Business and Center for Applied Social Research, Norman, Oklahoma Claude H. Miller, University of Oklahoma, Department of Communication, Norman, Oklahoma Elena Bessarabova, University of Oklahoma, Department of Communication, Norman, Oklahoma Yu-Hao Lee, University of Florida, Department of Telecommunication, Gainesville, Florida Scott N. Wilson, University of Oklahoma, K20 Center, Norman, Oklahoma Javier Elizondo, University of Oklahoma, K20 Center, Norman, Oklahoma Bradley J. Adame, Arizona State University, Hugh Downs School of Communication, Tempe, Arizona Joseph Valacich, University of Arizona, Eller College of Management, Tucson, Arizona Sara Straub, Independent Researcher, Norman, Oklahoma Judee K. Burgoon, University of Arizona, Eller College of Management, Tucson, Arizona Brianna Lane, Christopher Newport University, Department of Communication, Newport News, Virginia Cameron W. Piercy, University of Central Missouri, Department of Management, Warrensburg, Missouri David Wilson, University of Nebraska-Omaha, Department of Information Systems and Quantitative Analysis, Omaha, Nebraska Shawn King, Independent Researcher, Norman, Oklahoma Cindy Vincent, Salem State University, Department of Communications, Salem, Massachusetts Ryan M. Schuetzler, University of Arizona, Eller College of Management, Tucson, Arizona ABSTRACT Oneofthebenefitsofusingdigitalgamesforeducationisthatgamescanprovidefeedbackfor learnerstoassesstheirsituationandcorrecttheirmistakes.Weconductedtwostudiestoexaminethe effectivenessofdifferentfeedbackdesign(timing,duration,repeats,andfeedbacksource)inaserious gamedesignedtoteachlearnersaboutcognitivebiases.Wealsocomparedthedigitalgame-based learningconditiontoaprofessionaltrainingvideo.Overall,thedigitalgamewassignificantlymore .effectivethanthevideocondition.Longerdurationsandrepeatsimprovetheeffectsonbias-mitigation ,Surprisingly,therewasnosignificantdifferencebetweenjust-in-timefeedbackanddelayedfeedback .andcomputer-generatedfeedbackwasmoreeffectivethanfeedbackfromotherplayers KEywoRdS Cognitive Bias, Confirmation Bias, Feedback, Fundamental Attribution Error, Training Game DOI: 10.4018/IJGBL.2017100105  .Copyright©2017,IGIGlobal.CopyingordistributinginprintorelectronicformswithoutwrittenpermissionofIGIGlobalisprohibited  86 International Journal of Game-Based Learning Volume 7 • Issue 4 • October-December 2017 INTRODUCTIoN Proponentsofdigitalgame-basedlearningmaintainthatgamesandsimulationscanfacilitatelearning ,becausethey(a)catertothedigitalgenerationoflearners(Prensky,2005),(b)allowforimmersive activelearningincreasingengagementandretention,and(c)encouragenewformsofknowledge interactionunavailableinatraditionalcurricula(e.g.,perspective-taking,slowingdownorspeeding uptimeprocesses,accessinghazardousordistantenvironments(Jackson,2008).Importantly,digital gamesallowforimmediatefeedbackthatcanhelplearnerscorrecttheirmistakesandrewardlearners .formakingcorrectdecisions Theprovisionoffeedbackgenerallyimproveslearning,howeverthereareimportantcaveats ’regardinghowandwhenfeedbackisgiven.Digitalgamescanprovidefeedbackbasedonlearners paceanddecisionmaking(Azevedo&Bernard,1995).Recentstudieshaveexaminedthecostsand ,benefitsofofferingfeedbackduringinstruction(Hays,Kornell,&Bjork,2010),thetiming(Butler Karpicke,&Roediger,2007)andthesourceoffeedback(e.g.,ateacher,parent,peer,oracomputer agentinthegame(Goldberg&Cannon-Bowers,2015;Hattie&Timperley,2007).Weaddtothis bodyofresearchbypresentingtwostudiesexploringtheeffectsoffeedbacktiming(immediate vs.delayed)andfeedbacksource(computeragentsvs.humanpartners)inagame-basedlearning environmentdesignedtoteachlearnersaboutthepitfallsofcognitivebiases.Totesttheseeffects,we createdaseriousgamecalledMACBETH(MitigatingAnalystCognitiveBias byEliminatingTask Heuristics)1,whereinplayersaretaskedwithdetectingandpreventingaseriesofterroristthreats bygatheringandassessingintelligencedata(forMACBETHdevelopmentseeauthorcitation).The gamefocusesonknowledgeandmitigationofconfirmation bias(CB)andfundamental attribution error(FAE).Thetrainingeffectivenessofthegamewascomparedtoatraditionalinstructionalvideo .explainingFAEandCB,whichofcourseexcludedfeedback Using Feedback in a Serious Game to Mitigate Cognitive Biases Biasedinformationprocessingisoftencausedbytheover-relianceonheuristics—definedasmental shortcuts,orsimpledecisionrules—arisingfromconventionalbeliefs.Byprovidingswiftsolutions andminimizingcognitiveeffort,heuristicscanbenefitdecision-making;however,theymayoften alsoleadtoinsufficientconsiderationofrelevant,diagnosticinformation,resultinginincreaseduse &ofcognitiveshortcutsassociatedwithpoordecisionsandbiasedinformationprocessing(Tversky Kahneman,1974).Confirmationbiasharmssystematicinformation-processingbydirectingattention towardevidencethatconfirmsexistingattitudesandbeliefs(Lundgren&Prislin,1998)attheexpense ofweighingandexaminingpertinentavailableevidencethatmightotherwisedisconfirmerroneous assumptions.Similarly,FAEfostersatendencytofocusoninternal,dispositionalexplanationsof (others’behaviorsattheexpenseofexternal,situationalfactors(Harvey,Town,&Yarkin,1981 .likewisehinderingthedecision-makingprocess Cognitivebiasesaredifficulttochange:Theyaredeeplyembeddedwithinnaturalcognitive processes,andpeoplerarelyrecognizetheirbiaseddecision-making.Tomitigatebias,peoplemust ,firstbecomeawareoftheiruseofheuristics(Bornstein&Emler,2001)forwhichfeedbackcanhelp therebyleadingtobetter-informeddecisions.Feedbackingame-basedlearningcanbeeffectivewhen itprovidesplayersobjectivelearninggoalswithclearcriteriaforsuccess,alongwithmethodsfor .(improvementtoattaingoals(Erhel&Jamet,2013 Notallfeedbackbenefitslearning:Repeatednegativefeedback,forinstance,canleadtolowered .(expectations,reducedeffort,andamorenegativeself-image(Krenn,Würth,&Hergovich,2013 Formativeandcorrectiveoutcomefeedbackthroughsuggestionsandguidancecanhelpmodify 87 International Journal of Game-Based Learning Volume 7 • Issue 4 • October-December 2017 thinkingandbehaviorandimprovelearning(Shute,2008).Yet,performancedecrementsarelikely tooccuriftoomuchfeedbackinformationispresented,causingoverload.Thus,bothtimingand .quantityoffeedbackiscriticaltolearningandoptimalperformance ,Timinghasalsobeenexaminedtodiscerntheadvantagesofimmediateversusdelayedfeedback andameta-analysishasconcludeddelayedfeedbackisgenerallysuperiorinlaboratorystudies,since studentsareoftenrequiredtoexplicitlyconsiderandrespondtoit,whereasimmediatefeedbacktends tobemoreeffectiveinappliedstudies,suchasclassroomsettings(Kulik&Kulik,1988;vander Kleij,Eggen,Timmers,&Veldkamp,2012).Theamountofa“delay”varieswidelyinthestudies withfeedbackbeingprovidedfollowinganassessment,attheendofaday,oruptoaweekaftertask completion(vanderKleijetal.,2012).Althoughofferingfeedbackduringgameplaycanenhanceits .salience,allowingplayerstoadjusttheirdecisions,itcanalsobeadistraction,harmingenjoyment In-gamefeedbackcanslowgameplay,inhibitinggoalattainment,particularlywhenspeedofplay isabasisforadvancement(Ryan&Pintrich,1997).Ontheotherhand,despiteitspotentialfor ,slowingplay,detailedfeedbackearlyintheprocesscanleadtofasterlearning(Billings,2010;Tsai Tsai,&Lin,2015).Becauseplayerscanusein-task“just-in-time”(JIT)feedbacktoimprovetheir performanceandcorrectmistakes,webelieveitcanbemoreeffectivethanfeedbackdelayeduntil :aftertaskcompletion.Thus,wehypothesize .H1:JITfeedbackismoreeffectiveatmitigatingCBandFAEthandelayedfeedback &Knowledgeisentwinedwithpractice,andlearningviavideogamesisnoexception(Lave Wenger,1991).Discoveringhowtoplayanewgametakestime.Noviceuserscanbeoverwhelmed