Michigan Technological University Digital Commons @ Michigan Tech Dissertations, Master's Theses and Master's Dissertations, Master's Theses and Master's Reports - Open Reports 2013 Using a Prediction and Option Generation Paradigm to Understand Decision Making Joel Suss Michigan Technological University Follow this and additional works at: https://digitalcommons.mtu.edu/etds Part of the Cognitive Psychology Commons Copyright 2013 Joel Suss Recommended Citation Suss, Joel, "Using a Prediction and Option Generation Paradigm to Understand Decision Making", Dissertation, Michigan Technological University, 2013. https://doi.org/10.37099/mtu.dc.etds/495 Follow this and additional works at: https://digitalcommons.mtu.edu/etds Part of the Cognitive Psychology Commons USING A PREDICTION AND OPTION GENERATION PARADIGM TO UNDERSTAND DECISION MAKING By Joel Suss A DISSERTATION Submitted in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY In Applied Cognitive Science and Human Factors MICHIGAN TECHNOLOGICAL UNIVERSITY 2013 © 2013 Joel Suss This dissertation has been approved in partial fulfillment of the requirements for the Degree of DOCTOR OF PHILOSOPHY in Applied Cognitive Science and Human Factors. Department of Cognitive and Learning Sciences Dissertation Advisor: Dr Paul Ward Committee Member: Dr Edward Cokely Committee Member: Dr Shane Mueller Committee Member: Dr Neil Charness Committee Member: Dr K. Anders Ericsson Department Chair: Dr Bradley H. Baltensperger iii Table of Contents List of Figures………………………………………………………………………... vi List of Tables……………………………………………………………………….... vii Acknowledgements…………………………………………………………………... ix Abstract………………………………………………………………………………. xi 1. INTRODUCTION………………………………………………………………… 1 Outline of literature review (chapters 2 & 3)....………………………..…….. 4 2. LITERATURE REVIEW: SITUATION ASSESSMENT IN DECISION MAKING………………………………………………………………………….. 8 Situation assessment in dynamic domains…………………………………… 8 Situation assessment in recognition-primed decision making.……….. 9 Situation assessment in the model of situation awareness.………….... 13 Situation assessment in other complex and dynamic domains.……….. 17 Hypothesis generation in ill-defined problems.…………………………….... 23 Perceptual anticipation……………………………………………………….. 28 Perceptual anticipation: Related constructs...…………………………. 29 Perceptual anticipation in complex domains………………………...... 31 Perceptual anticipation in sport………………………………..... 32 Projection in dynamic domains…………………………………. 36 Hazard perception in driving...…………………………………. 39 Theoretically-motivated studies of skill in perceptual anticipation....... 42 Summary……………………………………………………………………... 47 3. LITERATURE REVIEW: RESPONSE PHASE OF DECISION MAKING…….. 48 Response generation and selection.………………………………………….. 48 Response generation/selection tasks to examine perceptual-cognitive expertise..…...………………………..................................................... 49 Response option generation in ill-defined problems………………...... 53 Empirical studies of response option generation in complex and dynamic domains...…………………..................................................... 56 Option generation/selection in the situation assessment and response phases on decision making..…………................................................... 62 Summary……………………………………………………………………... 65 4. GENERAL HYPOTHESES AND OUTLINE OF STUDIES 1 AND 2…....…….. 66 General hypotheses……………….………………………………………….. 66 Outline of Studies 1 and 2…….......………………………………………….. 67 5. STUDY 1………………………………………………………………………...... 69 Study 1A…...………………………………………………………………… 71 Method………………………………………………………………… 72 Participants……………………………………………………… 72 Materials and stimuli……………………………………………. 72 iv Option-generation tasks………………………………………… 74 Procedure……………………………………………………..… 76 Data preparation………………………………………………… 78 Study 1B...………………………………………………………………….... 78 Cognitive task analysis………………………………………………... 78 Method………………………………………………………….. 78 Participants………………………………………………... 78 Materials and stimuli……………………………………… 79 Procedure…………………………………………………. 79 Data analysis……………………………………………………. 80 Performance data……………………………………………………… 82 Method………………………………………………………….. 82 Participants………………………………………………... 82 Materials and stimuli……………………………………… 83 Procedure…………………………………………………. 83 Data preparation…………………………………………... 85 Study 1C...…………………………………………………………………… 85 Method………………………………………………………………… 86 Participants……………………………………………………… 86 Materials and stimuli……………………………………………. 86 Procedure……………………………………………………….. 88 Data preparation………………………………………………… 88 Data analysis and results for Studies 1A–C…..……………………………. 89 Data screening………………………………………………………… 89 Anticipation scores……………………………………………………. 89 Analyses employed……………………………………………………. 89 Identification of discriminating scenarios…………………………….. 92 Effect size calculation………………………………………………… 95 Summary of Studies 1A–C….....………………………………………….…. 95 6. STUDY 2…………………………………………………………………….......... 98 Hypotheses…………………………………………………………………… 99 Method……………………………………………………………………..… 103 Power analysis………………………………………………………… 103 Participants……………………………………………………………. 103 Materials and stimuli………………………………………………….. 104 Option-generation tasks……………………………………………….. 105 Assessment option-generation task……………………………... 105 Response option-generation task……………………………….. 107 Post-option-generation cognitive task analysis……………………..… 108 Retrospective verbal reports……………………………………. 109 Stimulated recall of in-event option generation………………… 109 Procedure……………………………………………………………… 111 Data analysis…………………………………………………………... 112 Signal detection analysis………………………………………... 112 Dependent variables…………………………………………….. 113 v Tests of the experimental hypotheses…………………………... 118 Supplementary analyses………………………………………… 120 Cognitive task analysis data…………………………………….. 121 Retrospective verbal report data………………………….. 122 Stimulated recall data……………………………………... 122 Results………………………………………………………………………... 125 Signal detection analysis……………………………………………… 125 Tests of the experimental hypotheses…………………………………. 126 Cognitive task analysis data…………………………………………... 134 Retrospective verbal report data………………………………... 137 Stimulated recall data…………………………………………… 138 Quantitative analysis……………………………………… 138 Qualitative analysis……………………………………….. 139 Summary……………………………………………………………………... 147 7. DISCUSSION…………………………………………………………………….. 151 Outcome performance…...…………………………………………………… 153 Option-generation measures.………………………………………………… 155 Supplementary points………………………………………………………… 162 Signal detection performance…………………………………………. 162 Taking the first-generated option……………………………………... 163 Entropy/information uncertainty……………………………………… 163 Limitations and future research directions…………………………………… 165 Summary...…………………………………………………………………… 168 8. REFERENCES…………………………………………………………………..... 170 9. APPENDICES…………………………………………………………………….. 189 A. Scenarios used in Studies 1 and 2………....………………………………….. 190 B. Exemplar scenarios……………………………………………………………. 191 C. Timeline of Blow Up scenario with description of critical cues and timing of occlusion points……………………………………………………………….. 193 D. List of occlusion point times (in seconds) for Studies 1A–C..…...…………… 194 E. Radio message/contextual introductions to the video scenarios……………… 195 F. G*Power power analysis output………………………………………………. 197 G. Thought statement types………………………………………………………. 198 vi List of Figures Figure 1. The mean number of task-relevant and task-irrelevant assessment/ response options generated, by experience group, for the six test trials….. 129 Figure 2. Description of the phases of the Domestic Punch scenario...…………….. 136 vii List of Tables Table 1. Occlusion points selected by experienced police trainers in Study 1B (Cognitive task analysis)………...…………………………………………. 80 Table 2. Optimal occlusion point for discriminating scenarios, with results of classification criteria……………………………………………………….. 94 Table 3. Effect size (Cohen’s h) measures for the discriminating scenarios and the overall sample-weighted effect size…………………………....................... 95 Table 4. Performance categories as functions of experience level, ability to predict the culminating event, and ability to take the best response option…...…… 124 Table 5. Proportion of times that participants drew/did not draw their gun by trial type and experience group………………………………………................. 125 Table 6. Measures of sensitivity and bias by experience group………………........... 126 Table 7. Summary of multiple regression analysis for variables predicting the number of correct predictions made during assessment (n = 49)…….......... 131 Table 8. Summary of multiple regression analysis for variables predicting the number of times participants took the best response option (n = 49)……… 132 Table 9. Number of participants in each contrast group for the six test trials…......... 135 Table 10. Central tendency measures of statement types elicited via retrospective verbal report (Domestic Punch scenario)…………………………………... 137 Table 11. Median number of statements, by scenario phase (with duration, in seconds), and experience level (Exp. = Experienced, Less-exp. = Less- experienced) for the Domestic Punch Scenario………………..................... 138 Table 12. Categorization of participants who completed the stimulated recall task for the Domestic Punch scenario………………………………......................... 139 Table 13. Stimulated recall statements for Phase 1 (Confrontation) of the Domestic Punch scenario by statement type, success in responding, and experience group……………………………………………………………………….. 141 Table 14. Stimulated