University of Central Florida STARS Electronic Theses and Dissertations, 2004-2019 2013 Explicit Feedback Within Game-based Training: Examining The Influence Of Source Modality Effects On Interaction Benjamin Goldberg University of Central Florida Part of the Psychology Commons Find similar works at: https://stars.library.ucf.edu/etd University of Central Florida Libraries http://library.ucf.edu This Doctoral Dissertation (Open Access) is brought to you for free and open access by STARS. It has been accepted for inclusion in Electronic Theses and Dissertations, 2004-2019 by an authorized administrator of STARS. For more information, please contact [email protected]. STARS Citation Goldberg, Benjamin, "Explicit Feedback Within Game-based Training: Examining The Influence Of Source Modality Effects On Interaction" (2013). Electronic Theses and Dissertations, 2004-2019. 2631. https://stars.library.ucf.edu/etd/2631 EXPLICIT FEEDBACK WITHIN GAME-BASED TRAINING: EXAMINING THE INFLUENCE OF SOURCE MODALITY EFFECTS ON INTERACTION by BENJAMIN S. GOLDBERG B.A. University of Florida, 2005 M.S. University of Central Florida, 2010 A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Modeling and Simulation in the College of Sciences at the University of Central Florida Orlando, Florida Summer Term 2013 Major Professors: Jan Cannon-Bowers & Clint Bowers © 2013 Benjamin S. Goldberg ii ABSTRACT This research aims to enhance Simulation-Based Training (SBT) applications to support training events in the absence of live instruction. The overarching purpose is to explore available tools for integrating intelligent tutoring communications in game-based learning platforms and to examine theory-based techniques for delivering explicit feedback in such environments. The primary tool influencing the design of this research was the Generalized Intelligent Framework for Tutoring (GIFT), a modular domain-independent architecture that provides the tools and methods to author, deliver, and evaluate intelligent tutoring technologies within any training platform. Influenced by research surrounding Social Cognitive Theory and Cognitive Load Theory, the resulting experiment tested varying approaches for utilizing an Embodied Pedagogical Agent (EPA) to function as a tutor during interaction in a game-based environment. Conditions were authored to assess the tradeoffs between embedding an EPA directly in a game, embedding an EPA in GIFT’s browser-based Tutor-User Interface (TUI), or using audio prompts alone with no social grounding. The resulting data supports the application of using an EPA embedded in GIFT’s TUI to provide explicit feedback during a game-based learning event. Analyses revealed conditions with an EPA situated in the TUI to be as effective as embedding the agent directly in the game environment. This inference is based on evidence showing reliable differences across conditions on the metrics of performance and self-reported mental demand and feedback usefulness items. This research provides source modality tradeoffs linked to tactics for relaying training relevant explicit information to a user based on real-time performance in a game. iii ACKNOWLEDGMENTS I underestimated the difficulty in acknowledging all of those worthy of recognition, as there are too many to list. It’s been a wild, fun and extremely stressful ride over the past few years and there are a number of individuals to thank. I apologize if I miss you. First and foremost, I would like to thank my advisor Dr. Jan Cannon-Bowers, my co- chair Dr. Clint Bowers, and the rest of my committee: Dr. Robert Sottilare, Dr. Rudy McDaniel, and Dr. Peter Kincaid. I couldn’t have asked for a better group of professionals to share this experience with. In particular, I would like to thank Jan and Clint for allowing me to tackle this mountain in my own way, and I appreciate the guidance and support throughout. Your expertise on game-based learning and experimental design was invaluable. Jan, you were instrumental right from the start. Your smarts and experience helped me through the difficult phases, and your coaching was just what I needed to get through the initial gauntlet of developing a topic. I’m thankful for the friendship we developed during this process. I would also like to thank a number of individuals from the U.S. Army Research Laboratory (ARL) and the Simulation and Training Technology Center (STTC). For starters, I would like to thank my fellow LITE Labbers for being an awesome group of individuals to work with on a daily basis. Dr. Heather Holden, Janice Connor, and Dr. Anne Sinatra, thanks for all the help and for being a listening ear when I needed one. I owe a special thanks to Dr. Keith Brawner. You were as critical as ever when we traveled to West Point for data collection. I would also like to thank John Hart, Chris Metevier and Chuck Amburn for helping me stay motivated throughout the entire process. Last but not least, I have a large debt of gratitude to Dr. Robert Sottilare. Bob served an integral role in my development over the past three years, and I iv would not have been able to accomplish this feat without his support and guidance. Thanks for being a dedicated leader and for mentoring me on both a professional and personal level. Next, I would like to thank my family and friends for all the love and support during my stint as a graduate student; especially for putting up with me during the rough times. My father Stephen and my mother Elaine were integral in keeping me sane through this process. Thanks for keeping me grounded and for always helping me keep my eye on the prize. A special thanks goes to my brother Jordan and my sister in-law Shelley. You gave me a place to escape the insanity and you were the perfect target allowing me to rant and vent about all the pains and pleasures of working through a dissertation. Lastly, I owe a big thank you to a few notable individuals from the United States Military Academy at West Point. Dr. Mike Matthews, you’re the man! I can’t express how great it’s been to work with you across multiple projects over the past few years. I consider myself lucky to have gotten to know you so well. You’re support and determination was essential to this study becoming a reality. I am indebted to you for your true professionalism and your gracious hospitality. I would also like to thank Colonel Jim Ness, Bess Georgoulas, and Dr. Elizabeth Velilla. You were essential pieces to my data collection, and your support and motivation for the research being conducted made the whole experience that much more enjoyable. v TABLE OF CONTENTS LIST OF FIGURES ....................................................................................................................... ix LIST OF TABLES ......................................................................................................................... xi LIST OF ACRONYMS ............................................................................................................... xiii CHAPTER 1: INTRODUCTION ................................................................................................... 1 Simulation-Based Training ......................................................................................................... 2 Feedback in Simulation-Based Training ................................................................................. 6 Intelligent Tutoring Systems ................................................................................................... 8 Statement of the Problem ............................................................................................................ 9 Purpose of the Study ................................................................................................................. 12 Research Questions ................................................................................................................... 14 CHAPTER 2: BACKGROUND ................................................................................................... 15 Chapter Two Summary ............................................................................................................. 15 Serious Games ........................................................................................................................... 16 Current State of Serious Games ............................................................................................ 19 Design Principles in Serious Games...................................................................................... 21 Components of Intelligent Tutoring Systems ............................................................................ 25 Learner Modeling .................................................................................................................. 28 Domain Modeling .................................................................................................................. 29 Pedagogical Modeling ........................................................................................................... 33 Communication Module ........................................................................................................ 34 Domain Independency and Current Limitations ................................................................... 34 CHAPTER 3: LITERATURE REVIEW ...................................................................................... 38 Chapter 3 Summary ..................................................................................................................