Air Force Institute of Technology AFIT Scholar Theses and Dissertations Student Graduate Works 3-2020 Introducing the Game Design Matrix: A Step-by-Step Process for Creating Serious Games Aaron J. Pendleton Follow this and additional works at: https://scholar.afit.edu/etd Part of the Educational Assessment, Evaluation, and Research Commons, Game Design Commons, and the Instructional Media Design Commons Recommended Citation Pendleton, Aaron J., "Introducing the Game Design Matrix: A Step-by-Step Process for Creating Serious Games" (2020). Theses and Dissertations. 4347. https://scholar.afit.edu/etd/4347 This Thesis is brought to you for free and open access by the Student Graduate Works at AFIT Scholar. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of AFIT Scholar. For more information, please contact [email protected]. INTRODUCING THE GAME DESIGN MATRIX: A STEP-BY-STEP PROCESS FOR CREATING SERIOUS GAMES THESIS Aaron J. Pendleton, Captain, USAF AFIT-ENG-MS-20-M-054 DEPARTMENT OF THE AIR FORCE AIR UNIVERSITY AIR FORCE INSTITUTE OF TECHNOLOGY Wright-Patterson Air Force Base, Ohio DISTRIBUTION STATEMENT A APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. The views expressed in this document are those of the author and do not reflect the official policy or position of the United States Air Force, the United States Department of Defense or the United States Government. This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States. AFIT-ENG-MS-20-M-054 INTRODUCING THE GAME DESIGN MATRIX: A STEP-BY-STEP PROCESS FOR CREATING LEARNING OBJECTIVE BASED SERIOUS GAMES THESIS Presented to the Faculty Department of Electrical and Computer Engineering Graduate School of Engineering and Management Air Force Institute of Technology Air University Air Education and Training Command in Partial Fulfillment of the Requirements for the Degree of Master of Science in Cyberspace Operations Aaron J. Pendleton, B.S. Captain, USAF March 2020 DISTRIBUTION STATEMENT A APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. AFIT-ENG-MS-20-M-054 INTRODUCING THE GAME DESIGN MATRIX: A STEP-BY-STEP PROCESS FOR CREATING LEARNING OBJECTIVE BASED SERIOUS GAMES THESIS Aaron J. Pendleton, B.S. Captain, USAF Committee Membership: James Okolica, Ph.D. Chair Gilbert Peterson, Ph.D. Member Mark Reith, Ph.D. Member AFIT-ENG-MS-20-M-054 Abstract The Game Design Matrix (GDM) was developed to make effective game design ac- cessible to novice game designers. Serious Games are a powerful tool for educators seeking to boost the level of student engagement and application in academic en- vironments, but they can be difficult to incorporate into existing courses due to availability and the cost of quality game design. Traditional serious game design is centered around an experienced or aspiring game designer with a particular vision for a game. This places educators in a situation where they recognize the value of serious games, but they cannot effectively create one without a significant investment in their development as a designer. GDM uses a game dynamics focused approach to scaffold novice game designers through the selection of game elements and incorporation of learning objectives to guide designers during the most challenging phase of creating a game. Case study games were designed by novices using GDM, and then tested in live classroom environments to assess the level of efficacy achieved through this new methodology. A qualitative assessment of the step-by-step GDM game design framework through these case study games yielded an effective and engaging series of serious games designed to teach elements of cybersecurity and software engineering graduate courses. The results of the games created by GDM to this point strongly indicate that GDM provides a scaffolding for novice designers to create an effective game. Furthermore, GDM games have been play-test ready in about twenty hours of design time. This is a marked improvement over games designed by novices in the literature review, which took longer to create and were consistently ineffective. iv AFIT-ENG-MS-20-M-054 This work is dedicated to my family. Thank you for all you do! v Acknowledgements I would like to express my sincere thanks to my fellow classmates for their support, camaraderie, and talent for making hard work a lot more fun. In a similar manner, this thesis would not have been possible without the help of my advisor, Dr. Okolica. Meeting with him and discussing the potential applications of this research was ex- tremely helpful, but it was his insight into the life of a student and encouragement that it would all be worth it in the end that I found to be particularly inspiring. The faculty of the GCO program consistently surprised me with their student oriented policies and time they took to get to know us. This was deeply appreciated and made my experience at AFIT very positive. Lastly I would like to thank my wife. Her loving support and patience made the irregular hours and late nights of grad school a much happier time. Aaron J. Pendleton vi Table of Contents Page Abstract . iv Dedication . .v Acknowledgements . vi List of Figures . xi List of Tables . xiii I. Introduction . .1 1.1 Background and Motivation . .1 1.2 Problem Statement . .2 1.3 Assumptions and Limitations . .3 1.4 Research Objectives . .3 1.5 Hypothesis . .4 1.6 Approach . .4 1.7 Contributions . .5 1.8 Results . .6 1.9 Organization . .6 II. Background and Related Work . .8 2.1 Overview . .8 2.2 Alternatives to Traditional Lecture-Based Teaching Methods .......................................................9 2.2.1 Flipped Classroom . .9 2.2.2 Project and Game-Based Learning . 10 2.3 Serious Games . 11 2.3.1 Origin of Serious Games . 11 2.3.2 Digital vs Physical Serious Games . 12 2.4 Serious Game Design . 12 2.4.1 MDA Framework . 13 2.4.2 Additional Serious Game Design Frameworks . 14 2.5 Applications of Serious Games to Cyber Education and Training . 16 2.6 Learning Taxonomies and Learning Objective Development. 18 2.6.1 Introduction to Learning Taxonomies: Bloom's Taxonomy . 18 2.6.2 Advancements in Learning Taxonomies . 18 vii Page 2.6.3 Developing Learning Objectives with Bloom's Taxonomy . 19 2.6.4 Implementing Educational Taxonomies in Computer Science and Cyber Education Applications . 22 2.7 Fundamental Elements of Serious Game Design . 22 2.7.1 Game Mechanics . 22 2.7.2 Game Mechanics Controversy . 24 2.7.3 Game Dynamics . 25 2.7.4 Translating Game Mechanics and Dynamics to Learning Objectives . 26 2.8 The Need for Step-By-Step Game Design Process . 26 2.9 Assessment Methods . 27 2.9.1 Learning Assessment Methods for Higher Levels of Bloom's Taxonomy . 27 2.9.2 Serious Game Assessment Methods . 28 2.9.3 Game Design Assessment Methods . 29 2.10 Technical Background for Case Study Game Design: Cyber Risk Analysis and Assessment . 31 2.10.1 Identifying the Need for Risk Management Education . 32 2.10.2 NIST Risk Management Framework . 33 2.10.3 Operational Implementation of RMF in the Air Force ................................................... 33 2.11 Summary . 34 III. Game Design Matrix Development and Assessment . 36 3.1 Overview . 36 3.2 Research Question . 37 3.2.1 Research Question One . 37 3.2.2 Research Question Two . 38 3.2.3 Research Assumptions . 38 3.3 Hypothesis . 38 3.4 Approach . 38 3.5 Contribution . 38 3.6 Study Setting . 39 3.7 The Game Design Matrix . 39 3.7.1 Important Definitions . 41 3.8 Step-By-Step Framework . 43 3.8.1 Step One: Define Educational Objectives . 43 3.8.2 Step Two: Select Game Dynamics . ..