DEVELOPMENT AND VALIDATION OF A PHYSICAL ACTIVITY GAMES PLAYABILITY SCALE BY ENG WAH TEO DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Kinesiology in the Graduate College of the University of Illinois at Urbana-Champaign, 2013 Urbana, Illinois Doctoral Committee: Professor Weimo Zhu, Chair Associate Professor Synthia Sydnor Associate Professor Amy Woods Associate Professor Kelly Bost ABSTRACT For the past three decades, the prevalence of childhood obesity has been on the rise and correspondingly engagement time in sedentary activities has escalated. In contrast, interest, and participation rates in physical education classes are declining. Fun and interesting physical activity (PA) games could help to prevent the decline and possibly reverse inactivity. The purpose of this study was to develop and validate a Physical Activity Playability Scale (PAGPS) in order to provide more detailed “game information” assisting end users (e.g., policy makers, PE teachers, et al.) in choosing the “best possible” children’s PA games. A two-stage development and validation process was employed for this study. Five content experts (N=5) were recruited to draft and develop the PAGPS scale. By applying the heuristic approach, content experts selected, reviewed, commented, evaluated, and eventually determined the relevant PA games factors/subscales, which helped in establishing the content validity evidence for the PAGPS. Ten factors that were identified to represent game domain were Fun, Social, Cognitive, Physical, Skills, Game Structure, Language, Environment, Game Difficulty, and Player’s Characteristics, and a total of 116 items were developed for these factors. Two hundred PE teachers (N=200) were recruited in Malaysia to further determine the most suitable items for the PAGPS. Principal Component Analysis (PCA) was chosen to systematically trim the large amount of variables but maintain as much of the information from the PAGPS (draft) data set. A six-factor construct (with 99 items), including fun and social, cognitive, physical and skills, games structure and environment, game difficulty and player’s characteristics and language were confirmed for the PAGPS. Rasch ii Analysis, an item response theory approach, was then chosen for the item reduction process by taking advantage of the analysis (i.e., invariance, ability to locate all facets on a same scale, and additive) over the classical testing theory based approach. Items were deleted based upon three criterions: goodness-of-fit statistics, item difficulty (logits), and content balance. As a result, four shorter PAGPS versions were created, with 51-item, 36- item, 28-item and 20-item, respectively. The 51-item version was chosen because of its high correlation (r = .98) with the original 99-item version and its balanced content coverage. Cronbach’s alpha analysis was also performed to determine the internal structural consistency. Ten Malaysian PA games were selected to validate the 51-item PAGPS (Game Rating Scale), including One-Leg, Kali-Tui, Blind Man’s Bluff, Simon Says, Eagle and Hen, Hopscotch, Police and Thief, Duck Duck Goose, Monkey in the Middle, and Mr. Wolf. Sixty children (N=60) consisting of two age-groups (Grade 2: n= 30, Grade 5: n= 30) were recruited to play all ten PA games, their reactions towards each game were video-recorded for rating purposes. Ten raters (N=10) scored each PA game video (10 videos for Grade 2 and 10 videos for Grade 5, respectively) using the 51-item PAGPS. The rating scores were analyzed for inter-rater reliability evidence, discriminant evidence (P and K coefficient), and game descriptive statistics (validity evidence). Inter- rater reliability was found to be within a good range of .70-.91, P coefficient from .15- 1.00 and K coefficient from -.70-1.00. Together, reliability evidence (i.e., internal structure reliability and inter-rater reliability) and validity evidence (content validity, discriminant validity and games’ descriptive statistics) provided preliminary support for the psychometric quality of the PAGPS (Game Rating Scale). This study also illustrated that, with a combination of the convent balance, Rasch analysis can be used effectively iii for item reduction while maintaining the psychometric properties of the original measure. More PA researchers should take advantage of this method when developing and constructing measures. iv ACKNOWLEDGEMENTS First of all, I would like to extend special thanks to my advisor, Dr. Weimo Zhu, for his continued guidance and advice throughout my graduate studies at the University of Illinois. His class on Advance Statistics, especially Rasch modeling, has greatly benefited my dissertation process. I was also fortunate to have a group of wonderful committee members: Dr. Synthia, Dr. Amy, and Dr. Kelly for their enriching guidance. Their mentoring has allowed me to gain many valuable insights from different facets of game studies. Many thanks to my current (Marco, Elena, Yan, Evan) and former (Yongsik, Yong, Sara, Bhibha) Kinesmetrics lab members and the visiting scholars, thank you all for your support, and most of all friendship. Special thanks to Heidi our lab coordinator for assisting with IRB protocol and proofreading. Without her, this dissertation could not have been completed in time. To the faculty, staff, and friends in the Department of Kinesiology, especially Dr. Carlton, Dr. Petruzzello, Tina, Linda, Julie, and friends in Champaign/Urbana (i.e. YoonSo, Rob, Joe, John, Duane, CAFÉ , UIUC Table tennis team, UIUC Photography Club, and CU Internationals, thank you all for making my American experience so colorful. I want to record my heart-felt gratitude to the Malaysian government and University Malaya, without their financial support this doctoral journey would not be possible. In addition, I am also grateful to my fellow colleagues and staff in Sport Center, Dr. Selina, Dr. Ashril and Dr. Victor for helping me with recruitment process. Not to v forget Professor Salleh (Director of Sport Center) and Dr. Che Hashim (Head of PhD Extension Unit) for approving my “leave” to complete my final defense. To my loving Malaysian parents (Kooi Siong and Ean Gim) and U.S. godparents (Bob and Mary) for their unfailing care, support, and motivation whenever I am down or doubted myself. And finally thank God Almighty for helping me complete this long challenging journey. Thank you all. “What doesn’t kill you, makes you stronger” (Kelly Clarkson) vi TABLE OF CONTENTS LIST OF TABLES……………………………………………………………………...viii LIST OF FIGURES…………………………………………………………………….....x LIST OF FORMULAS…………………………………………………………………..xii CHAPTER 1: INTRODUCTION…………………………………………………………1 CHAPTER 2: LITERATURE REVIEW………………………………………………….9 CHAPTER 3: METHODS……………………………………………………………….49 CHAPTER 4: RESULTS………………………………………………………………...79 CHAPTER 5: DISCUSSION…………………………………………………………...110 REFERENCES…………………………………………………………………………125 APPENDIX A: PHYSICAL ACTIVITY GAMES PLAYABILITY SCALE (PAGPS [GRS])……………………………………………………………….150 APPENDIX B: DESCRIPTIVE STATISTICS OF TEN PA GAMES………………...154 vii LIST OF TABLES Table 1: Definition of game...……………………………………………………………11 Table 2: Caillois classification of games………………………………………………...13 Table 3: Games classification……………………………………………………………13 Table 4: Stages of play...…………………………………………………………………17 Table 5: Age, skills development, and games………………….………………………...18 Table 6: Definition of play ……..………….…………………………………………….22 Table 7: Definitions of gameplay…...…….……………………………………………..36 Table 8: Type of rating scales……………………………………………………………41 Table 9: Classification of measurement scale……………………………………………42 Table 10: PAGPS category or factors (Draft 1)………………………………………….53 Table 11: Interpretation of mean-square fit statistics……………………………………67 Table 12: Interpretation of Cronbach’s alpha...………………………………………….71 Table 13: Interpretation of Kappa coefficient...…………………………………………76 Table 14: Descriptive statistics of PE teachers and raters……………………………….80 Table 15: KMO and Bartlett’s test………………………………………………………81 Table 16: Communalities table (Round 1)………………………………………………82 Table 17: Total variance explained (Round 1)…………………………………………..84 Table 18: Component matrix (Round 1)………………………………………………...86 Table 19: Communalities table (Round 2)………………………………………………87 Table 20: Rotated component matrix……………………………………………………88 Table 21: Summary of PCA deleted items………………………………………………89 Table 22: Summary of Rasch analysis…………………………………………………..90 viii Table 23: Paired samples correlation…………………………………………………….93 Table 24: Paired sample t-test……………………………………………………………93 Table 25: Summary of Cronbach’s alpha………………………………………………..94 Table 26: Summary of inter-rater reliability……………………………………………..95 Table 27: Summary of P coefficient and Kappa (K) coefficient………………………...98 Table 28: Frequency (F) and descriptive statistics for Game 1 (One-Leg)………...……99 Table 29: Summary of median and inter-quartile range of PA games…………………109 ix LIST OF FIGURES Figure 1: Prevalence of obesity among children and adolescents: United States, trends 1963-1965 through 2007-2008…………………………………………………………....3 Figure 2: Games for understanding classification……………………………………….15 Figure 3: Core content of games classification………………………….……………….15 Figure 4: Development games classification……………………………..……………...16 Figure 5: The interrelationship of play, games, and sports....……………………………32 Figure 6: Continuum between play, games, sports, and work …………………………..32 Figure 7: Relationship between design, computer game, and players…………………...35 Figure