The Ecological Validity of Traditional Pen-and-paper, Ecologically Oriented, and Virtual Reality Neuropsychological Test Measures to Cognitive Impairment and Real-World Function after Mild Traumatic Brain Injury by Eliyas Jeffay A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Department of Psychology University of Toronto © Copyright by Eliyas Jeffay, 2019 The Ecological Validity of Traditional Pen-and-paper, Ecologically Oriented, and Virtual Reality Neuropsychological Test Measures to Cognitive Impairment and Real-World Function after Mild Traumatic Brain Injury Eliyas Jeffay Doctor of Philosophy Department of Psychology University of Toronto 2019 Abstract A growing concern in neuropsychology is whether neuropsychological test measures (NTMs) can predict functional outcome (i.e., ecological validity). Ecological validity can be understood in two ways: veridicality (i.e., prediction of functional outcome is independent of how the test reflects the outcome being measured) and verisimilitude (i.e., tests resemble the functional outcome they are predicting). Historically, the veridicality approach has been utilized (i.e., by way of traditional, pen-and-paper NTMs), but there has been a movement to employ verisimilitude approaches (e.g., Behavioural Assessment for Dysexecutive Syndrome; BADS). These approaches were examined in patients experiencing ongoing cognitive complaints in the post-acute period of recovery (> 3 months) following a mild traumatic brain injury (mTBI). This was the principal inclusion criteria for the studies described here. A meta-analytic study was conducted and found that traditional NTMs were not sensitive to persistent cognitive complaints in this population across all domains. Studies 2 and 3 utilized archival data to determine whether NTMs could predict return to work (RTW) status. The BADS predicted employment status by way of medium-to-large effects, while traditional NTMs did not. Overall, these findings suggest ii the verisimilitude approach is more ecologically valid than the veridicality approach. As such, two virtual reality tests (VRTs) evaluating attention and executive function were developed to investigate their ability to predict RTW. Tests of attention (VR and traditional tests) significantly predicted group membership at 82% accuracy, with 82.6% sensitivity and 81.5% specificity. The attention shift trial of the VRT and to a lesser degree the total speed score of the Ruff 2 & 7 were predictive of employment status. Overall, this research provides empirical evidence for the verisimilitude approach when evaluating RTW status in patients who are in the post-acute period of recovery following mTBI. Moreover, it provides initial evidence for the clinical utility of VRTs to evaluate real-world functioning. iii Acknowledgments A PhD is not completed alone. Thus, I have many people to thank for their support and assistance throughout the process. First, I am forever indebted to my primary supervisor, mentor, “academic father”, and friend, Dr. Konstantine K. Zakzanis. From teaching Abnormal Psychology in undergrad to editing this dissertation, you have fostered my professional development in a way that strikes balance between giving independence and providing assistance. Since day one you openly referred to and treated me as your colleague. You are, inspiringly, the embodiment of the phrase ‘hard work truly pays off’, which has been the mantra that has motivated me through difficult times. Thank you, truly, for everything you have done. To my committee members, Dr. Anthony Ruocco and Dr. Mary Pat McAndrews, thank you for your time, guidance, thoughtful comments, and patience throughout my graduate years. On more than one occasion, I had reached out to each of you for advice on a wide range of topics. Every single time, you had provided a level of insight that was most helpful. I have been honoured to have been supervised by such a prestigious team and I do hope to be able to pay your teachings forward. To my lab family, Manu, Anosha, Matt, Steven, Paul, & Elisabeth, thank you for assisting in the pilot projects that were pivotal for the direction of this graduate work. Raymond, your time with us was tragically cut-short, but you will forever be remembered and missed. To my lab sisters, Diana, Kyrsten, Courtney, Sonya, Angie, Krysta, and Emnet thank you for always cheering me up and knowing when I needed help even though I would be too proud to ask for it. Zach, you are the reason I continued with undergrad and the reason for my place in the lab. I will never forget our early philosophical chats and your generosity in sharing your knowledge and experience with me. To my family, thank you for your honesty, love, and guidance. For many reasons, nothing in my life, including obtaining a PhD from one of the top universities in the world, would be possible without my parents, Abbas and Leila Jeffay. Immigrating to a country, learning a new language, adapting to a new culture, and raising three children has been the set example of the what can be iv achieved with perseverance and hard work. It is my life goal to pay you back and make you proud. To my two brothers, thank you for always knowing what to say to make me laugh when I was feeling low and for doing my share of the housework when I had to focus on my studies. To my best friend, Dr. Jeffrey Davis Knight, thank you for leading by example in showing that I can do better. Though it seems that we usually take two steps backwards and one step forward in our adventures, we’ve learned a lot together and have gained memories that I truly cherish. Plain and simple, I look up to you. I hope that our friendship continues until one of us has dementia. Not it. Lastly, to my dearest fiancé, Monica. How fortuitous that I was able to find the love of my life whilst initiating the work of my life. Thank you for being an endless source of motivation, support, and most of all patience throughout what seemed like a life-time. Despite studying psychology, I learned true patience and compassion by your lead. I love you and am excited to learn more from you as we embark on the next chapter of our lives together. I cannot possibly thank everyone. If I have left anyone out it was not intentional. Please consider this as a formal thank you to those that had helped me along this journey. You know who you are. v Table of Contents Acknowledgments.......................................................................................................................... iv Table of Contents ........................................................................................................................... vi List of Tables ...................................................................................................................................x List of Figures ................................................................................................................................ xi List of Abbreviations .................................................................................................................... xii Chapter 1 ..........................................................................................................................................1 General Introduction ...................................................................................................................1 1.1 Definitions............................................................................................................................2 1.2 Inconsistencies with Definitions ..........................................................................................3 1.3 Prevalence & Incidence .......................................................................................................4 1.4 Pathophysiology ...................................................................................................................7 1.5 Cognitive Sequelae & Typical Outcome .............................................................................8 1.6 TBI vs mTBI and Return to Work .....................................................................................13 1.7 Additional Issues with Neuropsychological Assessment ..................................................15 1.8 Ecological Validity ............................................................................................................18 1.9 Ecologically Oriented Tests ...............................................................................................19 1.10 Naturalistic Assessments ...................................................................................................20 1.11 Virtual Reality ....................................................................................................................20 1.12 Overview of Studies & Hypotheses ...................................................................................22 Chapter 2 ........................................................................................................................................24 Study 1: Neuropsychological Outcome Following the Post-Acute Phase of Mild Traumatic Brain Injury: A Meta-Analysis .................................................................................................24 2.1 Narrative vs Quantitative Reviews ....................................................................................24 2.2 Review of Previous mTBI Meta-Analyses ........................................................................25