The Implicit Artificial Grammar Task

The Implicit Artificial Grammar Task:

Preliminary Evaluation of its Potential for Detection of Noncredible Effort/Malingering

A dissertation presented to

the faculty of

the College of Arts and Sciences of Ohio University

In partial fulfillment

of the requirements for the degree

Doctor of Philosophy

Caitlin S. Reese

August 2014

This dissertation titled

The Implicit Artificial Grammar Task:

Preliminary Evaluation of its Potential for Detection of Noncredible Effort/Malingering

CAITLIN S. REESE

has been approved for

the Department of Psychology

and the College of Arts and Sciences by

Julie A. Suhr

Professor of Psychology

Robert Frank

Dean, College of Arts and Sciences

Abstract

REESE, CAITLIN S., Ph.D., August 2014, Psychology

The Implicit Artificial Grammar Task: Preliminary Evaluation of its Potential for

Detection of Noncredible Effort/Malingering

Director of Dissertation: Julie A. Suhr

The detection of noncredible effort/malingering is an essential component of neuropsychological assessment. Malingerers most often feign impairment on memory measures, and memory concerns represent the most often cited complaints in forensic neuropsychological contexts. Understandably, many noncredible effort/malingering measures composed of explicit memory processes have been developed, as have been coaching techniques intended to circumvent effort detection. Given these realities, the continued development of novel noncredible effort/malingering measures remains of paramount importance. The present study examined whether an implicit learning and forced-choice memory task, the artificial grammar task (AGT), could serve as a novel noncredible effort/malingering measure, because existing literature has shown memory- impaired patients do as well as controls on the AGT. It was hypothesized that individuals simulating head injury would perform worse on the AGT than head-injury controls and memory-impaired controls. Results showed that, as expected given prior studies of the

AGT, head-injury controls and memory-impaired controls did not perform differently.

Furthermore, as predicted, simulating participants performed worse than head-injury controls. Simulating participants did not perform worse than the memory-impaired controls. Importantly, exploratory analyses suggested that the AGT’s implicit learning 4 phase trial cutoff score showed initial promise, suggesting the score may serve to practically distinguish malingerers from those performing with best effort in populations with a history of mild head injury presenting in clinical and forensic settings.

Dedication

Dedicated to my family.

Acknowledgments

I would like to extend my sincere gratitude to my mentor, Dr. Julie A. Suhr, to my dissertation committee, and to the members of the Neuropsychology Laboratory at Ohio

University. All of you made this project possible.

Table of Contents

Page

Abstract………………………………………………………………………………….3 Dedication……………………………………………………………………………….5 Acknowledgements………………………………………….….………….……………6 List of Tables…………………………………………….…….………….…………….8 Introduction…………………………………………………………………..….………9 Method…………………….……………………………………………….…………..14 Participants…………………………………………………………….…..….…...14 Procedure..……………………………………………….….…………….……….16 Measures……………………….…………………………………………………..20 Results…………………………………….…………………………………………...26 Discussion……………………………….….………….……………….………….….33 References……………………………….…………………………………………….39 Appendix A: Forms Used in Experiment……….……………………………………..59 Instructions…………………………………………………………….…..……....59 Consent Form…………………………………………………………….………..61 Demographics/Medical History Questionnaire……………………………………65 Brain Injury Supplement…………………………………………………………..67 Manipulation Check……………………………………………………………….69 AGT Post-Task Form………………………………………………………… ..… 70 Post Study Form…………………………………………………………….……. 71 Appendix B: Score Ranges……………….…………………………………………...73 AGT Classification Accuracy Scores……………………………………………..73 AGT Learning Phase Trial Scores………………………………….….………….74 Appendix C: Additional Psychometrics…………………………………….….……..75 Additional Psychometrics…………………………………………….……….…..75

List of Tables

Page

Table 1: Demographic Information and Head Injury Characteristics across Groups….. 50 Table 2: The Standard Set of Acquisition and Test Strings……………….…………… 51 Table 3: Means and SD for Non-study Neuropsychological Test Performance……….. 52 Table 4: Percentages of Validated Effort Measure Failures between Groups…...... 53 Table 5: Descriptive Details Pertaining to Dependent Variables……………...………. 54 Table 6: Simulated Malingerer Sensitivity/Specificity based on Accuracy Scores……. 55 Table 7: Simulated Malingerer Sensitivity/Specificity based on Implicit Learning….... 56 Table 8: Noncredible Effort Sensitivity/Specificity based on AGT Learning....……..... 57 Table 9: Correlations between AGT and Other Administered Measures....….…...... 58

Introduction

As evidenced by its description in the recent publication of the Diagnostics and

Statistics Manual’s fifth edition (DSM-5; APA, 2013), and as a topic of many neuropsychology position papers (e.g., Bush, Ruff, Troster, Barth, Koffler, Pliskin, et al.,

2005, pp. 425-426; Heilbronner, Sweet, Morgan, Larrabee, Millis, and Conference

Participants, 2009), detection of noncredible effort/malingering is an essential feature of psychological and neuropsychological assessment (AACN, 2007; Sharland & Gfeller,

2007; Heilbronner et al., 2009; Slick, Tan, Strauss, & Hultsch, 2004; Slick, Sherman, &

Iverson, 1999). The term noncredible effort refers to invalid effort without implying conscious awareness or deliberate intent, while malingering can be understood as noncredible effort presented in the form of deliberately and consciously exaggerated or fabricated physical, psychological, or cognitive dysfunction with interest in an external incentive (e.g., obtaining financial compensation or drugs) or avoidance of civil or criminal responsibility (e.g., military duty, work, criminal prosecution; APA, 2013; Slick,

Sherman, & Iverson, 1999). However, both terms characterize invalid performance on psychological and neuropsychological tests, and because probable malingering prevalence base rates can exceed 50% (Larrabee, 2007), there are important clinical implications for noncredible effort/malingering detection.

To detect noncredible effort/malingering, it is important to recognize memory is both the most common cognitive complaint among people of all ages (e.g., Gino,

Mendes, Maroco, Ribeiro, Schmand, De Mendonca, et al., 2010) and is the most commonly malingered cognitive symptom (e.g., Mittenberg, Patton, Canyock & Condit, 10

2002; Williams, 1998), especially among individuals reporting a history of a mild head injury, which represents the highest base rate of malingerers (between 38.5% and

41.24%; Mittenberg et al., 2002). There are at least 14 peer-reviewed/accepted memory- based measures of noncredible effort/malingering (Sweet, Condit, & Nelson, 2008), and explicit memory test formats, including forced-choice memory recognition tests, are the most frequently used for detecting noncredible effort/malingering of memory impairment

(Nitch & Glassmire, 2007). Such measures have been developed based on research utilizing both known groups designs (Berry & Schipper, 2007; Dearth, Berry, Vickery,

Vagnini, Baser, Orey, et al., 2005) and simulated malingerer designs, in which the simulators feign cognitive impairment realistically enough to evade the examiner’s detection (e.g., Rogers, 2008; Iverson, Franzen, & McCracken, 1994; Haines & Norris,

2001). These research methods have their own strengths and weaknesses, with known groups designs having stronger external validity and simulated malingerer designs having stronger internal validity.

However, because well-validated, stand-alone, noncredible effort/malingering measures requiring explicit memory mechanisms may be familiar, they may be more susceptible to coaching. Accessible information about noncredible effort/malingering measures for which test security has not been enforced allows motivated clients and willing attorneys (Victor & Abeles, 2004) to manipulate evaluation outcomes (e.g., Suhr

& Gunstad, 2007). Given this interest in exploiting the integrity of existing noncredible effort/malingering measures, development of a noncredible effort/malingering measure 11 utilizing an implicit memory format may present a novel, perhaps less coachable, approach for understanding patterns of feigned memory impairment.

Implicit learning and memory contrasts explicit learning and memory in that the former does not require conscious or intentional cognitive processes, while the latter does

(Schacter & Curran, 2000). Specifically, the artificial grammar task (AGT), first introduced in Reber’s (1967) seminal research, is a task in which one initially becomes sensitive to the implicit rules of an artificial, finite-state grammar simply from exposure to complex exemplary strings during the implicit learning phase. Implicit memory for the implicitly learned rules is evidenced by the ability to later classify letter strings correctly despite the absence of explicit knowledge about the basis for the classification (Reber,

1967; Reber, 1989).

Studies suggest that implicit memory processes are neuroanatomically separate from explicit memory processes, activating non-hippocampal brain areas (Squire, 2009;

Thomas, Hunt, Vizueta, Sommer, Durston, Yang, & Worden, 2004). Indeed, fMRI neuroimaging data has shown that, following exposure to the artificial grammar learning task, when healthy volunteers make implicit grammatical judgments (using the forced- choice, yes or no format), the left frontal and lateral occipital cortices are activated, while activation of the precuneus, which interacts with the left prefrontal cortex in the recollection of episodic memories (Lundstrom, Andersson, Johansson, Fransson, &

Ingvar, 2003) is suppressed (Seger, Prabhakaran, Poldrack, & Gabrieli, 2000).

Furthermore, results of studies with amnesic patients with medial temporal lobe/hippocampus dysfunction reflect the differentiation between implicit and explicit 12 memory. Amnesic patients have severe explicit memory deficits (Fama, Pitel, & Sullivan,

2012; Kessels & Kopelman, 2012), but exhibit normal performance on implicit priming tasks (see Hayes, Fortier, Levine, Milberg, & McGlinchey, 2012 for a review), implicit category learning tasks (see Speekenbrink, Channon & Shanks, 2008 for a review), implicit sequence learning tasks (see Vinter, Pacton, Witt & Perruchet, 2010 for a review), and artificial grammar learning tasks (see Pothos, 2007 for a review). Specific to the AGT, across a number of studies, amnesic patients have evidenced normal performance, indicating they are able to learn abstract representations of the grammar rules and use implicit memory to later classify new letter strings correctly without explicit memory or any basis for the classification accuracy. People with diencephalic amnesia caused by Korsakoff’s syndrome, thalamic infarction, or penetrating brain injury and people with amnesia due to hippocampal damage (caused by anoxia, ischemia, or unknown events and confirmed with magnetic resonance imaging) learn to make forced- choice, artificial grammar grammaticality judgments as well as healthy controls and better than chance, despite their explicit memory impairment for studied exemplars

(Knowlton, Ramus, & Squire, 1992). Other patient populations who have been shown to complete the AGT as well as healthy controls include people with Parkinson’s disease

(Reber & Squire, 1999), Huntington’s disease (Knowlton, Squire, Paulsen, et al., 1996), schizophrenia (Danion, Meulemans, Kauffmann-Muller, & Vermaat, 2001), autism spectrum conditions (Brown, Aczel, Jimenez, Kaufman & Grant, 2010), probable

Alzheimer’s disease diagnoses (Reber, Martinez, & Weintraub, 2003), high test anxiety 13

(Rathus, Reber, Manza, & Kushner, 1994), and adult developmental dyslexia (Russeler,

Gerth, & Munte, 2007).

Given these findings, developing noncredible effort/malingering measures requiring implicit memory processes may result in a novel approach to noncredible effort/malingering testing that is more robust against the effects of coaching, while suitable to distinguish credible from noncredible effort/ malingering. Unlike explicit memory processes, implicit memory processes are intact in healthy individuals and individuals with memory impairment. Thus, if aspiring malingerers with intentions to feign impairment on an implicit memory task do not perform with their best effort, it is possible they will perform worse than patients with memory impairment and head-injury controls.

To our knowledge, no one has investigated the use of any implicit memory task to determine its potential effectiveness in noncredible effort detection. In the present study, we utilized a simulated malingerer design to compare AGT performance in individuals with a history of mild traumatic brain injury (asked either to feign cognitive impairment or do their best); we also included a comparison group of individuals with documented memory impairment (instructed to do their best). We hypothesized simulated malingerers would perform worse on the AGT than both head-injury controls and memory-impaired controls and that these differences would allow us to identify preliminary cutoff scores detecting noncredible effort/malingering sensitivity and specificity. Given the results of prior research, we did not expect that participants with memory impairment would perform differently from head-injury controls. 14

Method

Participants

Head-injury control and simulated malingerer participants (N= 100) were male and female psychology students recruited from psychology courses at a medium-sized

Midwestern university. Based on responses to a pre-experiment questionnaire, individuals who endorsed a history of mild head injury/concussion with a loss of consciousness that lasted between a few seconds and 30 minutes were eligible to participate in the study. Participants signed up for the study via an electronic enrollment system, and the university students were randomly assigned to simulated malingerer

(N=50) or head-injury control (N=50) groups (the random assignment list was created using an online randomizer, respective folders were prepared before participation, and group assignment was confirmed via manipulation check). Participants were initially offered course credit for their participation; however, after slow initial recruitment, 83 of the 100 participants were offered an additional $10 recruitment incentive funded by a small university grant. Incentivized participants’ age (M = 19.02, SE = .10) did not differ from participants who did not receive an incentive (M = 18.82, SE = .18), t(98)=-8.89, p=.38, nor did incentivized participants’ level of education (M = 13.43, SE = .08) differ from those without an incentive (M = 13.24, SE = .14), t(98)=-1.03, p=.31. Gender was evenly distributed between participants with and without an incentive, 2(1)=.50, p =.48, as was randomized group assignment, 2(1)=3.47, p =.06. Based on the 5 head-injury controls and 10 simulated malingerers who did not receive the incentive, and the 43 head- injury controls and 24 simulated malingerers who did receive the incentive, the absence 15 or presence of the incentive did not differentially impact performance on the AGT (ps ranging from .14 to .18).

Thirteen participants’ data were excluded because they indicated they did not follow their instructions during the manipulation check; all of these participants were assigned to be simulated malingerers. Five more participants’ (2 head-injury controls and

3 simulated malingerers) data were excluded because these participants endorsed head injuries with a loss of consciousness lasting more than 30 minutes (ranging from 60 minutes to 14 days). Basic demographics and head injury data for the final 82 participants

(48 head-injury controls and 34 simulated malingerers) included in the analyses are reported in Table 1. Head-injury controls did not differ from simulated malingerers in age, t(80)=.437, p=.66; education, t(80)=.395, p=.69; race/ethnicity, 2(1)=.176, p =.68; or gender, 2(1)=1.75, p =.19. With regard to head injury variables, head-injury controls did not differ from simulated malingerers in length of loss of consciousness of their most recent injury, t(80)=-1.048, p=.30, amount of time since last head injury, t(51.65)=-1.307, p=.21, or total number of head injuries endorsed, t(69)=-.700, p=.49. The choice of whether or not to seek treatment following the mild head injury was also evenly distributed between head-injury controls and simulated malingerers,  2(1)=.818, p =.37.

All male and female memory-impaired patient participants (N= 8) were invited to participate in a neuropsychology and aging study conducted within the same medium- sized Midwestern university. The participants had a history of participating in studies within the laboratory, and they were offered an updated cognitive screening evaluation as part of their participation. Participants were included in the present analyses if their 16 cognitive screening evaluation results revealed significant memory impairment. One participant endorsed no medical conditions. Five participants endorsed only one medical condition (i.e., asthma, glaucoma hypertension, hypercholesterolemia, and kidney stones). One participant endorsed hypertension, hypercholesterolemia, Type II diabetes, and irritable bowel syndrome, and another participant reported having colon cancer (in remission), a lung condition (yet to be diagnosed), and trigeminal neuralgia. Four of these participants also had cataracts. The mean age of the memory-impaired sample was 73.5

(SD=8.6), ranging from 66 to 88 years old. The memory-impaired participants were older than the undergraduate head-injury controls and simulated malingers, F(2,87)=1650.48, p<.001, and they had more education, F(2,87)=72.98, p<.001. Gender and ethnicity were evenly distributed among all three groups 2(gender 2( )=1.88, p =.39; ethnicity

2(6)=1.57, p =.95).

Procedure

Head-injury controls and simulated malingerers.

Once participants enrolled and arrived at the laboratory, they completed informed consent. Then the investigator handed them a pre-made envelope with instructions and said, “This folder contains an envelope with your instructions. I’m purposefully not supposed to know what your instructions say, and I don’t, because these folders were arranged before the study began. I’m going to step out and I’d like you to take five minutes to look over the contents of the envelope. You will find an instructions sheet on which you will be asked to circle a difficulty rating. Once you have reviewed the contents 17 and circled your rating, please return everything to the envelope and set it on the floor.

Then, please wait patiently. I have to give everyone five minutes before I return.”

Participants randomly assigned to simulate symptoms of a mild head injury were given the following additional instructions in their envelope: “Today you will take a series of neuropsychological tests that assess motor speed, attention, memory, and thinking skills. You are being asked to believably pretend that you have significant problems (e.g., representative of brain damage) with motor speed, attention, memory, and thinking skills tests. In other words, pretend that you are someone involved in a lawsuit, and you want to pretend to have brain damage in order to win a financial settlement.

What might you do to indicate (even though you do not) that you have permanent and significant problems with motor speed, attention, memory, and thinking skills while taking neuropsychological tests? There is no wrong answer. Supplemental materials have also been provided for you to read and give you additional ideas regarding how to pretend to have significant problems with motor speed, attention, memory, and thinking skills. The examiner who gives you the tests does not know that you will be pretending to have significant problems representative of brain damage, and of course, you don’t want to get caught pretending, therefore it is important to remind you to believably pretend to have such problems however you see fit. Please take a few moments to review the supplemental materials that you’ve received.” The folders issued to participants simulating symptoms of traumatic brain injury also included a supplemental handout developed from a website describing symptoms of brain injury

(http://www.braininjury.com/symptoms.html; see Appendix A). 18

Participants randomly assigned to perform with their best effort were given the following instructions in their envelope: “Today you will take a series of neuropsychological tests that assess motor speed, attention, memory, and thinking skills.

You are asked to give your best effort on all of these tests.”

All participants also reviewed the other contents of their folders while separated from the examiner (see Appendix A to view all forms). They were asked to rate the difficulty of their given instructions and completed a brief questionnaire collecting basic demographic information and medical history, including history of mild head injury.

Details of past head injuries were of particular interest and included: duration of lost consciousness, approximate amount of time since the head injury, number of head injuries sustained in the past, head injury cause (e.g., vehicular accident, sports related, fall, blunt force trauma), and date of most recent head injury.

Following the examiner’s return, each participant completed a neuropsychological battery that lasted approximately 2 hours. In order to enhance the ecological validity of the study, each undergraduate participant was administered a battery of measures including the AGT (Reber, 1967), the Word Memory Test (Green, 2005), the Wechsler

Adult Intelligence Scale – Fourth Edition (WAIS-IV; Wechsler, 2008) Digit Span subtest, the Auditory Verbal Learning Test (AVLT; Rey, 1964), and the Minneapolis

Multiphasic Personality Inventory-Second Edition-Restructured Form (Tellegan & Ben-

Porath, 2008). Following their participation, participants independently completed a manipulation check, after which the examiner asked them questions about their 19 performance on the AGT. Next they were debriefed, and the majority of participants were provided $10 compensation for their participation.

Memory-impaired patients.

The memory-impaired controls were recruited to participate in a separate neuropsychology and aging experiment within the same university laboratory.

Participants in the larger study were over the age of 60 years old, fluent English speakers, and active drivers carrying any active driver’s license. These participants completed a short semi-structured interview and driving history questionnaire before completing a neuropsychological battery, which included the Repeatable Battery for the Assessment of

Neuropsychological Status (R BANS; Randolph, 2012), the Trail Making Test (TMT;

Reitan & Wolfson, 1985), Delis-Kaplan Executive Function System Color-Word

Interference Test (Delis, Kaplan, & Kramer, 2001), WAIS-IV Block Design (Wechsler,

2008), Wechsler Test of Adult Reading (WTAR; Wechsler, 2001), Useful Field of View

(UFOV; Visual Awareness Research Group, 2009), and Hazard Perception Test (Cheng,

Ng, & Lee, 2011). The AGT (Reber, 1967) was addended to that study’s IRB following its inception. Following the IRB approval, all older participants were administered the

AGT task along with the aforementioned battery. We then had access to all older participants’ deidentified demographic information, explicit (declarative) memory data, and AGT data, for use in the present analyses. Older participants who had a score in the impaired range on either recall of the RBANS’ list, story, and/or figure details, or had low recognition scores, qualified for the present study (i.e., were categorized as memory- impaired). 20

Measures

All the participants were administered the AGT (Reber, 1967), which was the present study’s primary measure of interest. The AGT measures one’s ability to develop an implicit understanding of a system of letters, which Reber called a finite artificial grammar system (see depiction below), and the abstract rules of the system. In this experiment, we used Reber and Allen’s (1978) artificial grammar system and corresponding grammatical and nongrammatical letter strings (see Table 2).

Figure. This is the finite-state grammar/Markovian system first used by Reber and Allen

(1978). It depicts how grammatical strings are generated by following the paths starting at #0 and continuing until one of the three exiting paths is taken, with each path generating the letter that labels it. This grammar has been used commonly in the literature, as cited in Redington and Chater (1996).

The letter strings were determined by starting at #0 and traversing along the arrows, adding a letter to the string with each transition, until exiting along one of the arrows leading out of the system. That is, within a finite-state grammar system, the letter strings are derived from left-to-right in a nonhierarchical fashion; they are mathematically tractable, and they have an intrinsic probabilistic structure. The system is said to isolate implicit learning because, without exposure to the system’s depiction, the code of the system cannot be broken by conscious effort (Reber, 1989). A total of 20 grammatical letter strings were generated for the implicit learning phase, while 25 grammatical letter strings (three to six letters in length) and 25 nongrammatical letter strings (violating the rule system at one position within the letter string) were generated for the classification phase. In table 2, the letter strings are listed, and the point at which a grammatical rule was broken is emboldened.

We used a computerized version of Reber’s task. The program included the implicit learning phase, a five minute delay, and the classification phase. During the implicit learning phase, participants were told they would see strings of letters and asked to reproduce the letter strings. They then viewed a grammatical letter string on the computer screen for 3 seconds and were asked to type that letter string correctly after it disappeared. If they were incorrect, they had two additional trials to see and type the correct letter string. After three attempts, the computer program presented the next letter string, and the participant was once again tasked with typing that letter string correctly.

During the implicit learning phase, the participants were presented the 20 grammatical 22 letter strings, with the same procedures, twice (thereby having the opportunity to correctly type 40 grammatical letter strings).

After a five-minute delay, the classification phase began by informing participants, for the first time, that the letter strings they saw previously were formed according to a complex set of rules, and they were then asked to categorically determine

(using a forced-choice, yes-no format) whether or not new letter strings followed the complex set of rules. They were instructed that the rules were very complex, that they may not be able to figure them out, and that they may want to go with their "gut feeling" to determine whether or not new items followed the rules. Participants were then shown a previously randomized list of 25 new grammatical letter strings and 25 nongrammatical letter strings. The strings were presented one at a time, and participants classified each string as correct or incorrect (yes or no), depending on whether or not they appeared to conform to the rules. Without interruption, the same 50 strings were presented, and participants classified them, a second time. Participants’ response reaction time was recorded.

The AGT’s construct validity, that is, the measure’s ability to isolate implicit learning and memory, has been evidenced by studies with participants with memory impairment, where explicit learning/memory is impaired while performance on the implicit AGT is intact, as reviewed above. Consistent with the existing literature (e.g.,

Knowlton, Ramus, & Squire, 1992; Knowlton, Squire, Paulsen, Swerdlow, Swenson, &

Butters, 1996; Reber, Martinez, & Weintraub, 2003; deVries, Barth, & Floel, 2009), our 23 primary dependent variable of interest was the percentage of grammatical and nongrammatical letter strings accurately identified during the classification phase.

Secondarily, some researchers (e.g., Knowlton, Squire, Paulsen, Swerdlow, &

Butters, 1996) have described (yet have not statistically compared) potential group differences based on the total number of letter string trials attempted to correctly reproduce the letter strings presented during the implicit learning phase. We were also interested in this dependent variable, as it is also a component of the AGT, while far less researched than the classification accuracy score.

In addition, when comparing amnesic patients’ and head-injury controls’ performance on another implicit memory test, the serial reaction time test, implicit memory has been evaluated by key press reaction time. That is, participants press a key corresponding to a cue in one of four locations, and their reaction time is faster when presented with a familiar sequence as compared to a random sequence of locations

(Reber & Squire, 1998). To our knowledge, reaction time has not been researched using the AGT, and we were interested in conducting exploratory analyses to evaluate any reaction time differences. Therefore, reaction time data, in the form of a speeded mouse click following the presentation of grammatical (versus nongrammatical) and correct

(versus incorrect) letter strings was also collected.

Other germane measures administered to head-injury controls and simulated malingerers.

Undergraduate head-injury controls and simulated malingerers were also administered a brief questionnaire collecting basic demographic information and medical 24 history, including history of mild head injury (see Appendix A for the form). Noncredible effort/malingering measures tapping explicit cognitive processes and personality factors were also included in the battery administered to head-injury controls and simulated malingerers. The tests included the well-validated, freestanding, forced-choice Word

Memory Test, with a cutoff of ≤ 82.5 (WMT; Green, 2005), in addition to the embedded

Reliable Digit Span index cut-scores of ≤6 or ≤7 (Jasinski, Berry, Shandera, & Clark,

2011) and enhanced/revised Reliable Digit Span index’s cut-score of ≤ 11 (Reese, Suhr,

& Riddle, 2012; Young, Sawyer, Roper, & Baughman, 2012). A measure of personality and emotional functioning, the MMPI-2-RF (Ben-Porath & Tellegen, 2008) was also administered because the Response Bias Scale (RBS) is a sensitive, incrementally valid

(compared to the MMPI-2 validity scales) predictor of over-reported memory complaints

(Gervais, Ben-Porath, Wygant, & Sellbom, 2010). These measures have been found to have excellent criterion validity for discriminating between noncredible and credible effort, and more detailed psychometrics for these instruments are available in Appendix

C. In the present study, exploratory analyses were conducted with these measures to compare their ability to discriminate simulators from head-injury controls relative to the

AGT variables (Descriptive details regarding performance on the neuropsychological measures are included in Table 3).

Other germane measures administered to memory-impaired participants.

Memory-impaired participants were administered a semi-structured interview to collect basic identifying information and medical background. In addition, the Repeatable

Battery for the Assessment of Neuropsychological Status with updated norms (RBANS; 25

Randolph, 2012) was administered because this measure has been deemed to have good sensitivity and specificity for the detection of dementia, even in the initial stages, as well as other neurological conditions affecting memory (D uff, Hunphreys Clark, O’Bryant,

Mold, Schiffer, & Sutker, 2008; Garcia, Leahy, Corradi, & Forchetti, 2007; Hobson,

Hall, Humphreys-Clark, Schrismsher, & O’Bryant, 2010; Pachet, 2007). Therefore, specific measures of interest included subtests representing the delayed memory index, including list recall (delayed free recall for unrelated verbal information), story recall

(delayed free recall for conceptually related verbal information), figure recall (delayed free recall for conceptually related visuospatial and detail information), and list recognition (delayed recognition memory for unrelated verbal information). Impaired performances (standard scores falling at least 1.5 SD below the mean) on one or more of these measures were indicative of memory impairment. Detailed psychometrics for these measures are included in Appendix C.

Results

Checks for the normality of AGT classification accuracy scores revealed that head-injury controls’ classification accuracy generally fit a normal distribution, z=0, p >

.05, without kurtosis, z = -.68, p > .05. Likewise, AGT classification accuracy scores revealed that memory-impaired participants’ classification accuracy generally fit a normal distribution, z=1.49, p > .05, without kurtosis, z = .87, p > .05. Simulated malingerers’ classification accuracy data also fit a normal distribution, z=-1.37, p > .05, without kurtosis, z = .26, p > .05. There were no statistical outliers within any group.

Consistent with group assignment and suggesting that the manipulation was successful, simulated malingerers performed significantly worse on all non-AGT measures relative to head-injury controls; see Table 4.

AGT Classification Accuracy

The AGT classification accuracy score is the common dependent variable across studies utilizing the AGT, and we hypothesized simulated malingerers’ classification phase accuracy would be worse than head-injury controls’ classification phase accuracy.

As hypothesized, simulated malingerers performed worse on AGT classification accuracy than head-injury controls, t(80)=1.99, p=.05. The effect size was small to medium

(d=.44). See Table 5.

It was also hypothesized that simulated malingerers would perform worse on

AGT classification phase accuracy than memory-impaired participants. Contrary to this hypothesis, simulated malingerers’ AGT classification accuracy was not different from participants with memory impairment, although it was in the correct direction, t(40)=-.60, 27 p=.55, d=-.24. However, consistent with expectations, head-injury controls’ AGT classification accuracy did not differ from the memory-impaired participants’ classification accuracy, t(54)=-.56, p=.58, d=.22. See Table 5.

Given confirmation of the hypothesized difference between simulated malingerers and head-injury controls’ AGT classification accuracy, we were interested in determining whether or not any accuracy score might serve to effectively distinguish simulated malingerers from both head-injury controls and participants with memory impairment.

Based on speculative failure cutoffs (determined by a comparison of group cumulative percentages and trial and error), proposed AGT classification accuracy failure cutoffs resulted in pass/failure rates that were evenly distributed among the three groups. No cutoff score adequately distinguished simulated malingerers from head-injury controls and memory-impaired controls. See Table 6.

AGT Implicit Learning Trials

In addition to the classification phase accuracy score, which represents how well participants applied the implicit rules they learned during the AGT implicit learning phase, another study (Knowlton et al., 1996) attempted to track implicit learning differences by totaling the number of letter string presentations, or learning trials, utilized to correctly reproduce letter strings. We expected simulated malingerers would perform worse, that is require more learning trials to correctly reproduce the letter strings, than head-injury controls. As expected, simulated malingerers utilized more trials than head- injury controls, t(35.51)=-5.52, p<.001. The effect size was large (d=1.33). See Table 5. 28

Contrary to our hypothesis that simulated malingerers would also utilize more trials than memory-impaired participants, simulated malingerers’ performance did not differ from memory-impaired participants, t(40)=-1.80, p=.08, d=.79. Consistent with our hypothesis that there would be no difference in the total number of learning trials utilized by head-injury controls and memory-impaired participants, head-injury controls did not differ from memory-impaired participants, t(7.28)=-1.21, p=.26, d=.58.

Given that, as expected, simulated malingerers utilized more learning trials than head-injury controls, we were interested in determining whether or not a particular implicit learning trial total could serve to accurately distinguish simulated malingerers from head-injury controls and participants with memory impairment. A supplemental table in Appendix B displays the range of the total learning trials utilized among all participants and the cumulative percentages of participants who scored at or below each number. Speculative failure cutoff scores were based on a comparison of the cumulative percentages among groups and trial and error. Table 7 displays the proposed trial cutoffs

(for which failure was based on any trial totals that exceeded the cutoffs). Chi square analyses (also listed in Table 7) showed that given any of the listed cutoffs, AGT failure based on total learning trials, was not evenly distributed among the three groups. For example, given a trial cutoff of 52, for which any trial total over 52 was coded as a failure, failure was not evenly distributed among the groups, 2(2)=34.47, p < .001.

Given that cutoff, 23 of 34 (68%) simulated malingerers failed, while only 3 of 48 (6%) head-injury controls failed, and 3 of 8 (37%) memory-impaired participants failed based on the total number of trials it took them to accurately reproduce the letter strings that 29 were presented to them. In other words, failure sensitivity was 68% and specificity for head-injury controls was 94%; however, specificity for memory-impaired participants was 63%.

To consider the possible practical utility of this AGT implicit learning trial cutoff, it was then important to compare the results to the sample’s performance to existing

(explicit) embedded and standalone noncredible effort measures’ cut scores. Compared to other stand-alone and embedded measures of effort included in this study’s battery, the

AGT implicit learning trial cutoff of 52 resulted in the highest sensitivity for noncredible effort, with sufficient specificity for best effort in head-injury controls (see Table 8 for comparison).

Reaction time analyses.

To our knowledge, reaction time has not been researched using the AGT, and we were interested in conducting exploratory analyses to evaluate any reaction time differences in response to the AGT items. First of all, regarding classification phase response time in general, head-injury controls’ response time was positively skewed and positively kurtotic, and two head-injury controls’ response times represented statistical outliers. Removal of outliers normalized the data. Similarly, simulated malingerers’ classification phase response time was positively skewed and positively kurtotic. One simulated malingerer’s classification phase response time represented an outlier, yet this was not surprising given the instructions to feign. For the purposes of this study, data were analyzed with outlier scores in the models. We opted not to use nonparametric statistics when the outliers were retained, as many data analysts believe most tests are 30 extremely robust against violations of normality (Ito, 1980; vonEye & Bogat, 2004) and results in a fairly similar type I error rate and power (Finch, 2005).

We then investigated possible AGT response time differences based on the serial reaction time literature. The serial reaction time work has found amnesic patients’ reaction time (a corresponding key press to a cue presented in one of four locations) is faster following the presentation of a familiar sequence, as compared to a random sequence of locations (Reber & Squire, 1998). Thus, for the first exploratory analysis, we investigated participants’ response times (see Table 5) when they classified grammatical

(implicitly learned/familiar) items versus nongrammatical items.

A mixed model Analysis of Variance (ANOVA) revealed no main effect of

2 group, F(2,87)=1.293, p=.28, p =.03. There was a significant main effect of letter string type (grammatical or nongrammatical), F (1,87)=39.15, <.001, =.31. Group also interacted with grammatical or nongrammatical letter strings in some way, F(2,87)=4.03, p=.02, =.09. Post hoc follow-ups indicated all participants responded faster to grammatical items than nongrammatical items (memory-impaired participants t(7)=-2.40, p<.05, d=-0.49; head-injury controls t(47)=-3.75, p<.001, d=-0.19; and simulated malingerers t(33)=-3.97, p<.001, d=-0.21). The effect size of the difference in speed of response was largest for memory-impaired controls (a medium effect size versus the smaller effect size in the other two groups), likely explaining the significant interaction.

Faster response times on novel, yet grammatical, letter string items is consistent with other areas of the implicit learning and memory literature (i.e., the faster key press following a familiar, versus unfamiliar, sequence). 31

It was also of interest to investigate participants’ response times when choosing correct or incorrect answers (see Table 5). A mixed model ANOVA revealed a significant

2 main effect for correct or incorrect answer response time, F (1,87)=4.045, p=.047,  p

=.04, with generally faster responses to correct as opposed to incorrect items, regardless of group membership. There was no significant main effect for group, F(2,87)=1.322, p=.27, =.03, and there was no significant interaction between group and correct or incorrect answer response times, F (2,87)=.087, p=.917, =.002. Post-hoc tests to follow-up on the within subjects main effect showed that, although all were in the direction of showing faster responses to correct items, none of the three groups’ average correct answer response times significantly differed from their average incorrect answer response times. This was the case for head-injury controls, t(47)=-1.97, p=.06, d=.14, memory-impaired controls, t(7)=-.51, p=.63, d=.16, and simulated malingerers, t(34)=-

1.86, p=.07, d=.12.

Supplemental analyses.

For both simulated malingerers and head-injury controls, two-tailed bivariate

Pearson correlations were performed to examine relationships between AGT classification accuracy, AGT implicit learning trial performance and neuropsychological measures, separately for each group. For simulated malingerers, better AGT classification accuracy was associated with better WMT immediate recognition, r = .35, p = .04 and

WMT delayed recognition, r = .37, p = .03. Whereas, better AGT implicit learning trial performance was associated with better performance on all other measures, with large effect sizes (see Table 9; all ps ≤.01). However, for head-injury controls, better AGT 32 classification accuracy was related to better AVLT total learning, r = .32, p = .03, while better AGT implicit learning trial performance was related to better performance on many other measures, yet not to the same magnitude as was the case for simulated malingerers

(see Table 9 for additional details).

Discussion

The present study investigated whether the AGT, an implicit learning and forced- choice memory task, could serve as a novel noncredible effort/malingering measure. The existing AGT literature has predominantly focused on identifying clinical groups, and rather extensive research has shown healthy controls and people with memory impairment (as well as other neurological conditions) perform equally well on this task.

In light of this research, and given research findings showing that simulated malingerers perform worse than controls on explicit, forced-choice effort measures (Green, 2005), the present study was the first step in attempting to develop a novel measure of noncredible effort/malingering based on an implicit memory task.

We believe that proposed first step was well accomplished with the detection of a potential AGT implicit learning trial cutoff score. Given that cutoff score’s noncredible effort/malingering sensitivity and specificity, equivalent in accuracy to the WMT (which has been referred to as the “Gold Standard” noncredible effort/malingering measure; e.g.,

O’Bryant &, Lucas, 2006), the present findings suggest that the AGT has potential to be used in noncredible effort/malingering detection. It is noteworthy that, while implicit learning takes place due to exposure to grammatical letter strings, to advance through the learning phase task, participants must use conscious, declarative processing of the given information (i.e., consciously recalling and retyping specific letter strings), and it is likely it was this conscious, declarative processing that was more readily manipulated by simulated malingerers. The fact that supplemental analyses revealed better AGT implicit 34 learning trial performance was associated with better performance on many declarative, neuropsychological measures supports this notion, at least to some extent.

Alternatively, analysis of noncredible effort/malingering sensitivity and specificity data suggested that a cutoff score with reasonable accuracy could not be found for AGT classification accuracy scores. A general lack of association between performances on AGT classification accuracy and existing declarative, neuropsychological measures suggested AGT classification accuracy does seem to be related to implicit, non-declarative memory processes. With that being said, the fact that the significant association between simulated malingerers’ AGT classification accuracy and WMT immediate recognition and delayed recognition carried small to medium effect sizes implies further analyses in future studies may be warranted.

Consistent with expectations, the memory-impaired participants did not differ in

AGT performance relative to head-injury controls. However, contrary to predictions, simulators did not perform significantly worse than memory-impaired participants on either AGT variable. Overall, the simulated malingerers performed with enough sophistication that their averages were not distinguishable from the memory-impaired participants, although they were distinguishable from the head-injury controls. It is notable that this pattern is sometimes seen for noncredible effort/malingering measures requiring explicit memory (e.g., Teichner & Wagner, 2004; Dean, Victor, & Boone,

2009); even explicit memory-based measures of noncredible effort/malingering have a difficult time distinguishing malingered memory impairment from true memory impairment. Having a clearly memory-impaired comparison group is a stringent standard, 35 particularly for a measure of noncredible effort/malingering that is more likely to be administered to individuals with a mild head injury and no real-world evidence of amnesia/memory impairment. However, the lack of findings in the present study may further be influenced by the small sample size for the memory impaired group and its impact on power, especially given the AGT learning trial comparison at least approached significance, t(40)=-1.80, p=.08, d=.79.

Supplemental analyses investigating grammatical/nongrammatical and correct/incorrect item reaction times revealed all participants responded faster to grammatical items than nongrammatical items. In general, consistent with Reber and

Squire’s work (1998), reaction time in response to implicitly learned information was faster. Being that there were no differences in response time patterns between the simulated malingerers and other groups, the results suggested there is little promise in using AGT item response time in detecting noncredible effort/malingering.

Limitations and Future Directions

One possible limitation in the present study was the fact that the AGT was administered by computer, while all but one other previous study utilized hand administration of the AGT. It is possible the difference in test modality impacted optimal performance in the older, memory-impaired cohort, who was likely not as used to computer test taking. Research has suggested test modality (e.g., hand administration versus computerized administration of the Wisconsin Card Sorting Test) can impose marginally significant differences in the context of age differences (Rhodes, 2004).

Future research may consider increasing the memory-impaired participants’ sample size 36 and administering the AGT in both hand-administered and computerized form, to allow for a comparison of administration modality differences (especially in the older population).

In fact, given that this study did not use a memory-impaired comparison group that was age-matched to the other two participant groups, future studies may utilize a memory-impaired comparison group composed of ages better matched to the current simulated malingerers and head-injury controls. Particularly with regard to response time data, it may be important to consider general age-related response time differences, and a closer age-matched, memory-impaired group would be appropriate to further evaluate

AGT reaction time data in future studies. In addition, while use of a memory-impaired sample was appropriate to better isolate implicit learning and memory in this study, this or any memory-impaired group represents a very stringent comparison for identifying noncredible effort sensitivity in clinical settings assessing noncredible effort/malingering in mild head injury, in which severe memory impairment would not be expected to be a typical consequence.

Another possible limitation to the present study was the fact that our head-injury controls and simulated malingerer groups were composed of undergraduate students who were not seeking active treatment or compensation for their historical head injury, which potentially limits generalizability. Moreover, we did not confirm that the participants' self-reporting of mild head injury characteristics was objectively accurate; however, this is ecologically valid and consistent with other studies, and clinical realities, in which presenting individuals with a mild head injury history (who did not seek medical 37 attention at the time of injury) must rely on their self-recollected head injury characteristics. In addition, the students' demographics potentially imposed further limitations in generalizability with regard to age, education, race/ethnicity, etc. Some have argued, more broadly, that the use of a simulator design reduces the generalizability of findings. Rogers (1988) critiqued simulation studies because, while they offer a high degree of experimental control and internal validity, their external, ecological validity is limited. Inman and Berry (2002) made recommendations to improve upon simulator designs by including detailed malingering instructions, manipulation checks, an incentive, a control group, multiple indices, appropriate analyses, a test administrator who is unaware of the experimental condition, malingerers and controls with a history of head injury, clinically established cutting scores, and a battery of tests that mimics the experience of a standard neuropsychological assessment. In the present study, we adopted most of these recommendations. However, while an incentive was offered for recruitment into the study, it was not offered as a contingency related to participation. Some research

(Bernard, 1990; Erdal, 2004; and Weber, 2008) has found that simulating malingerers

(typically college undergraduates) offered a monetary incentive do not evade malingering detection at a higher rate than simulators who do not receive monetary incentives, and they sometimes simulate noncredible responding more flagrantly than undergraduate simulators who were not offered an incentive (Erdal, 2004), suggesting this is a minor concern. In general, then, the present simulated malingerer study was designed to best optimize internal and external validity, but further examination of the use of the AGT as a measure of noncredible effort/ malingering should utilize other research designs as well. 38

Considering the present study’s limitations, future studies should examine the noncredible effort/malingering sensitivity and specificity detection accuracy of our AGT implicit learning trial’s identified cutoffs. With the interest of cross-validation, future studies’ clinical control groups should be composed of individuals with more severe head injuries, and a broader range of neurological and psychological conditions, to replicate the AGT cutoff score’s strong specificity. To cross-validate the AGT cutoff’s sensitivity, future known-groups research designs should investigate AGT performance comparisons among age-matched patients with a history of a mild head injury seeking compensation

(where categorization of noncredible effort/malingering groups is determined by performance on established explicit effort measures; Slick, Sherman, & Iverson, 1999).

Future studies should also strive to obtain samples more diverse in age, education, and racial/ethnic status to better represent the likely samples in which the AGT would be utilized as a measure of noncredible effort/malingering.

References

American Academy of Clinical Neuropsychology (2007). American Academy of Clinical

Neuropsychology (AACN) practice guidelines for neuropsychological assessment and consultation. The Clinical Neuropsychologist, 21, 209–231.

doi: 10.1080/13854040903155063

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental

disorders (4th ed.). Washington, DC: Author.

Ben-Porath, Y. S. & Tellegen, A. (2008). MMPI–2–RF user’s guide for reports. Minneapolis: University of Minnesota Press.

Bernard, L. C. (1990). Prospects for faking believable memory deficits on neuropsychological tests and the use of incentives in simulation research. Journal of Clinical and Experimental Neuropsychology, 12(5), 715-728. doi: 10.1080/01688639008401014

Berry, D. R. & Schipper, L. J. (2007). Detection of feigned psychiatric symptoms during forensic neuropsychological examinations. In G. J. Larrabee (Ed.), Assessment of malingered neuropsychological deficits (pp. 226-263). New York: Oxford University Press.

Brown, J., Aczel, B., Jimenez, L., Barry Kaufman, S., & Plaisted Grant, K. (2010). Intact implicit learning in autism spectrum conditions. The Quarterly Journal of Experimental Psychology, 63(9), 1789-1812. doi:10.1080/17470210903536910

Bush, S. S., Ruff, R. M., Troster, A. I., Barth, J. T., Koffler, S.P., Pliskin, N. H., et al. (NAN Policy and Planning Committee). (2005). Symptom validity assessment: Practical issues and medical necessity. NAN position paper. Archives of Clinical Neuropsychology, 20(4), 419-426. doi:10.1016/j.acn.2005.02.002

Crowe, S. F. (1998). The differential contribution of mental tracking, cognitive flexibility, visual search, and motor speed to performance on parts A and B of the 40

Trail Making Test. Journal of Clinical Psychology, 54(5), 585-591. doi:10.1002/(SICI)1097-4679

Cheng, A. S., Ng, T. C., & Lee, H. C. (2011). A comparison of the hazard perception ability of accident-involved and accident-free motorcycle riders. Accident Analysis & Prevention, 43(4), 1464-1471. doi:10.1016/j.aap.2011.02.024

Danion , J., Meulemans, T., Kaufmann-Muller, F., & Vermaat, H. (2001). Intact Implicit learning in Schizophrenia. The American Journal of Psychiatry, 158(6), 944-948. doi:10.1176/appi.ajp.158.6.944

Dean, A. C., Victor, T. L., Boone, K. B., Philpott, L. M., & Hess, R. A. (2009). Dementia and effort test performance. The Clinical Neuropsychologist, 23(1), 133-152.

doi: 10.1080/13854040701819050

Dearth, C. S., Berry, D. T. R., Vickery, C. D., Vagnini, V. L., Baer, R. E., Orey, S. A., et al. (2005). Detection of feigned head injury symptoms on the MMPI-2 in head injured patients and community controls. Archives of Clinical Neuropsychology, 20, 95–110. doi: 10.1016/j.acn.2004.03.004

Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis-Kaplan executive function system (D-KEFS). Psychological Corporation. de Vries, M. H., Barth, A. C., Maiworm, S., Knecht, S., Zwitserlood, P., & Flöel, A. (2010). Electrical stimulation of Broca's area enhances implicit learning of an artificial grammar. Journal of Cognitive Neuroscience, 22(11), 2427-2436. doi:10.1162/jocn.2009.21385

Duff, K., Humphreys Clark, J. D., O’Bryant, S. E., Mold, J. W., Schiffer, R. B., & Sutker, P. B. (2008). Utility of the RBANS in detecting cognitive impairment associated with Alzheimer's disease: Sensitivity, specificity, and positive and negative predictive powers. Archives of Clinical Neuropsychology, 23(5), 603-612. doi:10.1016/j.acn.2008.06.004 41

Erdal, K. (2004). The effects of motivation, coaching, and knowledge of neuropsychology on the simulated malingering of head injury. Archives of Clinical Neuropsychology, 19(1), 73-88. doi: 10.1016/S0887-6177(02)00214-7

Fals-Stewart, W. (1992). An interrater reliability study of the Trail Making Test (Parts A and B). Perceptual and Motor Skills, 74(1), 39-42.

Fama, R., Pitel, A. L., & Sullivan, E. V. (2012). Anterograde episodic memory in Korsakoff syndrome. Neuropsychology Review, 22(2). doi: 10.1007/s11065-012- 9207-0

Finch, H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29(4), 278-295. doi: 10.1177/0146621605275728

Garcia, C., Leahy, B., Corradi, K., & Forchetti, C. (2008). Component structure of the Repeatable Battery for the Assessment of Neuropsychological Status in dementia. Archives of Clinical Neuropsychology, 23(1), 63-72.doi: 10.1016/j.acn.2007.08.008

Gervais, R. O., Ben-Porath, Y. S., Wygant, D. B., & Sellbom, M. (2010). Incremental validity of the MMPI-2-RF over-reporting scales and RBS in assessing the veracity of memory complaints. Archives of Clinical Neuropsychology, 25(4), 274-284. doi: 10.1093/arclin/acq018

Gervais, R. O., Rohling, M. L., Green, P., & Ford, W. (2004). A comparison of WMT, CARB, and TOMM failure rates in non-head injury disability claimants.Archives of Clinical Neuropsychology, 19(4), 475-487. doi: 10.1016/j.acn.2003.05.001

Ginó, S., Mendes, T., Maroco, J., Ribeiro, F., Schmand, B. A., de Mendonça, A., & Guerreiro, M. (2009). Memory complaints are frequent but qualitatively different in young and elderly healthy people. Gerontology, 56(3), 272-277.

doi: 10.1159/000240048

Green, P. (2005). Green's Word Memory Test for Microsoft Windows: User's manual. Green's Publications Incorporated. 42

Green, P., Allen, L. M., & Astner, K. (1996). The Word Memory Test: A user’s guide to the oral and computer-administered forms, US Version 1.1. Durham, NC: CogniSyst.

Groth-Marnat, G. & Teal, M. (2000). Block design as a measure of everyday spatial ability: a study of ecological validity. Perceptual and Motor Skills, 90(2), 522- 526. doi:10.2466/pms.2000.90.2.522

Haines, M.E. & Norris, M.P. (2001). Comparing student and patient simulated malingerers’ on standard neuropsychological measures to detect feigned cognitive deficits. The Clinical Neuropsychologist, 15 (2), 171-182. doi:10.1076/clin.15.2.171.1891

Hayes, S., Fortier, C., Levine, A., Milberg, W., & McGlinchey, R. (2012). Implicit memory in Korsakoff's syndrome: a review of procedural learning and priming studies. Neuropsychology Review, 22(2), 132-153. doi: 10.1007/s11065-012- 9204-3

Heilbronner, R.L., Sweet, J.J., Morgan, J.E., Larrabee, G.J., Millis, S.R. & Conference Participants. (2009). American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical neuropsychologist, 23(7), 1093-1129.

doi: 10.1080/13854040903155063

Hobson, V. L., Hall, J. R., Humphreys-Clark, J. D., Schrimsher, G. W., & O'Bryant, S. E. (2010). Identifying functional impairment with scores from the repeatable battery for the assessment of neuropsychological status (RBANS). International Journal of Geriatric Psychiatry, 25(5), 525-530. doi:10.1002/gps.2382

Greiffenstein, M. F., Baker, W., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6, 218-224. doi: 10.1037/1040-3590.6.3.218 43

Inman, T. H. & Berry, D. T. R. (2002). Cross-validation of indicators of malingering: A comparison of nine neuropsychological tests, four tests of malingering, and behavioral observations. Archives of Clinical Neuropsychology, 17, 1-23.

doi:10.1016/S0887-6177(00)00073-1

Ito, P. K. (1980). 7 Robustness of ANOVA and MANOVA test procedures. Handbook of statistics, 1, 199-236.

Iverson, G., Franzen, M., & McCracken, L. (1994). Application of a forced-choice memory procedure designed to detect experimental malingering. Archives of Clinical Neuropsychology, 9(5), 437-450. doi:10.1016/0887-6177(94)90006-X

Jasinski, L., Berry, D., Shandera, A., & Clark, J. (2011). Use of the Wechsler Adult Intelligence Scale Digit Span subtest for malingering detection: A meta-analytic review. Neuropsychology, Development, And Cognition. Section A, Journal Of Clinical And Experimental Neuropsychology, 33(3), 300-314.

doi:10.1080/13803395.2010.516743

Kessels, R. & Kopelman, M. (2012). Context memory in Korsakoff's syndrome. Neuropsychology Review, 22(2), 117-131. doi: 10.1007/s11065-012-9202-5

Knowlton, B., Ramus, S.J., Squire, L.R. (1992). Intact artificial grammar learning in amnesia: Dissociation of classification learning and explicit memory for specific instances. Psychological Science, 3(3), 172. doi:10.1111/j.1467- 9280.1992.tb00021.x

Knowlton, B.J., Squire, L.R., Paulsen, J.S., Swerdlow, N.R., Swenson, M., & Butters, N. (1996). Dissociations within nondeclarative memory in Huntington's disease. Neuropsychology, 10(4), 538. doi:10.1037/0894-4105.10.4.538

Larrabee, G. J. (2007). Preface and Introduction: Malingering, research designs, and base rates. In G. J. Larrabee (Ed.), Assessment of malingered neuropsychological deficits (pp. 3-10). New York: Oxford University Press. 44

Lundstrom, B. N., Petersson, K. M., Andersson, J., Johansson, M., Fransson, P., & Ingvar, M. (2003). Isolating the retrieval of imagined pictures during episodic memory: Activation of the left precuneus and left prefrontal cortex. Neuroimage, 20(4), 1934-1943. doi: 10.1016/j.bbr.2011.03.031

Mitrushina, M. & Satz, P. (1991). Effect of repeated administration of a neuropsychological battery in the elderly. Journal of Clinical Psychology, 47(6), 790-801. doi:10.1002/1097-4679(199111)47:6<790::AID- JCLP2270470610>3.0.CO;2-C

Mittenberg, W., Patton, C., Canyock, E., & Condit, D. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 24(8), 1094-1102. doi:10.1076/jcen.24.8.1094.8379

Nitch, S. R. & Glassmire, D. M. (2007). Non-forced choice measures to detect noncredible cognitive performance. In K.B. Boone (Ed.), Assessment of feigned cognitive impairment: A neuropsychological perspective (pp. 27-43). New York: The Guilford Press.

O’Bryant, S.E. & Lucas, J.A. (2006). Estimating the predictive value of the Test of Memory Malingering: An illustrative example for clinicians. The Clinical Neuropsychologist, 20(3), 533-540. doi: 10.1080/13854040590967568

Pachet, A. K. (2007). Construct validity of the Repeatable Battery of Neuropsychological Status (RBANS) with acquired brain injury patients. The Clinical Neuropsychologist, 21(2), 286-297. doi:10.1080/13854040500376823

Pothos, E. (2007). Theories of artificial grammar learning. Psychological Bulletin, 133(2), 227-244. doi: 10.1037/0033-2909.133.2.227

Randolph, C. (2012). Repeatable Battery for the Assessment of Neuropsychological Status Manual. San Antonio, TX: Psychological Corporation.

Rathus, J. H., Reber, A. S., Manza, A., & Kushner, M. (1994). Implicit and explicit learning: Differential effects of affective states. Perceptual & Motor Skills, 79(1), 163. doi: 10.2466/pms.1994.79.1.163 45

Reber, A. S. (1967). Implicit learning of artificial grammars. Journal of Verbal Learning and Verbal Behavior, 77, 317-327. doi: 10.1016/S0022-5371(67)80149-X

Reber, A. S. (1989). Implicit learning and tacit knowledge. Journal of Experimental Psychology: General, 118(3), 219. doi:10.1037/0096-3445.118.3.219

Reber, A. S. & Allen, R. (1978). Analogy and abstraction strategies in synthetic grammar learning: A functionalist interpretation. Cognition, 6, 189-221. doi:10.1016/0010- 0277(78)90013-6

Reber, P., Martinez, L., & Weintraub, S. (2003). Artificial grammar learning in Alzheimer's disease. Cognitive, Affective & Behavioral Neuroscience, 3(2), 145- 153. doi: 10.3758/CABN.3.2.145

Reber, A.S. & Squire, L.R. (1998). Encapsulation of implicit and explicit memory in sequence learning. Journal of Cognitive Neuroscience, 10(2), 248-263. doi:10.1162/089892998562681

Reber, A.S. & Squire, L. R. (1999). Intact learning of artificial grammars and intact category learning by patients with Parkinson's disease. Behavioral Neuroscience, 113(2), 235-242. doi:10.1037/0735-7044.113.2.235.

Redington, M. & Chater, N. (1996). Transfer in artificial grammar learning: A reevaluation. Journal of Experimental Psychology, 125(2), 123-138. doi: 10.1037/00963445.125.2.123

Reese, C. S., Suhr, J. A., & Riddle, T. L. (2012). Exploration of Malingering Indices in the Wechsler Adult Intelligence Scale-Digit Span Subtest. Archives of clinical neuropsychology, 27 (2), 176-181. doi: 10.1093/arclin/acr117

Reitan, R. M., & Wolfson, D. (1985). Neuroanatomy and Neuropathology: A Clinical Guide for Neuropsychologists. Neuro Psychology Press

Reitan, R.M. & Wolfson, D. (1993). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation (2nd ed.). Tucson, AZ: Neuropsychology Press. 46

Rey, A. (1964). L'examen clinique en psychologie. [The Clinical Exam in Psychology]. Paris: Presses universitaires de France.

Rhodes, M. G. (2004). Age-related differences in performance on the Wisconsin card sorting test: a meta-analytic review. Psychology and Aging, 19(3), 482-494. doi: 10.1037/0882-7974.19.3.482

Ríos, M., Periáñez, J. A., & Muñoz-Céspedes, J. M. (2004). Attentional control and slowness of information processing after severe traumatic brain injury. Brain Injury, 18(3), 257-272. doi:10.1080/02699050310001617442

Rogers, R. (1988). In R. Rogers (Ed.), Clinical assessment of malingering and deception (pp.309-327). New York: Guilford.

Rogers, R. (2008). Introduction. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed., pp. 8-9, 367). New York: Guilford.

Russeler, J., Gerth, I., & Munte, T. (2006). Implicit learning is intact in adult developmental dyslexic readers: Evidence from the Serial Reaction Time Task and Artificial Grammar Learning. Journal of Clinical And Experimental Neuropsychology, 28(5), 808-827. doi:10.1080/13803390591001007

Schacter, D. L. & Buckner, R.L. (1998). Priming and the brain. Neuron, 20, 185-195.

Schacter, D. L. & Curran, T. (2000). Memory without remembering and remembering

without memory: implicit and false memories. In The new cognitive neurosciences, M. S. Gazzaniga (ed.), 829–840. Cambridge, MA: MIT Press.

Schmidt, M. (1996). Rey Auditory Verbal Learning Test: A Handbook. Los Angeles, California: Western Psychological Services.

Seger, C. A., Prabhakaran, V., Poldrack, R. A., & Gabrieli, J. D. E. (2000). Neural activity differs between explicit and implicit learning of artificial grammar strings: An fMRI study. Psychobiology, 28, 283–292. 47

Sharland, M.J. & Gfeller, J.D. (2007). A survey of neuropsychologists’ beliefs and practices with respect to the assessment of effort. Archives of Clinical Neuropsychology, 22, 213–223.doi: 10.1016/j.acn.2006.12.004

Slick, D. J., Sherman, E. M., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545-561. doi:10.1076/1385- 4046(199911)13:04;1-Y;FT545

Slick, D. J., Tan, J. E., Strauss, E. H., & Hultsch, D. F. (2004). Detecting malingering: A survey of experts’ practices. Archives of Clinical Neuropsychology, 19(4), 465- 473. doi: 10.1016/j.acn.2003.04.001

Speekenbrink, M., Channon, S., & Shanks, D. (2008). Learning strategies in amnesia. Neuroscience and Biobehavioral Reviews, 32(2), 292-310. doi:10.1016/j.neubiorev.2007.07.005

Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643-662. doi:10.1037/h0054651

Squire, L. (2009). Memory and brain systems: 1969-2009. The Journal of Neuroscience, 29(41), 12711-12716. doi: 10.1523/JNEUROSCI.3575-09.2009

Suhr, J. A. & Gunstad, J. J. (2007). Coaching and malingering: A review. In G. J. Larrabee (Ed.), Assessment of malingered neuropsychological deficits (pp. 287- 311). New York: Oxford University Press.

Sweet, J. J., Condit, D. C., & Nelson, N. W. (2008). Feigned amnesia and memory loss. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed., pp. 218–236). New York: Guilford.

Teichner, G. & Wagner, M. T. (2004). The Test of Memory Malingering (TOMM): Normative data from cognitively intact, cognitively impaired, and elderly patients with dementia. Archives of Clinical Neuropsychology, 19(3), 455-464.doi: 10.1016/S08876177(03)00078-7 48

Tellegen, A. & Ben-Porath, Y. S. (2008). MMPI-2-RF, Minnesota Multiphasic Personality Inventory-2 Restructured Form: Technical Manual. University of Minnesota Press.

Thomas, K., Hunt, R., Vizueta, N., Sommer, T., Durston, S., Yang, Y., & Worden, M. (2004). Evidence of developmental differences in implicit sequence learning: An fMRI study of children and adults. Journal of Cognitive Neuroscience, 16(8), 1339-1351. doi:10.1162/0898929042304688

Uchiyama, C., D'Elia, L., Dellinger, A., Becker, J., Selnes, O., Wesch, J., & Miller, E. (1995). Alternate forms of the Auditory-Verbal Learning Test: issues of test comparability, longitudinal reliability, and moderating demographic variables. Archives of Clinical Neuropsychology: The Official Journal of The National Academy of Neuropsychologists, 10(2), 133-145. doi:10.1016/0887- 6177(94)E0034-M

Victor, T. & Abeles, N. (2004). Coaching clients to take psychological and neuropsychological tests: A clash of ethical obligations. Professional Psychology- Research and Practice, 35(4), 373-379.doi: 10.1037/0735-7028.35.4.373

Vinter, A. A., Pacton, S., Witt, A., & Perruchet, P. (2010). Implicit learning, development, and education. In J.P. Didier & E. Bigand (Eds.), Rethinking physical and rehabilitation medicine (pp. 111-127). Paris: Springer Verlag.

Visual Awareness Research Group, Inc. (2009). Useful Field of View user’s guide. Punta Gorda, FL: Visual Awareness Research Group, Inc.

Von Eye, A. & Bogat, G. A. (2004). Testing the assumption of multivariate normality. Psychology Science, 46, 243-258.

Weber, K.A. (2008). Improving external validity with simulators in malingering research. Dissertation Abstracts International, 69, 6B. (UMI No. AAI3321203) Retrieved October 9, 2009, from PsychInfo.

Wechsler, D. (2001). Wechsler Test of Adult Reading: WTAR. San Antonio, TX: Pearson. 49

Wechsler, D. (2008). Wechsler Adult Intelligence Scale–Fourth Edition. San Antonio, TX: Pearson.

Wechsler, D. (1987). Wechsler Memory Scale-Revised. San Antonio, TX: Pearson.

Williams, J.M. (1998). The malingering of memory disorder. In C.R. Reynolds (Ed.), Detection of malingering during head injury litigation (pp. 105-132). New York: Plenum Press.

Young, J. C., Sawyer, R. J., Roper, B. L., & Baughman, B. C. (2012). Expansion and re- examination of Digit Span effort indices on the WAIS-IV. The Clinical Neuropsychologist, 26(1), 147-159. doi: 10.1080/13854046.2011.647083 50

Table 1

Demographic Information and Head Injury Characteristics Across Groups

H.I. S.M. Controls Memory Impaired

Demographics Age (Mean, SD) 18.94, .85 19.02, .79 73.5, 8.60 Education (Mean, SD) 13.35, .69 13.42, .74 17.5, 2.14 Gender (% Male) 65% 50% 63% Race/Ethnicity (% Caucasian) 94% 92% 100%

Head Injury Characteristics Mean , SD Mean, SD LOC Length (minutes) 3.03, 7.28 1.70, 4.11 Years Since Most Recent Head Injury 5.31, 3.89 4.24, 2.89 Total Number of Head Injuries 1.41, .91 1.29, .64 Treatment Following Injury (% No) 55% 66%

Note:LOC=Loss of Consciousness; S.M.=Simulated Malingerers; H.I.=Head Injury

Table 2

The Standard Set of Acquisition and Test Strings, First Used by Reber and Allen (1978)

Test Training Grammatical Nongrammatical MSSSSV VXSSSV VXRRS MSSVS MSSSV VXX MSV MSSVRX VXRVM MSVRX MVRXVS XVRXRR MSVRXM MSVRXV XSSSSV MVRX MSVRXR MXVV MVRXRR MVRXM MMVRX MVRXSV VXVRXR MVRSR MVRXV MSSSVS MSRVRX MVRXVS VXRM SSVS VXM MVS MSSVSR VXRR MSVS RVS VXRRM MSSVRX MXVS VXRRRR MVRXR VRRRM VXSSVS VXRRR VVXRM VXSVRX VXSV VXRS VXSVS VXR MSRV VXVRX VXVS VXMRXV VXVRXV MSV MSM VXVS VXRRRM SXRRM VXSSV MXVRXM VXV MSVRSR VXVRX SVSSXV VXVRXV XRVXV MVRXRM RRRXV

Note. For nongrammatical strings, bolding indicates the point of grammatical violation.

Table 3

Means and SD for Non-study Neuropsychological Test Performance in Study Groups

S.M. H.I. Controls

Mean, SD Mean, SD WMT Immediate Recognition 72.24, 22.7 97.33, 8.67 WMT Delayed Recognition 75.0, 21.85 98.06, 7.48 WMT Consistency 73.76, 18.12 96.52, 9.87 Digit Span Forward 8.12, 2.38 10.88, 2.03 Digit Span Backward 6.79, 1.74 8.92, 2.45 Digit Span Sequencing 7.58, 2.05 9.40, 1.87 Digit Span Total Raw Score 22.55, 5.15 29.19, 5.19 AVLT Learning Total 43.15, 7.99 51.10, 9.86 AVLT Immediate Recall 8.09, 2.79 11.79, 2.70 AVLT 30-Minute Delayed Recall 7.32, 3.48 11.08, 2.89 AVLT 30-Minute Delayed Recognition 12.59, 2.24 14.23, .90

Note . S.M.=Simulated Malingerer and H.I.=Head-Injury; WMT IR = WMT Immediate Recognition; WMT DR = WMT Delayed Recognition; DS FWD = Digit Span Forward; DS BWD = Digit Span Backward; DS SEQ = Digit Span Sequencing; DS TOTAL = Digit Span Total Raw Score; AVLT TOTAL = AVLT Total Number of Words Learned; AVLT IR = AVLT Immediate Recall; AVLT 30-DR = AVLT 30-Minute Delayed Recall; and AVLT 30-REC = AVLT 30-Minute Recognition

able 4

Percentages of Validated Effort Measure Failures between Groups

Effort Measure Failure S.M. H.I. Controls Χ2(2) p WMT (≤82.5) 68% 6% 34.65 <.001 Original RDS (≤6) 27% 0% 14.27 <.001 Original RDS (≤7) 50% 6% 20.66 <.001 Enhanced RDS (≤11) 42% 8% 13.15 <.001 MMPI-2-RF RBS (≥80) 48% 9% 10.85 =.001

Note. S.M.=Simulated Malingerer and H.I.=Head-Injury

Table 5

Descriptive Details Pertaining to Dependent Variables

S.M. H.I. Controls Memory Impaired

Mean, SD Mean, SD Mean, SD AGT Classification Accuracy 55.44, 7.84 58.77, 7.17 57.25, 6.96 Score1 AGT Implicit Learning Phase 73.32, 27.81 46.50, 6.43 54.63, 18.82 Trials1 Grammatical Item Response Time 2.11, 1.04 2.46, 1.14 2.41, 1.28 (sec.)2 Nongrammatical Item Response 2.33, 1.04 2.68, 1.21 3.08, 1.43 Time (sec.) Correct Item Response Time (sec.) 2.17, 0.98 2.52, 1.20 2.65, 1.19 Incorrect Item Response Time 2.30, 1.17 2.69, 1.22 2.88, 1.69 (sec.)

Note: S.M.=Simulated Malingerers; H.I.=Head Injury 1-Significant differences between S.M. and H.I. 2-Significantly different than Nongrammatical Item Response Time in all groups. Sec.=seconds

Table 6

Simulated Mal ingerer Sensitivity/Specificity based on Accuracy Scores

Specificity Specificity Accuracy Sensitivity (Ctrl) (MI) Χ2(2) p 51 29% 85% 75% 2.71 0.26 59 65% 44% 25% 1.31 0.52 64 88% 21% 12.5% 1.29 0.53 67 97% 12.5% 12.5% 2.39 0.30

Note. Failure was categorized by scores less than or equal to the listed scores.

Table 7

Simulated Malingerer Sensitivity /Specificity based on Implicit Learning Trial Cutoff Score

Specificity Trial Cutoff Sensitivity (Ctrl) Specificity (MI) Χ2(2) p 50 76% 81% 63% 27.26 <.001 51 68% 90% 63% 28.91 <.001 52 68% 94% 63% 34.47 <.001 53 68% 94% 63% 34.47 <.001 54 65% 94% 63% 31.9 <.001

Note. Failure was categorized by trial scores that exceeded the trial cutoff.

able8

Noncredible Effort Sensitivity/Specificity based on AGT Learning Trials as compared to Well Validated Effort Measures

Specificity Specificity Effort Measure Sensitivity (Ctrl) (MI) Χ2(2) p AGT Learning Trials (>52) 68% 94% 63% 34.47 <.001 WMT (≤82.5) 68% 94% n/a 34.65 <.001 Original RDS (≤6) 26% 100% n/a 14.27 <.001 Original RDS (≤7) 50% 94% n/a 20.66 <.001 Enhanced RDS (≤11) 42% 92% n/a 13.15 <.001 MMPI-2-RF RBS (≥80) 48% 91.0% n/a 10.85 =.001

Table 9

Correlations between AGT and Other Administered Measures

AGT Classification Accuracy AGT Implicit Learning Trials S.M. H.I. Controls S.M. H.I. Controls WMT IR .35* .06 -.76** -.26 WMT DR .37* .24 -.69** -.32* DS FWD .06 .09 -.61** -.09 DS BWD .06 .23 -.70** -.33* DS SEQ -.15 .22 -.57** -.33* DS TOTAL -.02 .22 -.76** -.31* AVLT TOTAL .15 .32* -.54** -.41** AVLT IR .18 .19 -.44** -.26 AVLT 30-DR .10 .23 -.58** -.38** AVLT 30-REC .17 .20 -71** -.32*

Note. S.M.=Simulated Malingerer and H.I.=Head-Injury; WMT IR = WMT Immediate Recognition; WMT DR = WMT Delayed Recognition; DS FWD = Digit Span Forward; DS BWD = Digit Span Backward; DS SEQ = Digit Span Sequencing; DS TOTAL = Digit Span Total Raw Score; AVLT TOTAL = AVLT Total Number of Words Learned; AVLT IR = AVLT Immediate Recall; AVLT 30-DR = AVLT 30-Minute Delayed Recall; and AVLT 30-REC = AVLT 30-Minute Recognition ** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). 59

Appendix A: Forms Used in Experiment

Instructions

Researchers: Caitlin Reese, M.A/M.S. and Julie A. Suhr, Ph.D., Ohio University

Department: Psychology

Today you will take a series of neuropsychological tests that assess motor speed, attention, memory, and thinking skills. You are asked to give your best effort on all of these tests.

How difficult do you think it will be to do your best on the motor speed, attention, memory, and thinking skills tests today?

1 2 3 4 5 not difficult somewhat difficult very difficult

Please do not share your instructions with anyone else, including the individual who gives you your tests, as different people have different instructions. The individual testing you will return momentarily. Please wait patiently.

Thanks for your participation.

Instructions

Researchers: Caitlin Reese, M.A./ M.S. and Julie A. Suhr, Ph.D., Ohio University

Department: Psychology

Today you will take a series of neuropsychological tests that assess motor speed, attention, memory, and thinking skills. You are being asked to believably pretend that you have significant problems (e.g., representative of brain damage) with motor speed, attention, memory, and thinking skills tests. In other words, pretend that you are someone involved in a lawsuit, and you want to pretend to have brain damage in order to win a financial settlement. What might you do to indicate (even though you do not) that you have permanent and significant problems with motor speed, attention, memory, and thinking skills while taking neuropsychological tests? There is no wrong answer. Supplemental materials have also been provided for you to read and give you additional ideas regarding how to pretend to have significant problems with motor speed, attention, memory, and thinking skills. The examiner who gives you the tests does not know that you will be pretending to have significant problems representative of brain damage, and of course, you don’t want to get caught pretending, therefore it is important to remind you to believably pretend to have such problems however you see fit. Please take a few moments to review the supplemental materials that you’ve received.

How difficult do you think it will be to pretend that you have significant problems (e.g., representative of brain damage) with motor speed, attention, memory, and thinking skills tests?

1 2 3 4 5 not difficult somewhat difficult very difficult

Just as a reminder, please do not share your instructions with anyone else, including the individual who gives you your tests, as different people have different instructions. The individual testing you will return momentarily. Please wait patiently.

Thanks for your participation.

Consent Form

Title of Research: Performance on Cognitive Tests Under Different Conditions Researchers: Caitlin Reese, M.A/M.S. and Julie A. Suhr, Ph.D., Ohio University

Department: Psychology

You are being asked to participate in research. For you to be able to decide whether you want to participate in this project, you should understand what the project is about, as well as the possible risks and benefits in order to make an informed decision. This process is known as informed consent. This form describes the purpose, procedures, possible benefits, and risks. It also explains how your personal information will be used and protected. Once you have read this form and your questions about the study are answered, you will be asked to sign it. This will allow your participation in this study. You should receive a copy of this document to take with you.

Explanation of Study You are invited to participate in a research study exploring performance on different types of cognitive tests under different conditions. The oral, computerized, and written tests assess cognitive abilities such as attention, verbal fluency, and general thinking skills. Participation is voluntary and may be discontinued at any time without penalty.

All tests will be administered by a graduate student or trained undergraduate research assistant supervised by the study directors. We expect that your participation will take approximately one and one half hours.

Risks and Discomforts There are no known risks associated with this study. Though it is possible that some participants may experience discomfort while participating in our study, we are not aware of any reason why this study would impose any discomfort upon anyone, and we remind participants that they have no obligation to complete any surveys or cognitive tests if they do not wish to, for any reason.

The data you provide will be maintained confidentially. A participant number will be unique and will not be connected to identifying information in any way.

Benefits Participants in this study will benefit from it by learning more about neuropsychological testing. Your participation will also enhance our understanding of what types of cognitive tests are suited for the most valid neuropsychological testing. Such knowledge would be important for neuropsychologists who practice in both clinical and forensic settings.

Confidentiality and Records All information obtained from you in this study will be kept strictly confidential. This information will be identified according to a unique subject identification number. There is no way that your name can or will be linked to your data. In addition, the data from this study will be kept in a locked storage facility and accessible only to authorized individuals.

Compensation You will receive one and one half experimental credit points for full participation in this study. If for any reason you feel you need to discontinue your participation in this study, you will receive one-half experimental credit point reflecting the time you spent in the study (i.e., you will receive one-half point for each half-hour you spend in the study).

Contact Information If you have any questions about your participation, please do not hesitate to ask the experimenter. You may also contact the study co-investigators, Caitlin Reese, M.A., M.S. ([email protected], (419) 509-1742) or Julie A. Suhr, Ph.D. ([email protected], (740) 593-1091) if you have additional questions or concerns.

If you have any questions regarding your rights as a research participant, please contact Jo Ellen Sherow, Director of Research Compliance, Ohio University, (740) 593-0664.

By signing below, you are agreeing that: you have read this consent form (or it has been read to you) and have been given the opportunity to ask questions known risks to you have been explained to your satisfaction. you understand Ohio University has no policy or plan to pay for any injuries you might receive as a result of participating in this research protocol you are 18 years of age or older your participation in this research is given voluntarily you may change your mind and stop participation at any time without penalty or loss of any benefits to which you may otherwise be entitled.

Signature Date

Printed Name

Version Date: [11/04/12]

Consent Form

Title of Research: Performance on Cognitive Tests Under Different Conditions Researchers: Caitlin Reese, M.A/M.S. and Julie A. Suhr, Ph.D., Ohio University

Department: Psychology

The data you provide will be maintained confidentially. A participant number will be unique and will not be connected to identifying information in any way.

Compensation You will receive $10 and one and one half experimental credit points for full participation in this study. If for any reason you feel you need to discontinue your participation in this study, you will receive one-half experimental credit point reflecting the time you spent in the study (i.e., you will receive one-half point for each half-hour you spend in the study). Likewise, if you decide to discontinue at any time, you will receive the amount of money commensurate with the time you spent in the study (i.e., you will receive $3.33 for each half-hour you spend in the study).

If you have any questions regarding your rights as a research participant, please contact Jo Ellen Sherow, Director of Research Compliance, Ohio University, (740) 593-0664.

Signature Date

Printed Name

Version Date: [11/04/12 65

Demographics/Medical History Questionnaire

DIRECTIONS: Please choose the best response for each question.

1. What is your age? ______

2. What is your current year in college? ______

3. What is your current overall college GPA? ______

4. What is your racial/ethnic identity? A. Caucasian, Non-Hispanic B. African American C. Latino or Hispanic D. Asian or Pacific Islander E. American Indian or Alaska Native F. Two or more races G. Other

5. Are you currently diagnosed with depression, anxiety, or other psychological ailment? A. No B. Yes If yes, what condition(s)______

6. Are you currently receiving treatment (e.g., prescription medication, counseling, herbal supplements) for depression, anxiety, or another psychological ailment? A. No B. Yes If yes, what treatment(s)?______

7. To the best of your knowledge, do you have a diagnosed learning disorder or attention deficit/hyperactivity disorder (ADHD)? A. No B. Yes If yes, which one(s)______

8. To the best of your knowledge, are you currently diagnosed with any neurological deficit, impairment, or diagnosis? A. No B. Yes If so, which one?______

9. Have you ever lost consciousness, even if only for a few seconds, following a head injury or concussion? A. No B. Yes

If so, about how long did you lose consciousness?______

Approximately when did this incident take place? ______

What were the circumstances surrounding the injury/ies (e.g., sports related, traffic related accident, accident associated with alcohol, childhood accident, blunt force trauma, etc.)?

Have you lost consciousness due to a head injury or concussion on any other occasions? ______

How many? ______

Approximately when did they occur (i.e., what year(s) or how many days/months ago?)

What were the circumstances surrounding these injury/ies (e.g., sports related, traffic related accident, accident associated with alcohol, childhood accident, blunt force trauma, etc.)?

Have you ever received any treatment following any loss of consciousness?

SYMPTOMS OF BRAIN INJURY

Any brain function can be disrupted by brain trauma: excessive sleepiness, inattention, difficulty concentrating, impaired memory, faulty judgment, depression, irritability, emotional outbursts, disturbed sleep, diminished libido, difficulty switching between two tasks, and slowed thinking.

The extent and the severity of cognitive neurologic dysfunction can be measured with the aid of neuropsychological testing. Neuropsychologists use their tests to localize dysfunction to specific areas of the brain.

In general, symptoms of traumatic brain injury should lessen over time as the brain heals but sometimes the symptoms worsen because of the patient's inability to adapt to the brain injury. For this and other reasons, it is not uncommon for psychological problems to arise and worsen after brain injury.

SYMPTOM CHECKLIST

A wide variety of symptoms can occur after "brain injury." The nature of the symptoms depends, in large part, on where the brain has been injured. Below find a list of possible physical and cognitive symptoms which can arise from damage to specific areas of the brain:

Loss of flexibility in thinking

Persistence of a single thought

Inability to focus on task

Mood changes

Changes in social behavior

Frontal Lobe: Forehead Changes in personality

Loss of simple movement of various Difficulty with problem solving body parts Inability to express language Inability to plan a sequence of complex movements needed to complete multi- Parietal Lobe: near the back and stepped tasks, such as making coffee top of the head

Loss of spontaneity in interacting with Inability to attend to more than one others object at a time

Inability to name an object Difficulties with reading and writing

Inability to locate the words for writing Temporal Lobes: side of head above ears Problems with reading Difficulty in recognizing faces Difficulty with drawing objects. Difficulty in understanding spoken Difficulty in distinguishing left from words right. Disturbance with selective attention to Difficulty with doing mathematics what we see and hear

Lack of awareness of certain body Difficulty with identification of, and parts and/or surrounding space that verbalization about objects leads to difficulties in self-care Short term memory loss Inability to focus visual attention Interference with long term memory Difficulties with eye and hand coordination Increased and decreased interest in sexual behavior Occipital Lobes: most posterior, at the back of the head Inability to categorize objects

Defects in vision Right lobe damage can cause persistent talking. Difficulty with locating objects in environment. Increased aggressive behavior

Difficulty with identifying colors

Production of hallucinations

Visual illusions - inaccurately seeing objects

Word blindness - inability to recognize words

Difficulty in recognizing drawn objects

Inability to recognize the movement of object

Manipulation Check

Earlier, you were given instructions regarding how you should perform on the tests that you were just administered.

Were you asked to (please circle your answer):

1. Perform with your best effort

2. Pretend to have cognitive impairment

Do you believe that, as best as you could, you followed the instructions that you earlier received while participating in this study?

YES NO

AGT Post-Task Form

-Do you think that what you learned from the first list influenced your learning on the second list?

Yes or No

- How good are you at learning these types of lists?

1 2 3 4 5

Not good at all Not Very Good Okay Good Very Good

-To what extent did you rely on your gut instinct to choose yes or no?

1 2 3 4 5

Not at all A Little Moderately Very Much A Great Amount

-Other than your gut instinct, did you use any other strategy/recognize any rules to help you choose yes or no?

1 2 3 4 5

Not at all A Little Moderately Very Much A Great Amount

-Can you briefly describe what you used/recognized? (FILL IN BELOW)

(ONLY IF A SIMULATED MALINGER, ASK): -You were instructed to fake impairment during this battery of tests, what did you do to perform worse than you otherwise would when you were shown strings of letters (SHOW INDEX CARD WITH A SAMPLE LETTER STRING TO REMIND HIM/HER OF THE STIMULI) and asked to say ‘yes’ or ‘no’ depending on if you thought the letter string followed the rules?

Post Study Form

Title of Research: Performance on Cognitive Tests Under Different Conditions Researchers: Caitlin Reese, M.A., M.S. & Julie A. Suhr, Ph.D., Ohio University Department: Psychology

Thank you for your participation. The study in which you just participated examined whether people who intend to malinger (i.e., fake cognitive impairments) perform differently on an implicit memory task compared to people who perform with their best effort. It is important to develop novel, sensitive, specific effort/malingering measures, because people continue to fake cognitive impairment during forensic neuropsychological assessments. People who pretend to have brain damage (malingerers) usually do so because they want to win a lawsuit or similar incentive, and they often do so after they have sustained only a mild head injury. In order for our results to generalize, or apply, to this population, participants (including you) were recruited because they reported having experienced a mild head injury in the past.

Some participants were asked to do their best on the cognitive tests, and some participants were asked to pretend to have brain dysfunction while taking the tests. Those individuals who were asked to pretend to have brain dysfunction received web-based information about head injury; this information was accurate yet incomplete. The website from which the head injury information came was a lawyer’s website that over-exaggerates and generalizes head injury symptom severity. It is true that groups of people who have sustained head injuries sometimes perform worse on cognitive tests than people who have not. However, the groups who tend to perform poorly have much more severe head injuries than average. Furthermore, many individuals who sustain a mild head injury do not have persistent cognitive deficits following the injury. For more information regarding this fact, please see the following journal articles:

Macciocchi, S. N., Barth, J. T., Alves, W., Rimel, R. W., & Jane, J. A. (1996). Neuropsychological functioning and recovery after mild head injury in collegiate athletes, Neurosurgery, 39 (3), 510-514.

Thompson, M. D. & Irby, J. W. Jr. (2003). Recovery from mild head injury in pediatric populations. Seminars In Pediatric Neurology, 10 (2), 130-139.

Binder, L. M., Rohling, M. L.,& Larrabee, G. J. (1997). A review of mild head trauma. Part I: Meta-analytic review of neuropsychological studies. Journal of Clinical and Experimental Neuropsychology, 19 (3), 367-377

Given your high level of functioning (attending college), it is unlikely that you have any persistent cognitive deficits related to mild head injury. However, if you have additional questions, please ask your test administrator and/or contact Dr. Julie Suhr (contact information below).

As many other people will be participating in this study during the quarter, we ask that you please do not share details about this study with anyone else, so that people remain unaware of the specific details of the study given above.

If you have additional questions or concerns regarding your participation, please contact one of the co-investigators below.

Graduate Researcher: Caitlin Reese, M.A., M.S. 044H Porter Hall (419) 509-1742 [email protected].

Faculty Advisor: Julie Suhr, Ph.D. 250 Porter Hall (740)593-1091 [email protected]

In addition, if you are concerned about the study materials used or questions asked and wish to speak with a professional, or if you would like more information or reading material on this topic, please contact one of the following resources:

Ohio University Counseling and Psychological Services: 593-1616

Ohio University Psychology and Social Work Clinic: 593-1092

If you have any questions regarding your rights as a research participant, please contact Jo Ellen Sherow, Director of Research Compliance, Ohio University, (740)593-0664.

Appendix B: Score Ranges

AGT Classification Accuracy Scores

H.I. Controls S.M. Memory Impaired % Accuracy Cum. %age % Accuracy Cum. %age % Accuracy Cum. %age 44 4.2 34 2.9 50 25 47 6.3 43 5.9 54 37.5 49 10.4 44 8.8 56 50 50 12.5 46 14.7 57 75 51 14.6 47 17.6 63 87.5 52 18.8 48 20.6 71 100 53 25 49 23.5 54 29.2 50 26.5 55 33.3 51 29.4 56 39.6 53 35.3 57 43.8 54 38.2 58 50 55 50 59 56.3 56 55.9 60 60.4 57 58.8 62 66.7 58 64.7 63 72.9 60 67.6 64 79.2 61 73.5 65 83.3 62 82.4 67 87.5 63 85.3 68 91.7 64 88.2 69 93.8 66 94.1 71 95.8 67 97.1 72 97.9 68 100 74 100

AGT Learning Phase Trial Scores

H.I. Controls S.M. Memory Impaired Total Trials Cum. %age Total Trials Cum. %age Total Trials Cum. %age 40 6.3 42 2.9 40 12.5 41 14.6 43 5.9 41 25 42 27.1 44 11.8 42 37.5 43 35.4 48 14.7 47 50 44 41.7 50 23.5 48 62.5 45 56.3 51 32.4 59 75 46 62.5 54 35.3 64 87.5 47 66.7 55 38.2 96 100 48 72.9 61 41.2 49 75 63 44.1 50 81.3 67 47.1 51 89.6 68 50 52 93.8 69 52.9 57 95.8 71 55.9 58 97.9 73 58.8 79 100 75 61.8 77 64.7 79 67.6 81 70.6 85 73.5 87 76.5 88 82.4 93 85.3 106 88.2 120 91.2 122 94.1 130 97.1 157 100

Appendix C: Additional Psychometrics

Additional Psychometrics for Measures Administered to Undergraduate Groups

Green’s Word Memory Test. Green’s Word Memory Test (WMT; Green, Allen,

& Astner, 1996) is a well validated measure of noncredible effort/malingering utilizing a forced-choice recognition memory format. A list of 20 word pairs with strong semantic associations (e.g., dog-cat) is presented twice. Learning trials are followed by an immediate forced-choice recognition trial of each of the 40 words in the 20 word pairs. In the recognition trial, each target word is paired with a word with much lower association

(e.g., dog-rat). With no advance warning, a delayed recognition trial, in which targets are paired with new foils (e.g., dog- cow), is given after 30 minutes. The immediate and delayed recognition scores, together with the consistency of responding between the two recognition trials, constitute the primary measures of effort. When any of these scores fall below the recommended cutoff scores, the participant’s effort is considered suspect

(Green et al., 1996). In general, the Word Memory Test has been found to be more sensitive than other forced-choice measures, and like most forced-choice measures, it boasts good specificity (Gervais, Russell, Green, Allen, Ferrari, & Pieschl, 2004).

Wechsler Adult Intelligence Scale – IV digit span subtest. The Wechsler

Adult Intelligence Scale – IV Digit Span subtest (WAIS-IV; Wechsler, 2008) requires participants to repeat digits in forward order, reverse order, and sequential order.

Forward Digit Span assesses attention while reverse and sequential Digit Span assesses working memory. Digit Span’s internal consistency reliability coefficient is excellent r

=.93. Subscales’ internal consistency is: Digit Span Forward r =.81, Digit Span

Backward r =.82, and Digit Span Sequencing r =.83. Regarding convergent validity,

Digit Span is highly correlated with other working memory subtests (Arithmetic r =.60;

Letter Number Sequencing r =.69). The Digit Span subtest also correlates with the

Repeatable Battery for the Assessment of Neuropsychological Status (Randolph, 2012)

Attention index (r =.65). Embedded within the Digit Span subtest is the Reliable Digit

Span (RDS; Greiffenstein, Baker, & Gola, 1994) effort index utilizing the Forward and

Backward subscales. Jasinski and colleagues (2011) meta-analyzed 9 studies utilizing known patient groups and found an RDS cut score of 7.1 resulted in malingering sensitivity of 63.3% and specificity of 86.1%. With the published addition of the

Sequencing subscale to the Digit Span subtest (Wechsler, 2008), an enhanced/revised

Reliable Digit Span index’s cut-score of ≤ 11 (Reese, Suhr, & Riddle, 2012; Young et al.,

2012) was found to enhance RDS’s malingering detection utility.

Minnesota Multiphasic Personality Inventory 2 – Restructured Format

Response Bias Scale. The Minnesota Multiphasic Personality Inventory 2 – Restructured

Format (MMPI-2-RF; Ben-Porath & Tellegen, 2008) is a self-report personality inventory that assesses major symptoms of psychopathology, personality characteristics, and behavioral proclivities. The extensive psychometric findings reported in the MMPI-

2-RF technical manual (Ben-Porath & Tellegen, 2008) demonstrate that the MMPI-2-

RF’s 50 scales provide reliable and valid measures of protocol validity, personality, and psychopathology. For example, the Validity and substantive (Higher-Order;

Somatic/Cognitive and Internalizing; Externalizing, Interpersonal, and Interest; and

Personality Psychopathology Five) subscales show strong internal consistency and test- retest reliability across a number of populations (e.g., healthy controls, psychiatric inpatients [community and VA hospitals] and outpatients [community mental health centers]) and with stability over time (Ben-Porath & Tellegen, 2008). One validity scale of particular interest is the Response Bias Scale (RBS), which was designed specifically to identify negative response bias in forensic evaluations. RBS’s items were selected on the basis of correlations with performance on performance-based symptom validity measures in a sample of disability claimants and personal injury litigants. For example,

RBS has been shown to distinguish those who passed or failed the WMT (Green, Allen,

& Astner, 1996) in a sample of 1,151 non-head-injured disability claimants (Ben-Porath

& Tellegen, 2008). RBS has also been deemed a sensitive, incrementally valid (compared to the MMPI-2 validity scales) predictor of over-reported memory complaints (Gervais,

Ben-Porath, Wygant, & Sellbom, 2010).

Rey Auditory Verbal Learning Test. The Auditory Verbal Learning Test

(AVLT; Rey, 1964) measures list learning, immediate memory, delayed recall, and susceptibility to interference. In this test, participants listen to 15 words over 5 trials at the rate of 1 word per second, and after each trial are asked to recall as many words as they can remember. Participants are then presented with a second 15-word list and are asked to recall these words. This acts as an interference trial. Following this, participants are asked to recall as many words from the first list as they can remember. The score they receive for this performance reflects immediate memory. There is also a 30-minute, and

60-minute, delayed recall and recognition component to this test, from which four scores

78 are determined. However, during this study, only the 30-minute recall and recognition trial was administered. When the whole measure is administered, test-retest reliability over one year intervals has been shown to be marginal to adequate. Regarding concurrent validity, the AVLT loads with other verbal memory and general verbal ability measures.

It does not load with motor or visuospatial measures, and it strongly correlates with other verbal memory measures. It has been shown to discriminate between normal and mixed neurological groups, and different patient groups perform differently on the test

(Schmidt, 1996).

Additional Psychometrics for Measures Administered to Memory-Impaired

Repeatable Battery for the Assessment of Neuropsychological Status memory subscales. The Repeatable Battery for the Assessment of Neuropsychological Status

(RBANS; Randolph, 2012) is a comprehensive, yet relatively brief neuropsychological screening measure designed to assess for dementia or mild neurocognitive disorders, with adult norms ranging from the ages of 20 and 89. This measure’s 12 subtests evaluate immediate and delayed memory, language, attention, and visuospatial/constructional abilities (Randolph, 2012). The measure has evidenced good sensitivity and specificity for the detection of dementia, even in its early stages (Hobson, Hall, Humphreys-Clark,

Schrismsher, & O’Bryant, 2010). With regard to the memory subscales specifically,

RBANS Delayed Memory Index correlates well with other measures of explicit memory, including the Wechsler Memory Scales-Revised (Wechsler, 1987) Verbal Memory Index

(r =.69), Delayed Recall Index (r =.49), Logical Memory – percent retained (r =.34), and

Visual Reproduction – percent retained (r =.38), and has been shown to adequately

79 discriminate patients with documented memory impairment from healthy controls and patients with other (non-memory) neuropsychological impairments (Randolph, 2012).

Semi-structured interview and driving history questionnaire. A semi- structured interview was administered to collect basic demographic information. Based on information related to vision and eye health, participants were encouraged to wear their corrective lenses, as necessary. Within the older adult study, the Driving History

Questionnaire was administered to determine participants’ years of driving experience, frequency of recent driving (preceding 2 years) on various types of roads (e.g. highways, city streets, country roads), recent driving accidents or near-accidents, history of traffic violations, and any personal accommodations made for driving (e.g., not driving at night, driving at slower speeds).

Trail Making Test. The Trail Making Test (TMT; Reitan & Wolfson, 1985) is a two-part test (parts A and B). Part A presents a series of encircled numbers (1 through

25) printed randomly on a sheet of paper, in which the goal is to draw a line, sequentially from one circle to the next, until all the encircled numbers are connected. Part B consists of encircled numbers (1-13) and letters (A-L), where this time the goal is to execute mental flexibility by connecting the circles, alternating between numbers and letters, in order, rapidly, and without errors. Part A is considered a visual processing speed task

(Rios, Periáñez, & Muñoz-Céspedes, 2004) and has been shown to correlate most strongly with cognitive-perceptual tasks of speed and visual search (Crowe, 1998), while

Part B is a measure of executive functioning, requiring cognitive flexibility (Rios et al.,

2004). Test-retest reliability for older adults, following a one-year interval, ranges from

.53 to .64 for Part A and from .67 to .72 for Part B (Mitrushina & Satz, 1991). Inter-rater reliability ranges from .94 (Part A) to .90 (Part B; Fals-Stewart, 1992).

Delis-Kaplan Executive Function System – Word-Color Interference subtest.

The Word-Color Interference Test from the Delis-Kaplan Executive Function System (D-

KEFS; Delis, Kaplan, & Kramer, 2001) measures executive response inhibition. It is comprised of two basic naming conditions and an interference condition similar to the

Stroop (Stroop, 1935) measure. However, in addition, it also includes a set switching condition. The internal consistency reliability for this measure, for individuals 60 to 89 years old, ranges between r = 0.77 and r = 0.81 for the different conditions. The test- retest reliability, with a mean interval of 25 days between tests for individuals 50 to 89 years old, ranges between r = 0.50 and r = 0.57 depending on the condition (Delis,

Kaplan, & Kramer, 2001).

Block Design. The Block Design subtest of the WAIS-IV (Wechsler, 2008) is a test of visual perception and visuoconstructional skills, in which red and white colored blocks must be arranged to reproduce a one-dimensional design presented on a page. The score for each stimulus ranges from zero to four, dependent upon response time. Test- retest reliability of α = .80 (the average interval being 22 days) and internal consistency ranging from α = .80 to α = .89 have been reported for older adult samples (Wechsler,

2008). Correlations with other measures of visuospatial/constructional abilities have also been reported (Gro th-Marnat & Teal, 2000; Wechsler, 2008).

Wechsler Test of Adult Reading. The Wechsler Test of Adult Reading (WTAR;

Wechsler, 2001) is a measure of estimated premorbid intellectual ability, as determined

81 by performance on a reading recognition task in which individuals are asked to correctly read 50 words aloud. The WTAR is highly correlated with other measures of reading recognition, such as the National Adult Reading Test (r = .78), the Wechsler Individual

Achievement Test Basic Reading subtest (r = .75), the Wide Range Achievement Test –

Revised (r = .73), and the American National Adult Reading Test (.90). Also, the WTAR standard score can be utilized to estimate the Wechsler Adult Intelligence Scale—III full scale IQ in American adults between the ages of 55 and 89, with correlations ranging from .68 to .78, and within 10 points for over 70% of adult participants (Wechsler, 2001).

Internal consistency alpha coefficients have ranged from .92 and .97, while test-retest reliability (mean interval of 35 days) is high (r = .94). ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !

Thesis and Dissertation Services ! !