Within-Person Across-Neuropsychological Test Variability and Incident Dementia

PRELIMINARY COMMUNICATION

Roee Holtzer, PhD Context Neuropsychological tests are used to predict and diagnose dementia. How- Joe Verghese, MD ever, to our knowledge, no studies to date have examined whether within-person across- Cuiling Wang, PhD neuropsychological test variability predicts dementia. Objective To examine whether within-person across-neuropsychological test vari- Charles B. Hall, PhD ability predicts future dementia. Richard B. Lipton, MD Design The Einstein Aging Study (EAS) is a population-based longitudinal study of aging and dementia located in Bronx County, New York. We used Cox proportional EVELOPING STRATEGIES TO hazards models using age as the time scale to estimate hazard ratios (HRs) for perfor- improve the prediction mance on individual neuropsychological tests (Free and Cued Selective Reminding Test, and diagnosis of dementia Digit Symbol Substitution subtest of the Wechsler Adult Intelligence Scale Revised, has paramount therapeutic and the Vocabulary subtest of the Wechsler Adult Intelligence Scale Revised) and for Dand public health implications.1 Neu- within-person across-neuropsychological test variability as predictors of incident de- ropsychological tests are used to mentia. Analyses were stratified by sex, and controlled for education and medical ill- evaluate the diagnosis of dementia,2,3 ness. the transitional stages that precede it, Setting and Participants A total of 1797 participants (age Ն70 years) enrolled in such as mild cognitive impairment,4,5 the EAS between October 1993 and December 2007. Participants seen for the base- and cognitive changes over time.6 line visit only (n=750), prevalent dementia cases (n=72), and those with missing fol- Level of performance on tests of low-up information (n=78) were excluded. A total of 897 individuals were included memory7,8 and executive function9,10 in this investigation. Participants had follow-up visits every 12 to 18 months. has been reported to predict future Main Outcome Measure Incident dementia. dementia, and memory declines Results Sixty-one cases of incident dementia were identified during follow-up (mean more rapidly up to 7 years prior to [SD], 3.3 [2.4] years), of which 26 were in the highest quartile of within-person across- diagnosis of dementia.11,12 neuropsychological test variability. Adjusting for sex, education, and medical illness, vari- When neuropsychological tests are ability was associated with incident dementia (HR for 1-point difference in variability, used for diagnostic purposes, an 3.93 [95% confidence interval {CI}, 2.04-7.56]). The association persisted even after individual’s level of performance on adjusting for level of performance on individual neuropsychological tests (HR for 1-point difference in variability, 2.10 [95% CI, 1.04-4.23]). Comparing Cox models using neu- specific tests is measured against ropsychological tests with and without within-person across-neuropsychological test vari- healthy normative samples to deter- ablity showed that the former improved the prediction of dementia. Sensitivity in a model mine cognitive impairment.13,14 How- predicting dementia at 1 year also improved when neuropsychological test variability ever, this approach does not take was included. into account intraindividual variabil- Conclusions In this population, within-person across-neuropsychological test vari- ity in cognitive function. The tax- ability was associated with development of incident dementia independent of neuro- onomy of intraindividual variability, psychological test performance. This finding needs to be confirmed in future studies. considered as inconsistency in cogni- JAMA. 2008;300(7):823-830 www.jama.com tive performance within a person, includes the following definitions: task administered on 1 occasion or Author Affiliations: Departments of Neurology (Drs Holtzer, Verghese, and Lipton) and Epidemiology and (1) variability on the same cognitive over short periods (cross-sectional Population Health (Drs Wang, Hall, and Lipton), Ferkauf task across multiple assessments over studies show that this form of vari- Graduate School of Psychology (Dr Holtzer), and Insti- 16 tute for Aging Research (Dr Lipton), Albert Einstein Col- a long period (such variability may ability is increased in aging and lege of Medicine, Yeshiva University, Bronx, New York. be high in healthy individuals across dementia,17,18 but has shown variable Corresponding Author: Roee Holtzer, PhD, Albert Ein- 15 stein College of Medicine, Yeshiva University, Rousso the life span ); (2) variability on results in individuals with mild Bldg, Room 306, 1165 Morris Park Ave, Bronx, NY repeated trials of a single cognitive cognitive impairment19,20); and (3) 10461 ([email protected]).

Downloaded From: https://jamanetwork.com/ on 10/01/2021 INCIDENT DEMENTIA AND NEUROPSYCHOLOGICAL TEST VARIABILITY

variability in performance across care-eligible individuals were first con- view at baseline and the in-person neuropsychological tests adminis- tacted by letter, then by telephone, interviews in the EAS, but the scores tered in a single session (such vari- explaining the purpose of the study. The are not used when assigning dementia ability is expected in the normal telephone interview included verbal diagnosis in consensus case confer- population,21,22 including older consent, a brief medical history ques- ences.24) Second, impairments on adults without dementia,23 reflecting tionnaire, and telephone-based cogni- basic activities on the Lawton-Brody the individual’s relative cognitive tive screening tests.24 Following the in- activities of daily living,29 secondary to strengths and weaknesses). This terview, an age-stratified sample of cognitive impairment, had to be docu- study focused on within-person individuals who matched on a com- mented using previously described across-neuropsychological test vari- puterized randomization procedure was procedures.30 Incident dementia using ability because it can be estimated invited for further evaluation at the the above algorithmic procedures was directly from standardized clinical medical center. This procedure was used as the main outcome measure in neuropsychological procedures. implemented to ensure that the indi- the primary analyses reported herein. Increased within-person across- viduals included in the EAS repre- Agreement between the algorithmic neuropsychological test variability has sented the Bronx population at that age and clinical diagnosis of dementia was been related to poor cognitive func- stratum, and that those who agreed to 96% in this cohort. tion in normal older adults in cross- participate were not different from non- sectional studies.23 Accordingly, we responders in terms of key demo- Clinical Diagnosis of Dementia examined whether increased within- graphic characteristics. Written in- The EAS participants suspected to have person across-neuropsychological test formed consent was obtained at clinic dementia received a complete diagnos- variability was associated with future visits according to study protocols and tic workup as previously described for dementia after controlling for level of approved by the Committee on Clini- this cohort.31 Diagnoses of dementia performance on individual neuropsy- cal Investigation (institutional review were assigned according to the crite- chological tests. We also assessed board of the Albert Einstein College of ria of the Diagnostic and Statistical whether variability improved the sen- Medicine). Manual of Mental Disorders, Fourth Edi- sitivity and specificity of a model pre- A total of 1797 participants were en- tion,32 at case conferences attended by dicting development of dementia 1 year rolled in the EAS between October 1993 at least 1 study neurologist, a neuro- later. and December 2007. Participants seen psychologist, and a geriatric nurse or for the baseline visit only (n=750), social worker. Alzheimer disease was METHODS prevalent dementia cases (n=72), and diagnosed according to the criteria for Study Population those with missing information (n=78) probable disease detailed by the Na- Participants were enrolled in the Ein- were excluded. Hence, a total of 897 tional Institute of Neurological and stein Aging Study (EAS), a longitudi- participants were included in this Communicative Disorders and Stroke nal study of aging and dementia lo- investigation. Participants had fol- and the Alzheimer Disease and Re- cated at the Albert Einstein College of low-up visits every 12 to 18 months, at lated Disorders Association.3 The State Medicine in Bronx County, New York. which they underwent detailed neuro- of California Alzheimer Disease Diag- The study design, recruitment, and fol- logical and neuropsychological evalu- nostic and Treatment Centers criteria low-up methods have been previously ations. was used to assign diagnoses of prob- described.24,25 Briefly, the EAS used tele- able, possible, or mixed-vascular de- phone-based screening procedures to Algorithmic Diagnosis of Dementia mentia.33 In individuals diagnosed with recruit and follow-up a community- To address the issue of diagnostic cir- dementia, neuroimaging was used to based cohort since 1993. The primary cularity, we derived an algorithmic help allocate the diagnosis of probable aim of the EAS was to identify risk fac- diagnosis of dementia that was inde- Alzheimer disease or probable vascu- tors for dementia. Eligibility criteria re- pendent of the neuropsychological test lar dementia. We have reported good quired that participants be aged 70 years performance. Two conditions had to agreement between clinical diagnoses or older, reside in the Bronx, and speak be met. First, participants had to make of Alzheimer disease,34 vascular demen- English. Exclusion criteria included se- 8 or more errors on the Blessed Infor- tia,31 and dementia with Lewy bod- vere audiovisual disturbances that mation Memory Concentration test ies,35 and pathological findings in our would interfere with completion of neu- (BIMC; best score: 0 errors and worst study. For the purpose of this analy- ropsychological tests, inability to am- possible score: 32 errors).26 This test sis, clinical diagnosis of dementia sub- bulate even with a walking aid or in a has high test-retest reliability (0.86) types was used only in secondary analy- wheelchair, and institutionalization. Po- and correlates well with the pathology ses to explore whether associations with tential participants older than 70 years of Alzheimer disease.27,28 (The BIMC variability varied as a function of dis- from the Centers for Medicare & Med- test is used to screen for cognitive eases processes. Hence, only pure sub- icaid Services population lists of Medi- impairment on the telephone inter- types (Alzheimer and vascular) but not

Downloaded From: https://jamanetwork.com/ on 10/01/2021 INCIDENT DEMENTIA AND NEUROPSYCHOLOGICAL TEST VARIABILITY

mixed were considered in the explor- K=3 (FCSRT, WAIS-R Vocabulary, censored at the time of their last study atory analyses. and WAIS-R Digit Symbol Substitu- visit when their status as not having de- tion) and mentia was determined. Neuropsychological Assessment To assess the amount of variability K The tests included in this battery have = of development of incident dementia Ai Zik been validated for use in the aging popu- k = 1 explained by the models, we com- lation in our and other aging stud- pared the goodness of fit of nested Cox ies.8,36,37 Our recent studies reveal that is the individual’s mean z-trans- proportional hazards models using the factor analysis of the neuropsychologi- formed score based on the 3 tests. partial likelihood ratio test.44 We com- cal test battery consistently yielded 3 em- pared a model with the 3 individual pirically derived and statistically or- Illness Index neuropsychological tests with a model thogonal cognitive domains including Consistent with our previous stud- that added within-person across- verbal IQ, attention/executive func- ies,38,39,43 dichotomous rating (present neuropsychological test variability. Fi- tion, and memory.38,39 For the purpose or absent) of diabetes, chronic heart fail- nally, using the McNemar test we of this study, we identified 3 tests that ure, arthritis, hypertension, depres- examined whether within-person represented the 3 cognitive domains sion, stroke, Parkinson disease, chronic across-neuropsychological test vari- mentioned above, and were available for obstructive pulmonary disease, an- ability significantly improved sensitiv- all 897 participants included in this gina, and myocardial infarction was ity for the prediction of new cases of de- study. The Vocabulary (total score) sub- used to calculate an illness index sum- mentia within 1 year. test of the Wechsler Adult Intelligence mary score (range, 0-10). Medical his- We used age as the time scale be- Scale Revised (WAIS-R40), considered a tory was obtained from multiple sources cause it is considered more appropri- hold test in that it is not sensitive to the including significant others and fam- ate than follow-up time in cohort stud- effect of aging and age-related diseases, ily physicians when available. The study ies.45 When age serves as the time scale, represented the verbal IQ domain. The physician obtained the medical his- the hazard function can be directly in- Digit Symbol Substitution subtest of the tory during the neurological examina- terpreted as the age-specific incidence WAIS-R40 is commonly used to assess at- tion and trained research assistants con- function and age is accounted for in the tention and executive function. The Free ducted the structured clinical interview. nonparametric term of the hazard func- and Cued Selective Reminding Test (FC- Smoking was ascertained but did not tion, providing a more flexible and ef- SRT; free recall) is commonly used to as- contribute to variability in incident de- fective control of age.45 Time to event sess verbal memory and is sensitive to mentia in this study and was not in- was from age at the baseline assess- dementia.41 cluded in the analyses reported herein. ment accounting for the left trunca- tion at study inclusion to age at which Estimate of Within-Person Statistical Analyses diagnosis of dementia was ascertained Variability Across We performed 3 Cox proportional haz- or to final study contact for partici- Neuropsychological Tests ards regression analyses to assess the pants without dementia. In addition, all The method used herein to estimate relationship between neuropsychologi- multivariate analyses were stratified by within-person across-test variability was cal tests and dementia. First, we exam- sex and controlled for education and described in other studies as well.23,42 ined whether level of performance on medical illness. For all primary analy- Assumptions of normality were met for the neuropsychological tests at base- ses, incident dementia was defined the distribution of scores on the FCSRT, line predicted the development of using the algorithmic procedures de- WAIS-R Vocabulary, and WAIS-R Digit dementia. Second, we examined the as- scribed earlier. Proportional hazards as- Symbol Substitution tests. The raw sociation between within-person across- sumptions of the models were exam- scores of each test were z transformed neuropsychological test variability and ined analytically and graphically and on the basis of the distribution of the the development of dementia. Third, we were met. entire sample (n=897). Then, the z- assessed whether within-person across- Finally, Cox proportional hazards transformed test scores were used to neuropsychological variability pro- regression analysis with age as the calculate within-person variability vided incremental prediction of inci- time scale was used to examine across the 3 tests using this equation. dent dementia above and beyond what whether within-person across- Variability= was predicted by the absolute level of neuropsychological test variability performance on the neuropsychologi- predicted incident dementia subtypes (Z − A )2 cal tests by controlling for perfor- (eg, vascular and Alzheimer) defined ik i , (K − 1) mance differences on the WAIS-R Vo- by consensus diagnosis as previously cabulary, FCSRT, and WAIS-R Digit described. Given the low number of where Zik is the kth cognitive test score Symbol Substitution. Individuals who incident cases of Alzheimer dementia for ith individual. Here, k=1,...,K, died before developing dementia were and vascular dementia, as well as

Downloaded From: https://jamanetwork.com/ on 10/01/2021 INCIDENT DEMENTIA AND NEUROPSYCHOLOGICAL TEST VARIABILITY

concerns for potential diagnostic cir- participants, there were 61 cases of in- and were adjusted for sex, education, cularity, these analyses were consid- cident dementia (6.8%) (defined by the and medical illness. In the first model, ered exploratory. All analyses were algorithmic procedure described ear- higher scores on the FCSRT (hazard ra- performed using SAS statistical soft- lier) identified during the follow-up pe- tio [HR] for 1-point difference on the ware version 9.1 (SAS Institute Inc, riod (mean [SD], 3.3 [2.4] years). On test, 0.87 [95% confidence interval {CI}, Cary, North Carolina) and S-Plus the basis of the consensus clinical 0.84-0.91]; PϽ.001) and WAIS-R Digit version 8.0 (Insightful Corp, Seattle, diagnostic procedures, 47 partici- Symbol Substitution (HR for 1-point Washington). The significance level pants developed incident dementia of difference on the test, 0.97 [95% CI, was set at .05 and all tests were the Alzheimer type and 18 partici- 0.94-1.00]; P=.04) were associated with 2-sided. pants developed incident vascular de- a lower risk of developing dementia. mentia. During the study, 128 indi- Level of performance on the WAIS-R RESULTS viduals died, as expected for the age of Vocabulary test was not associated with Demographics this cohort. Of these, 18 had devel- future dementia (HR for 1-point dif- Participants were community resi- oped incident dementia. ference on the test, 1.01 [95% CI, 0.98- dents who were relatively healthy and 1.03]; P=.67). had normal global mental functioning Prediction of Dementia The second Cox proportional haz- based on BIMC scores that were within All Cox proportional hazards regres- ards regression analysis also used age the normal range (TABLE). Of the 897 sion models used age as the time scale as the time scale and adjusted for sex, education, and medical illness. Within- person across-neuropsychological test Table. Summary of Participant Characteristics per Quartile of Within-Person Across-Neuropsychological Test Variability at Baseline variability was significantly associated Variability by Quartilea with incident dementia defined algo- Entire rithmically (HR for 1-point difference Sample 1 2 3 4 in variability, 3.93 [95% CI, 2.04- Characteristic (N = 897) (n = 225) (n = 224) (n = 224) (n = 224) 7.56]; PϽ.001). Risk of dementia in- Mean (SD) Age, y 78.6 (5.3) 78.3 (5.1) 78.8 (5.6) 78.4 (5.2) 79 (5.3) creased as a function of greater within- Education, y 13.3 (3.6) 13.6 (3.2) 13.4 (3.3) 13.2 (3.8) 12.9 (3.9) person variability. Blessed Information Memory 2.6 (2.5) 2.2 (2.1) 2.5 (2.5) 2.4 (2.2) 3.5 (2.9) The third Cox proportional hazards Concentration Testb regression analysis controlled for each Free and Cued Selective 30.0 (6.8) 30.8 (4.9) 30.9 (5.7) 30.4 (5.9) 27.8 (9.3) c of the neuropsychological test scores to Reminding Test determine the independent associa- Vocabulary subtest 46.9 (13.9) 49.9 (10.9) 48.8 (11.7) 47.4 (14.2) 42.6 (17.3) of WAIS-Rd tion of variability with incident demen- Digit Symbol Substitution 34.5 (11.4) 35.9 (8.6) 35.7 (10.5) 33.3 (11.7) 32.9 (14.1) tia. The model used age as the time scale subtest of WAIS-Re and adjusted for sex, education, and Variability 0.74 (0.40) 0.29 (0.11) 0.56 (0.07) 0.82 (0.09) 1.29 (0.26) medical illness. Similar to the previ- Illness index 1.2 (1.0) 1.2 (1.0) 1.1 (1.0) 1.2 (1.0) 1.1 (1.1) ous analysis, the results revealed that No. (%) higher scores on the FCSRT (HR for Women 538 (60) 134 (60) 146 (65) 129 (58) 129 (58) 1-point difference on the test, 0.89 [95% Incident dementia cases 61 (7) 3 (1) 19 (8) 13 (6) 26 (12) CI, 0.85-0.92]; PϽ.001) and WAIS-R Depression 83 (9) 28 (12) 23 (10) 14 (6) 18 (8) Digit Symbol Substitution (HR for Angina 90 (10) 22 (10) 22 (10) 25 (11) 21 (9) 1-point difference on the test, 0.97 [95% Arthritis 51 (6) 13 (6) 12 (5) 15 (7) 11 (5) CI, 0.94-1.00]; P=.04) were associ- Diabetes 120 (13) 29 (13) 24 (11) 31 (14) 36 (16) ated with lower risk of incident demen- Chronic obstructive 35 (4) 11 (5) 9 (4) 8 (4) 7 (3) tia, and the WAIS-R Vocabulary test was pulmonary disease not associated with it (HR for 1-point Hypertension 464 (52) 114 (51) 117 (52) 121 (54) 112 (50) difference on the test, 1.00 [95% CI, Myocardial infarction 90 (10) 25 (11) 18 (8) 26 (12) 21 (9) 0.98-1.03]; P=.81). Importantly, even Chronic heart failure 12 (1) 2 (1) 4 (2) 4 (2) 2 (1) after controlling for individual neuro- Parkinson disease 12 (1) 4 (2) 2 (1) 2 (1) 4 (2) psychological test scores, within- Stroke 86 (10) 22 (10) 17 (8) 25 (11) 22 (10) person across-neuropsychological test Abbreviation: WAIS-R, Wechsler Adult Intelligence Scale Revised. variability remained significantly asso- a Variability denotes within-person across-neuropsychological test variability. Increased variability denotes worse cognitive function. ciated with incident dementia (HR for b Possible score range is 0 to 32. c Possible score range is 0 to 48. 1-point difference in variability, 2.10 d Possible score range is 0 to 70. [95% CI, 1.04-4.23]; P=.03). The HR e Possible score range is 0 to 93. of variability was lower after adjusting

Downloaded From: https://jamanetwork.com/ on 10/01/2021 INCIDENT DEMENTIA AND NEUROPSYCHOLOGICAL TEST VARIABILITY

for performance level on the indi- dementia using age as the time scale. this construct.47 To our knowledge, this vidual tests, as expected, but the cor- Within-person across-neuropsycho- is the first study to demonstrate that relations between variability and per- logical test variability dichotomized into within-person across-neuropsychologi- formance on the neuropsychological the highest (ie, worst) quartile vs the cal test variability is associated with in- tests were relatively low (FCSRT, rest revealed that (consistent with the cident dementia in a population- r=−0.21, PϽ.001; WAIS-R Digit Sym- previous results) higher variability based cohort aged 70 years or older, bol Substitution, r=−0.12, PϽ.001; was associated with increased risk of in- even after adjusting for level of perfor- WAIS-R Vocabulary, r =−0.18, cident dementia (HR, 2.25; 95% CI, mance on each individual neuropsy- PϽ.001), suggesting that these mea- 1.32-3.86) (FIGURE, A). Additionally, chological test. sures were not collinear. The lower, but Kaplan-Meier curves for the FCSRT The premise guiding the study was significant HR for within-person across- (HR, 4.77; 95% CI, 2.80-8.10), WAIS-R that variability or inconsistency in per- neuropsychological test variability in Digit Symbol Substitution (HR, 2.43; formance across neuropsychological the third compared with the second 95% CI, 1.44-4.09), and WAIS-R Vo- tests would increase in the preclinical Cox analysis, along with HRs for level cabulary tests (HR, 1.51; 95% CI, 0.88- stages of dementia compared with nor- of performance on the individual neu- 2.57) dichotomized into the lowest (ie, mal aging. This, in part, is attributed to ropsychological tests that were simi- worst) quartile vs the rest are illus- the decline in memory function years lar in the 2 analyses, are evidence that trated in the Figure, B-D, respectively. before the diagnosis of dementia.11 As part, but not all, of the effect of vari- opposed to summary scores or intra- ability is mediated through changes in Exploratory Analyses individual indices that estimate func- levels of cognitive performance. Using Clinical Consensus Diagnosis tion within 1 cognitive domain, within- We used the partial likelihood ratio of Dementia Subtypes person across-neuropsychological test test44 to examine whether adding Cox proportional hazards regression variability may be conceptualized as a within-person across-neuropsychologi- analyses with age serving as the time single representation of variability cal test variability to neuropsychologi- scale, adjusting for sex, education, and across multiple domains subserved by cal test performance improved predic- illness index, were used to examine several cortical regions and networks. tion of dementia compared with using whether associations between variabil- In this context, this form of cognitive neuropsychological test performance ity and clinical diagnosis of dementia variability may be considered a signa- alone. The results were significant (log using consensus case conference pro- ture of decline in cerebral integrity in partial likelihood ratio with and with- cedures varied as a function of demen- the early stages of dementing ill- out variability, respectively, −200.5 and tia subtype. Variability predicted nesses. ␹2 −202.6; 1=4.19, P=.04), indicating that development of both incident demen- In choosing the individual tests prediction of incident dementia was im- tia of the Alzheimer type (n=47; HR, used to calculate within-person proved by adding within-person across- 3.63 [95% CI, 1.79-7.37]), and inci- across-neuropsychological test vari- neuropsychological test variability. We dent vascular dementia (n=18; HR, ability, we aimed to maximize the also examined whether within-person 5.26 [95% CI, 1.70-16.30]). The HR number of participants, represent dif- across-neuropsychological test vari- for Alzheimer disease after adjusting ferent cognitive domains, and increase ability improved the sensitivity for the for neuropsychological test results was generalizability by including a few prediction of dementia within 1 year. 1.68 (95% CI, 0.80-3.55; P=.17); these easily administered tests that are com- In a model that included the 3 indi- results are comparable with dementia monly used in other studies and by vidual neuropsychological tests and overall but power was limited to dem- clinicians assessing cognitive impair- controlled for age, sex, education, onstrate a significant difference. The ment in elderly individuals. Within- and disease illness, a score combining disease processes underlying pure sub- person across-neuropsychological test the coefficients of all covariates in types of Alzheimer dementia and vas- variability can be computed using the model that resulted in 80% speci- cular dementias are different. Thus, it tests other than those used in this ficity for detecting dementia yielded appears that variability is sensitive to study. Further, this form of cognitive 83% sensitivity for predicting demen- dementia irrespective of disease sub- variability can be estimated using tia within 1 year. Including within- type. standard and widely used clinical and person across-neuropsychological test neuropsychological assessment proce- variability in this model, maintaining COMMENT dures that are typically given in 1 test- specificity at 80%, significantly in- Our assessment of intraindividual vari- ing session. Hence, the potential clini- creased the sensitivity for predicting ability addresses within-person differ- cal utility of this aspect of cognitive dementia within 1 year to 88% (McNe- ences in cognitive function and its use function is quite appealing because it mar test, P=.01). as a marker of pathology.46 The tax- requires no changes to standard Kaplan-Meier curves were used to de- onomy of intraindividual variability of- assessment procedures in aging stud- lineate the cumulative risk of incident fers several operational definitions for ies or assessment of cognitive disorder

Downloaded From: https://jamanetwork.com/ on 10/01/2021 INCIDENT DEMENTIA AND NEUROPSYCHOLOGICAL TEST VARIABILITY

in clinic settings. It is noteworthy that frontal regions.51,52 However, at pre- variability may capture the summation variability on subdomains of the sent, little is known about within- of differing sensitivities of brain Mini-Mental State Examination48 was person across-cognitive test variability, regions and networks to various dis- related to short-term decline on a the theoretical and neuroanatomical ease processes as opposed to estimat- global measure of cognitive function basis for this putative construct. Our ing the effect of disease on a single in a small cohort of centenarians.42 findings suggest that even though brain region. The relatively low corre- Hence, examining across-cognitive substantial in older adults without lations between variability and the domain variability in screening mea- dementia,23 within-person across- level of performance on the individual sures, including those currently used neuropsychological test variability neuropsychological tests appear to to support the diagnosis of mild cog- may be pathological when a certain support this notion. nitive impairments,49 is of interest. threshold is exceeded. It is noteworthy It is of further interest to examine Research and theories concerning that variability was sensitive to both whether and the extent to which dif- within-person variability on single Alzheimer and vascular dementia sub- ferent aspects of intraindividual vari- measures, although relatively recent, types, which vary in terms of their eti- ability in cognitive functions are suggest that increased inconsistency ology and cognitive profile.33,53,54 related in terms of the theory, underly- represents impaired top-down execu- Therefore, as a signature of early ing mechanism, and utility in predict- tive control processes50 subserved by decline in global cerebral integrity, ing outcomes of interest. For instance,

Figure. Kaplan-Meier Curves for the Cumulative Risk of Incident Dementia

A Within-person across-neuropsychological test variability B Free and Cued Selective Reminding Test (FCSRT)

1.0 1.0

0.8 0.8

0.6 0.6 HR, 2.25 (95% CI, 1.32-3.86) HR, 4.77 (95% CI, 2.80-8.10) 0.4 0.4

Variability FCSRT score Dementia-Free Survival Dementia-Free 0.2 Quartiles 1-3 0.2 Quartiles 2-4 Quartile 4 (worst) Quartile 1 (worst)

0 0 70 75 80 85 90 95 100 70 75 80 85 90 95 100 Age, y Age, y No. at risk No. at risk Quartiles 1-3 3492 156 127 36 9 1 Quartiles 2-4 37106 159 121 36 27 1 Quartile 4 (worst) 1717 54 34 16 3 0 Quartile 1 (worst) 1616 45 40 19 3 0

C Digit Symbol Substitution (DSYM) subtest of WAIS-R D Vocabulary subtest of WAIS-R

1.0 1.0

0.8 0.8

0.6 0.6 HR, 2.43 (95% CI, 1.44-4.09) HR, 1.51 (95% CI, 0.88-2.57) 0.4 0.4

DSYM score Vocabulary score Dementia-Free Survival Dementia-Free 0.2 Quartiles 2-4 0.2 Quartiles 2-4 Quartile 1 (worst) Quartile 1 (worst)

0 0 70 75 80 85 90 95 100 70 75 80 85 90 95 100 Age, y Age, y No. at risk No. at risk Quartiles 2-4 3390 173 116 28 10 0 Quartiles 2-4 3195 160 120 37 6 1 Quartile 1 (worst) 1520 41 42 23 4 1 Quartile 1 (worst) 2828 42 36 16 4 0

Variability denotes within-person across-neuropsychological test variability. Increased variability denotes worse cognitive function. CI indicates confidence interval; HR, hazard ratio; WAIS-R, Wechsler Adult Intelligence Scale Revised.

Downloaded From: https://jamanetwork.com/ on 10/01/2021 INCIDENT DEMENTIA AND NEUROPSYCHOLOGICAL TEST VARIABILITY

research suggests that individuals are age, sex distribution, and level of edu- REFERENCES not consistently diagnosed with mild cation (data available from author on 1. DeCarli C. Mild cognitive impairment: preva- cognitive impairment across multiple request). Nonetheless, the potential for lence, prognosis, aetiology, and treatment. Lancet 55,56 Neurol. 2003;2(1):15-21. sessions. Although some fluctua- selection bias should be considered. We 2. American Psychiatric Association. Diagnostic and tions may be attributed to error in emphasize that a single variability mea- Statistical Manual of Mental Disorders. 4th ed, text revision. Washington, DC: American Psychiatric As- measurement and limitations of this sure cannot replace a complete neuro- sociation; 2000. construct, it would be of interest to psychological examination, nor do we 3. McKhann G, Drachman D, Folstein M, Katzman examine the relationship between advocate that 3 neuropsychological tests R, Price D, Stadlan EM. Clinical diagnosis of Alzhei- mer’s disease: report of the NINCDS-ADRDA Work across-testing and within-testing ses- are sufficient to provide adequate as- Group under the auspices of Department of Health sion variability and whether the latter sessment of an individual’s cognitive and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34(7):939-944. improves understanding and predic- function. Instead, we propose that 4. Petersen RC, Doody R, Kurz A, et al. Current con- tion of transitional stages in cognitive measures of within-person across-test cepts in mild cognitive impairment. Arch Neurol. 2001; 58(12):1985-1992. aging. variability be viewed as complemen- 5. Petersen RC, Smith GE, Waring SC, Ivnik RJ, The limitations of the study should tary to standard assessment proce- Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. be considered. First, the sample, al- dures that are used to predict future 1999;56(3):303-308. though representative of the Bronx, risk of dementia. Finally, our results 6. Potter GG, Attix DK. An integrated model for geri- consisted of volunteers who resided in were identified in a single population; atric neuropsychological assessment. In: Attix DK, Welsh-Bohmer KA, eds. Geriatric Neuropsychology: the community and who were rela- these results should be validated in a Assessment and Intervention. New York, NY: Guil- tively healthy and willing to travel to new population and their value in dis- ford Publications; 2006:5-26. 7. Grober E, Lipton RB, Hall C, Crystal H. Memory im- the medical center. Hence, the gener- criminating between populations that pairment on free and cued selective reminding pre- alizability of the findings may poten- will and will not develop dementia dicts dementia. Neurology. 2000;54(4):827-832. 8. Masur DM, Sliwinski M, Lipton RB, Blau AD, Crystal tially be limited. Second, the diagno- should be defined before intraperson HA. Neuropsychological prediction of dementia and sis of incident dementia was based on across-neuropsychological test vari- the absence of dementia in healthy elderly persons. an algorithmic procedure that was inability is used routinely in clinical set- Neurology. 1994;44(8):1427-1432. 9. Albert MS, Moss MB, Tanzi R, Jones K. Preclinical dependent of neuropsychological test tings. prediction of AD using neuropsychological tests. JInt scores and consensus diagnosis. Hence, In summary, within-person across- Neuropsychol Soc. 2001;7(5):631-639. 10. Bäckman L, Jones S, Berger AK, Laukka EJ, Small diagnostic circularity is less likely to neuropsychological test variability was BJ. Cognitive impairment in preclinical Alzheimer’s dis- confound the findings. However, to associated with development of demen- ease: a meta-analysis. Neuropsychology. 2005; 19(4):520-531. examine longitudinal associations tia independently of performance of the 11. Hall CB, Ying J, Kuo L, et al. Estimation of bivar- between this form of variability and neuropsychological tests. This finding iate measurements having different change points, with application to cognitive ageing. Stat Med. 2001; clinical diagnosis of dementia, within- needs to be replicated in different popu- 20(24):3695-3714. person across-neuropsychological test lations before it is applied in a clinical 12. Hall CB, Ying J, Kuo L, Lipton RB. Bayesian and variability ideally should be derived setting. profile likelihood change point methods for model- ing cognitive function over time. Computational Sta- from tests that are not part of the clini- tistics Data Analysis. 2003;42:91-109. cal diagnostic procedures. Third, as- Authors Contributions: Dr Holtzer had full access to 13. Lezak MD, Diane BH, Loring DW. Neuropsycho- all the data in the study and takes responsibility for logical Assessment. 4th ed. New York, NY: Oxford Uni- signment of subtypes of dementia is fal- the integrity of the data and the accuracy of the data versity Press; 2004. lible. Although the diagnoses were analysis. 14. Mitrushina M, Boone KB, Razani J, D’Elia L. Hand- Study concept and design: Holtzer, Verghese. book of Normative Data for Neuropsychological made according to standardized crite- Acquisition of data: Lipton, Verghese. Assessment. 2nd ed. New York, NY: Oxford Univer- ria, some misclassification is inevi- Analysis and interpretation of data: Holtzer, Verghese, sity Press; 2005. table. Further, differential associa- Hall, Wang, Lipton 15. Salthouse TA. Implications of within-person vari- Drafting of the manuscript: Holtzer, Verghese ability in cognitive and neuropsychological function- tions between variability and dementia Critical revision of the manuscript for important in- ing for the interpretation of change. Neuropsychology. subtypes may be demonstrated, per- tellectual content: Holtzer, Verghese, Hall, Wang, 2007;21(4):401-411. Lipton, 16. West R. The transient nature of executive con- haps when using neuropsychological Statistical analysis: Wang, Hall, Holtzer, Verghese. trol processes in younger and older adults. Eur J Cog- tests that are more sensitive to either Obtained funding: Lipton. nitive Psychol. 2001;13(1-2):91-105. 33,53,54 Administrative, technical, or material support: 17. Hultsch DF, MacDonald SW, Hunter MA, vascular or Alzheimer diseases. Verghese, Lipton. Levy-Bencheton J, Strauss E. Intraindividual variabil- Fourth, attrition is a major issue of con- Study supervision: Holtzer, Verghese ity in cognitive performance in older adults: compari- Financial Disclosures: None reported. son of adults with mild dementia, adults with arthri- cern in any longitudinal study. A sig- Funding/Support: The Einstein Aging Study is sup- tis, and healthy adults. Neuropsychology. 2000; nificant number of individuals en- ported by the National Institute on Aging program 14(4):588-598. rolled in the EAS were not eligible to project grant AGO3949. Dr Holtzer is supported by 18. Knotek PC, Bayles KA, Kaszniak AW. Response the National Institute on Aging Paul B. Beeson consistency on a semantic memory task in persons with participate in this investigation be- Award K23 AG030857. Dr Verghese is supported dementia of the Alzheimer type. Brain Lang. 1990; cause they had only the baseline evalu- by the National Institute on Aging grant 38(4):465-475. AG025119. 19. Christensen H, Dear KB, Anstey KJ, Parslow RA, ation and were awaiting the next yearly Role of the Sponsor: The National Institute on Aging Sachdev P, Jorm AF. Within-occasion intraindividual visit. However, the subsample in- did not have any role in the design and conduct of variability and preclinical diagnostic status: is intrain- the study; collection, management, analysis, and individual variability an indicator of mild cognitive cluded in this study was not different terpretation of the data; and preparation, review, or impairment? Neuropsychology. 2005;19(3):309- from the entire EAS cohort in terms of approval of the manuscript. 317.

Downloaded From: https://jamanetwork.com/ on 10/01/2021 INCIDENT DEMENTIA AND NEUROPSYCHOLOGICAL TEST VARIABILITY

20. Dixon RA, Garrett DD, Lentz TL, MacDonald SW, 32. American Psychiatric Association. Diagnostic 44. Silvey SD. Statistical Inference. London, En- Strauss E, Hultsch DF. Neurocognitive markers of cog- and Statistical Manual of Mental Disorders. 4th ed. gland: Chapman & Hall; 1975. nitive impairment: exploring the roles of speed and Washington, DC: American Psychiatric Association; 45. Thie´ baut AC, Benichou J. Choice of time-scale in inconsistency. Neuropsychology. 2007;21(3):381- 1994. Cox’s model analysis of epidemiologic cohort data: a 399. 33. Chui HC, Victoroff JI, Margolin D, Jagust W, simulation study. Stat Med. 2004;23(24):3803- 21. Matarazzo JD, Daniel MH, Prifitera A, Herman Shankle R, Katzman R. Criteria for the diagnosis 3820. DO. Inter-subtest scatter in the WAIS–R standardiza- of ischemic vascular dementia proposed by the state 46. Stuss DT, Murphy KJ, Binns MA, Alexander MP. tion sample. J Clin Psychol. 1988;44(6):940-950. of California Alzheimer’s disease diagnostic and treat- Staying on the job: the frontal lobes control indi- 22. Matarazzo JD, Prifitera A. Subtest scatter and pre- ment centers. Neurology. 1992;42(3 pt 1):473- vidual performance variability. Brain. 2003;126 morbid intelligence: Lessons from the WAIS–R stan- 480. (pt 11):2363-2380. dardization sample. Psychol Assess. 1989;1(3): 34. Crystal HA, Dickson D, Davies P, Masur D, Grober 47. Li SC, Huxhold O, Schmiedek F. Aging and at- 186-191. E, Lipton RB. The relative frequency of “dementia of tenuated processing robustness: evidence from cog- 23. Schretlen DJ, Munro CA, Anthony JC, Pearlson unknown etiology” increases with age and is nearly nitive and sensorimotor functioning. Gerontology. GD. Examining the range of normal intraindividual vari- 50% in nonagenarians. Arch Neurol. 2000;57(5): 2004;50(1):28-34. ability in neuropsychological test performance. JInt 713-719. 48. Folstein MF, Folstein SE, McHugh PR. “Mini- Neuropsychol Soc. 2003;9(6):864-870. 35. Verghese J, Crystal HA, Dickson DW, Lipton RB. mental state”: a practical method for grading the cog- 24. Lipton RB, Katz MJ, Kuslansky G, et al. Screen- Validity of clinical criteria for the diagnosis of demen- nitive state of patients for the clinician. J Psychiatr Res. ing for dementia by telephone using the memory im- tia with Lewy bodies. Neurology. 1999;53(9):1974- 1975;12(3):189-198. pairment screen. J Am Geriatr Soc. 2003;51(10): 1982. 49. Nasreddine ZS, Phillips NA, Bedirian V, et al. The 1382-1390. 36. Katzman R, Aronson M, Fuld P, et al. Develop- Montreal Cognitive Assessment, MoCA: a brief screen- 25. Verghese J, Katz MJ, Derby CA, Kuslansky G, Hall ment of dementing illnesses in an 80-year-old volun- ing tool for mild cognitive impairment. J Am Geriatr CB, Lipton RB. Reliability and validity of a telephone- teer cohort. Ann Neurol. 1989;25(4):317-324. Soc. 2005;53(4):695-699. based mobility assessment questionnaire. Age Ageing. 37. Sliwinski M, Buschke H, Stewart WF, Masur D, 50. West R, Murphy KJ, Armilio ML, Craik FI, Stuss 2004;33(6):628-632. Lipton RB. The effect of dementia risk factors on com- DT. Lapses of intention and performance variability 26. Blessed G, Tomlinson BE, Roth M. The associa- parative and diagnostic selective reminding norms. reveal age-related increases in fluctuations of execu- tion between quantitative measures of dementia and J Int Neuropsychol Soc. 1997;3(4):317-326. tive control. Brain Cogn. 2002;49(3):402-419. of senile change in the cerebral grey matter of elderly 38. Holtzer R, Friedman R, Lipton RB, Katz M, Xue 51. Bellgrove MA, Hester R, Garavan H. The func- subjects. Br J Psychiatry. 1968;114(512):797-811. X, Verghese J. The relationship between specific cog- tional neuroanatomical correlates of response vari- 27. Fuld P. Psychological testing in the differential di- nitive functions and falls in aging. Neuropsychology. ability: evidence from a response inhibition task. agnosis of the dementias. In: Katzman TR, Bick KL, 2007;21(5):540-548. Neuropsychologia. 2004;42(14):1910-1916. eds. Alzheimer’s Disease: Senile Dementia and Re- 39. Holtzer R, Verghese J, Xue X, Lipton RB. Cogni- 52. Stuss DT, Toth JP, Franchi D, Alexander MP, Tipper lated Disorders. Vol 7. New York, NY: Raven Press; tive processes related to gait velocity: results from the S, Craik FI. Dissociation of attentional processes in pa- 1978:185-193. Einstein Aging Study. Neuropsychology. 2006; tients with focal frontal and posterior lesions. 28. Grober E, Dickson D, Sliwinski MJ, et al. Memory 20(2):215-223. Neuropsychologia. 1999;37(9):1005-1027. and mental status correlates of modified Braak staging. 40. Wechsler D. Wechsler Adult Intelligence 53. Jokinen H, Kalska H, Mantyla R, et al. Cognitive Neurobiol Aging. 1999;20(6):573-579. Scale-Revised. New York, NY: Psychological Corp; profile of subcortical ischaemic vascular disease. J Neu- 29. Lawton MP, Brody EM. Assessment of older 1981. rol Neurosurg Psychiatry. 2006;77(1):28-33. people: self-maintaining and instrumental activities of 41. Grober E, Buschke H, Crystal H, Bang S, Dresner 54. McPherson SE, Cummings JL. Neuropsychologi- daily living. Gerontologist. 1969;9(3):179-186. R. Screening for dementia by memory testing. cal aspects of vascular dementia. Brain Cogn. 1996; 30. Sarazin M, Berr C, De Rotrou J, et al. Amnestic Neurology. 1988;38(6):900-903. 31(2):269-282. syndrome of the medial temporal type identifies pro- 42. Kliegel M, Sliwinski M. MMSE cross-domain vari- 55. Bischkopf J, Busse A, Angermeyer MC. Mild cog- dromal AD: a longitudinal study. Neurology. 2007; ability predicts cognitive decline in centenarians. nitive impairment—a review of prevalence, inci- 69(19):1859-1867. Gerontology. 2004;50(1):39-43. dence and outcome according to current approaches. 31. Verghese J, Lipton RB, Hall CB, Kuslansky G, Katz 43. Verghese J, Wang C, Lipton RB, Holtzer R, Xue Acta Psychiatr Scand. 2002;106(6):403-414. MJ, Buschke H. Abnormality of gait as a predictor of X. Quantitative gait dysfunction and risk of cognitive 56. Collie A, Maruff P, Currie J. Behavioral charac- non-Alzheimer’s dementia. N Engl J Med. 2002; decline and dementia. J Neurol Neurosurg Psychiatry. terization of mild cognitive impairment. J Clin Exp 347(22):1761-1768. 2007;78(9):929-935. Neuropsychol. 2002;24(6):720-733.

No being can be what he is unless he is putting his essence into action in his field. —Arnold J. Toynbee (1889-1975)

Downloaded From: https://jamanetwork.com/ on 10/01/2021