Principles and Applications of the Quantified Process Approach

Cleveland State University College of Sciences and Health Professions

Deconstructing Mental Processes: Principles and Applications of the Quantified Process Approach

Amir Poreh, PhD Professor, Department of Psychology Cleveland State University Clinical Associate Professor of Psychiatry Case Western Reserve University School of Medicine

www.csuohio.ed Historical Background • American Pragmatic Empiricist Approach – Samuel Kohs – Yerkes Point Scale Method – The Wechsler Tests • Dr. Safford Ackerly and the cases of PA and JP – Arthur Benton - Iowa-Benton Flexible Approach – Ward Halstead – The Fixed Battery Approach • The Migration of European Phenomenological Approach – Kurt Goldstein and Heinz Werner – Edith Kaplan and the Boston Process Approach – Baxter vs . Temple (2005) • The Rise of Corporate Tests • The Quantified Process Approach

American Pragmatic Empiricist Approach • Ellis Island Psychological Tests Samuel C. Kohs “success, speed and accuracy each has its own diagnostic importance and in order to make the tests most effective all should, and must, be taken into account in the final score summation”. …

Yerkes (1921) claimed that the use of multiple indexes “ introduce a number of difficulties in administration and scoring which may be desirably avoided”.

American Pragmatic Empiricist Approach Yerkes & Bridges Point Scale (1914)

• Empirical and a-theoretical • Consistent with the anti-localization tradition

Robert M. Yerkes The Case of PA Dr. Safford Ackerly 1933 • A 37-year old woman with a slowly growing tumor PA’s Psychological Assessment • Following bilateral frontal resection – Performance on intelligence and memory measures remained intact. – She had difficulties in analyzing visual spatial information – She “stopped showing respect for her husband’s authority”.

The Case of JP Safford Ackerly and Arthur Benton, 1948 The authors followed this patient since he was 19 y/o until he reached fifty. They observed that his life was “a semblance of normality.”

The Case of JP (continued)

• JP was tested by both Arthur Benton and Ward Halstead. – His performance on measures of general intelligence and memory remained intact. The Ball and Field The Porteus Maze Test Test (Terman 1911)

The two measures that were sensitive to JP’s deficits The Case of JP and the Emergence of the Modern American Neuropsychological Tradition

Arthur Benton Ward Halstead

Iowa-Benton Flexible Approach Fixed Battery Approach The H-R Fixed Battery Approach

• Criterion keying approach (27 tests trimmed to 13)*

*Heinz Werner (1949) based on clinical rather than actuarial judgment W.C Halstead (1947). Brain and intelligence: A quantitative study of the frontal lobes. University of Chicago Press, Chicago, IL The Rise and Fall of the H-R Approach

400

350

300

250

200

150

100 Number ofPublications Number 50

0 1930-1949 1950-59 1960-69 1970-79 1980-89 1990-99 2000-09 2010-2014

Year of Publication • TCN=AACN 2005 SALARY SURVEY The Case of JP and the Emergence of the Modern American Neuropsychological Tradition

Arthur Benton Ward Halstead

Iowa-Benton Fix Battery Flexible Approach Approach Iowa-Benton Flexible Approach Arthur Benton

• calls for the administration of a battery of tests that are tailored to the presenting disorder • developed various new point scale measures ( i.e., prosopagnosia and visual spatial abilities ) Facial Recognition Test Line Orientation Task The European Phenomenological Approach and Its Migration to the United States

Dr. Kurt Goldstein (1878-1965) Heinz Werner (1890-1964) A student of Carl Wernicke Professor, University of (Chief of Neurology at the Hamburg Berlin Hospital of Moabit)

Kurt Goldstein and Testing

“The usual scoring method based on a scale of difficulty which has been standardized on a statistical basis offers no adequate instrument for determining the nature of the degree of impairment in a patient. Unless one takes into account the entire procedure, the specific reasons for the difficulty the patient encounters, one cannot simply read off from a score which task represents a greater difficulty and which a lesser. Any statistical evaluation has to be based upon a qualitative analysis of test results.” Goldstein, K & Scheerer, M, 1941, page 19 Kurt Goldstein’s Tests

Goldstein -Scheerer Tests “[Americans] wondered suspiciously about his (Goldstein’s) many-sided interests, which extended from medical research to psychology and philosophy. What was he really they asked: a physician, a psychologist or a philosopher?”

Marianne Simmel (Harvard), Kurt Goldstein, p. 15. “He was very big in Vienna.” Professor Heinz Werner

Clark University Collaborators: Seymour Wapner and Bernard Kaplan Heinz Werner’s Microgenesis

“Any human activity such as perceiving, thinking, acting, etc., is an unfolding process and whether it takes seconds or hours or days, occurs in developmental sequence” (Werner, 1956) Norman Geschwind and the Rise of American Behavioral Neurology

Chief of Neurology at the Boston VA Hospital The first clinical CT scanners were installed between 1974 and 1976 The Boston Process Approach

A human cerebral disconnection syndrome Geschwind and Edith Kaplan, Neurology 1962

P.J.K., a 41- y/o white married police officer who underwent a left frontal craniotomy and later developed apraxia of the left side and agraphia of the left arm

Edith Kaplan (1924-2009) Edith Kaplan’s Phenomenological Interpretation of Common Psychological Tests Measures Edith Kaplan Authored or Co-authored

Authors Test Name Publisher Year of Publication Kaplan, Goodglass, & Boston Naming Test Lea & 1983 Weintraub, (1983) Febiger Helm- Estabrooks &‎ Kaplan The Boston Stimulus Board Riverside 1989 Helm- Estabrooks &‎ Kaplan Card Board for Memory US patent 1991 Delis, Kramer, Kaplan, & Ober California Verbal Learning Test Pearson 1987 Powell et al. Microcog Pearson 1993 Leach el al. The Kaplan-Baycrest Pearson 2000 Neurcognitve Assessment Delis, Kaplan and Kramer The Delis- Kaplan Executive Pearson 2001 Function System The Limitations of the Boston Process Approach

Professor, I am sure you were aware when you accepted the position that it was publish or perish Baxter vs . Temple (2005) Rejection of the Boston Process Approach New Hampshire Supreme Court

• Shelby Baxter suffered from lead paint poisoning • She was tested by Barbara Bruno-Golden, Ed.D. • On the day of trial, the defendants moved to exclude the testimony of Dr. Bruno-Golden as being unreliable • The court concluded that the BPA “While generally accepted in the appropriate scientific literature as a sound clinical approach to evaluating injuries for brain injury, it fails to show that it is generally accepted in the making of a forensic assessment.".

Current Trends in American Neuropsychology A survey of INS, NAN, and APA Division 40 Rabina, Barrb, Burtonc Archives of Clinical Neuropsychology Jan 2004

Field of highest degree • 68% favored a flexible battery earned approach

Clinical psychology 62.1 • 20% favored a flexible approach (i.e., Clinical neuropsychology 10.7 based upon the needs of an School psychology 5.0 individual case) Counseling psychology 11.2 • 11% favored a standardized battery Neuroscience 1.5 (i.e., HRNB, LNNB) Other 9.5 • 51.2% employed technicians (Ruff et Total 100.0 al., 2006) Rise of Corporate Tests – The Pearson and PARS Corporations The Wechsler Tests

US popularity of measures

1 WAIS-R/WAIS-III 63.1

2 WMS-R/WMS-III 42.7

3 Trail Making Test 17.6

4 CVLT/CVLT-II 17.3

5 WISC-III 15.9

6 HRNB 15.5

7 WCST 11.5

8 ROCF 10.4

9 MMPI/MMPI-2 5.9 Cautions and Concerns Regarding the New Wechsler IQ and Memory Scales (Loring and Bowers, 2010)

• Face validity is established by recruiting “advisors” (multiple authors) who later promote the new measures • The availability of normative data is confused with validity • Paucity of solid validity studies • Normative data are proprietary and not open to scrutiny • In each release previously “new” subtests are discarded due to their failure and replaced with another set of “improved” measures The Wechsler Memory Scale III

Memory for Faces WMS-III Family Pictures WMS-III

10 sec Modern Neuroimaging Studies The Quantified Process Approach Deconstructing mental processes and creating new tests

A philosophical school of neuropsychological assessment that emphasizes the link between brain structures and behavioral functions by quantifying phenomenological aspects of behaviors and validating the data using criterion (brain lesion) and predictive (outcome/ecological) validity studies Methodologies

• The “Satellite” Testing Paradigm • The Composition Paradigm • The Decomposition Paradigm • Developing new cognitive measures that are consistent with current neurocognitive models and allow for process analysis

Poreh (2000, 2006) The “Satellite” Testing Paradigm • Simple tasks are introduced to compliment existing tests so as to clarify a given test performance . The Satellite Test methodology is based on the utilization of difference scores The Reliability of Difference Scores Decreases as the Correlation Between the Tests Increases

- The Composition Paradigm

• Data are compiled and new indices that reflect underlying constructs are developed The Rey Complex Figure Test ROCF Scoring Systems

Osterrieth (1944): • Elements with which the copy started Categorical scale ranging from I to 5 Reproduction Type • Drawing the rectangle separate from the details • Piece-by-piece versus holistic approach Visser ( 1973 ): Total • Drawing each straight line continuously Numerical scale ranging Score • Completeness of each line from 0 to 30 • Order in which a subset of lines are drawn Binder (1982): Drawing continuously or contiguously each of the Numerical scale ranging Configural Score following: horizontal bisector, vertical bisector. from 0 to 5 two main diagonals. vertices of the large pentagon Bennett-Levy (1984): Drawing all straight lines continuously Numerical scale Strategy Total Score Drawing symmetric components contiguously ranging from 0 to 34 Element with which copy is started Shorr et al. (1992): Drawing continuously or contiguously 20 lines and Numerical scale ranging Perceptual Cluster Ratio line junctures, most of which are i11 the framework: from 0 to 100 corrected for number of lines drawn Bylsma et al. ( 1995): Drawing I3 multi-line elements with contiguous Numeric scale ranging from 0 to 24 Qualitative Score lines; higher weights are given for the large rectangle. bisectors, and diagonals• Order in which the large rectangle, bisectors, and diagonals are drawn•

ANGELA K. TROYER AND HEATHER A. WJSHART 1997 Development of Theory Driven Empirical Indices

Chi (1997) . A detailed analysis of the task to determine the problem space so as to develop a computational Arnold Gesell scoring model . Define a solution path to identify which sequence of operators a particular subject utilizes 6 stages of a piecewise copying Bennet Levin 1984 ROCF Scoring Systems

ANGELA K. TROYER The Five Point Test FPT – A Nonverbal Fluency Test

Prof Marianne Regard Ruff’s Revision of the Five Point Test (employing a Satellite Test Paradigm)

1 min per section

• Failure to analyze all of the permissible operators results in interpretation problems • Reliability 0.58 to 9.69 (Ruff 1996) D-KEFS Design Fluency

• D-KEFS Design Fluency relies on motor planning and cognitive flexibility. The two additional trials do not contribute to performance (Suchy, Kraybill, Gidley, Larson, 2010)

• Unlike the original design fluency tests (Jones-Gotman and Milner, 1977), the D- KEF Design Fluency is not sensitive to left- frontal lobe damage Baldo, Shimamura, Delis and Kaplan, 2001 1 min per section

Clinical Examples Total Repeat Err Unique # elements Addition Sub. Rotation

18-28 41.3 3.3 0 38.0 20.3 2.5 .67 12.36 (N=39) 30-60 36.3 5.6 0 42.0 14.9 1.7 .86 7.14 (N=7) 65-75 34.0 2.8 .10 31.0 16.6 1.55 .20 9.6 (n=20) 75-90 20.3 3.0 1.7 17.3 8.0 .72 .33 2.9 (N=19) Verbal Fluency Tests POOR Repetition INTACT

Comprehension Poor Good Poor Good

Fluency Poor Good Good Poor Poor Good Poor Good

Broca Transcortical Wernicke Sensory Global Conduction Transcortical TranscorticalAnomia Mixed Motor

Benson 1979 Verbal Fluency Tests

Benton (1968) - Phonemic fluency Rosen, (1980) - Semantic fluency

• Discrepancies in phonemic and semantic fluency are an indication of anterior as opposed to posterior damage to speech areas. Butters et al., 1987; Monsch et al., 1992; Gourovitch, Goldberg, & Weinberger, 1996; Corcoran & Upton, 1993 - • The above conclusion has not always been supported. Joanette & Goulet, 1986 Applying the Clustering and Switching Methodology

• Clustering (Laine, 1988) Phonemic Semantic

- the generation of art differ in syllable hen consecutive words arm rooster Farm animals within a particular goose bat differ in vowel 2 fly subcategory. bit insects cockroach

• Switching (Gruenewald black beetle 2 sand rhymes pigeon & Lockhead, 1990) - the stand gull birds ability to shift between nightingale sum homonyms subcategories. canary 3 some

Avg. cluster 7/3= Applying the Clustering and Switching Methodology

• Troyer et al. (1997) - young adults generate more words and switch more often. Older adults produce larger clusters on phonemic fluency tasks than younger adults. • Troyer et al. (1998) - Phonemic fluency switching is impaired among frontal lobe damaged patients. Semantic-fluency clustering is impaired among temporal lobe damaged patients. • Robert et al. (1998) – Schizophrenics • Rich (1999) - Longitudinal study of patients with Huntington's disease. Semantic hen rooster Farm animals goose fly insects cockroach black beetle pigeon gull birds nightingale canary

Avg. cluster 7/3= The Decomposition Paradigm • An approach which investigates the relationship between test items of a given measure according to underlying facets, resulting in the development of new subscores Examination of the Global and Local Features of the Rey Osterrieth Complex Figure Using Faceted Smallest Space Analysis

FSSA map for RHI patients during the copy stage for global and local elements

FSSA map for RHI patients during the recall stage for right-, middle-, and left-spatially located elements Decomposing The Trail Making Test Decomposition of the Trail Making Test Amir Poreh, Ph.D., Ashley Miller, M.A., Philipp Dines, M.D., Ph.D., Jennifer Levin, Ph.D. (2012)

Only the last section of Trails B (11, K, 12, L, and 13) correlates with other measures of executive 1,72 functioning

0,52 0,38 0,06

1 2 3 4

Latency per section Patterns of TMT Performance among Dementia Patients (n=820)

• Type I - Failure on the 1-A-2-B-3 C • Type II - Failure on 11-K-12-L-13 (Most common) • Type III – Errors on the 1-A-2-B-3 C, improvement and then errors on H-9-I-10-J Eye Tracking and the TMT Decomposing the Rey Auditory Verbal Learning Test • RVLT and CVLT sensitivity to left temporal lobe epilepsy Loring et al. (2008) - 204 patients

Test Focus n M SD t-value P value Cohen’s d

AVLT Left 91 42.9 10.6 3.21 0.002 .47 Right 98 47.7 9.9 CVLT Left 113 43.8 11.1 2.20 0.03 .30 Right 99 43.1 9.9

The increased sensitivity of the AVLT to left temporal abnormality may reflect its increased sensitivity to deficits in relational learning Decomposing the Rey Auditory Verbal Learning Test Decomposing Rey Auditory Verbal Learning Test

Larry R. Squire and Arthur P. Shimamura 1986

• Rey Auditory Demo The Quantified Process Approach – A Blueprint for Developing New Measures Successful attempts in decomposing tests depends on whether a test is “pure”. Namely, it relates to a specific neuroanatomically based model

Assessment of executive functions: Review of instruments and identification of critical issues. Chan et al. 2008. Simulation of Real-life Decision Making

• The Iowa Gambling Task (Antoine Bechara et al. 1994)

Ventromedial prefrontal cortex Bechara,Damasio, Tranel, & Damasio1997 Tanabe et al. 2007 Logical Memory – Are we measuring what we need to measure?

WMS 1945 I immediate recall of two stories WMS- R (Russell, 1974, 1988) 2 stories followed by delayed recall and recognition (yes no) WMS-III The 2nd story is read twice WMS-IV Two stories with the 1st read twice and a new geriatric version (1st story read twice)

The Poreh Geriatric Complex Figure

THE RBANS LIKE POREH’S GERIATRIC COMPLEX FIGURE COMPLEX FIGURE The Poreh Nonverbal Memory Test

• The Search for a “Pure” Visual Memory Test: Pursuit of Perfection. (Heilbronner, 1992)

• Factor analytical studies rebuff the validity of existing nonverbal memory measures – WMS Visual Reproduction – Revised Visual Retention Test (BVRT: Benton, 1974) – Memory for Designs (MFD: Graham & Kendall, 1960) Criteria for Existing Measures

• Task Demand Characteristics – Test should include a delayed interval (Heilbronner 1992) – Repetition is essential for assessing learning (Poreh, 2005) • Nature of the Test Stimuli – Geometric designs lend themselves to being easily verbalized – (De Renzi 1968; Cermak & Tarlow, 1978) • Influence of Visuoconstructive Abilities – Tests that require visuoconstructive response (drawing) reduce the proportion of variance accounted for by nonverbal memory skills

Morris Water Maze Spatial memory test in rats, developed in 1981 by Richard Morris. Poreh Nonverbal Memory Test

The PNMT test is composed of a deck of cards with an array of squares. The final version of the test includes 4 cards with high-local cues and 5 cards with random arrays of 10 squares

Poreh Nonverbal Memory Test Alternate Version Data (rodent) from the Morris Maze Test Age Group PNMT RAVLT

Linear Quadratic Logarithmic Linear Quadratic Logarithmic 18-20 R=.921 R=.971 R=.968 R=.901 R=.997 R=.991 F=16.8 F=33.6 F= 92.0 F=27,3 F=361.3 F=316.9 p=0.03 p=0.03 p= 0.002** p=0.01 p=0.003** p=0.0001**

Correlations for PNMT, ROCF and RAVLT Rey Immediate Rey Delay RAVLT RAVLT

Recall-T Score Recall-T Score List A Delay 1-5 Trial 1 -.120 Poreh-Total-Delay -.288* -.454** -.288* -.236 -.324 Poreh 1 - 5 -.081 -.252* -.192 -.356** -.398** Poreh Trial 2-1 -.023 -.081 -.052 -.373** Visual Spatial Deficits in Abstinent Alcoholics

30,00 R² = 0,9379 ROCF ROCF 25,00 Control Group N=32 3min Delayed R² = 0,9097 PNMT Complex Total -0.33** -0.38** 20,00 Control PNMT Simple Total -.260 -0.39** Score 15,00 Experimental Experimental Group 10,00 Log. (Control) (N=35) Log. (Experimental ) PNMT Complex Total -0.49* -.540* 5,00

PNMT Simple Total -0.46* -0.43* 0,00 0 1 2 3 4 5 6 Trial * 0.01 level (1-tailed) ** 0.05 (1 tailed) PNMT Learning Curve for Complex Configurations Signal-Detection Theory

• Following Polack & Norman (1964) and Green & Swets (1966) as well as Shum et al. (2010), scoring of the PNMT can also be achieved using a nonparametric measure of recognition performance (Proportion of correct acceptance or p(A). p(A) = 0.5 (1 + HR – FP)

Where HR represents hit rate and FP false positive rate (FP) Process Scores: Average Distance Between Target and First Click

Distance between target and 1st click Process Scores – Average Distance Between Target and First Two Clicks

2 3 Conclusions

The future of neuropsychological assessment depends on the ability of researchers and clinicians to : (1) Develop and implement computer programs that enable efficient access and management of large sets of data (2) Alter tests to match advances in neuroscience

THE END