sign in sign up
<<
Home , Psychometrics

Overview

Psychometrics: • A brief history of psychometrics An introduction • The main types of tests • The 10 most common tests • Why psychometrics?: Clinical versus actuarial judgment

Psychometrics: An intro Psychometrics: An intro

A brief history • Testing for proficiency dates back to 2200 B.C., when the • Modern psychometrics dates to Sir Francis Galton (1822- Chinese emperor used grueling tests to assess fitness for 1911), ’s cousin office • Interested in (in fact, obsessed with) individual differences and their distribution • 1884-1890: Tested 17,000 individuals on height, weight, sizes of accessible body parts, + behavior: hand strength, visual acuity, RT etc • Demonstrated that objective tests could provide meaningful scores

Psychometrics: An intro Psychometrics: An intro

James Cattell Clark Wissler

• James Cattell !(studied with Wundt & Galton) first used • Clark Wissler (Cattell’s student) did the first basic the term ‘mental ’in 1890 validational research, examining the relation between the old ‘mental test’ scores and • His tests were in the ‘brass instruments’ • His results were largely discouraging tradition of Galton • He had only bright college students in his • mostly motor and acuity tests • Founded ‘’(1897) • Why is this a problem? • Wissler became an anthropologist with a strong environmentalist .

Psychometrics: An intro Psychometrics: An intro

1 The rise of psychometrics

• Goodenough (1949): The Galtonian approach was like • Lewis Terman (1916) produced a major revision of “inferring the nature of from the the nature of Binet’s stupidity or the qualities of water from those • Robert Yerkes (1919) convinced the US government to of….hydrogen and oxygen”. test 1.75 million army recruits • Alfred Binet (1905) introduced the first • Post WWI: emerged, making other modern test, which directly aptitude and personality tests possible tested higher psychological processes (real abilities & practical judgments) • i.e. picture naming, rhyme production, weight ordering, question answering, word definition. • Also motivated IQ (Stern, 1914): mental ‘age’ divided by chronological age

Psychometrics: An intro Psychometrics: An intro

What is a psychometric test? The main types of tests

• A test is a standardized procedure for behavior • Intelligence tests: Assess intelligence and describing it using scores or categories – Most tests are predictive of of some non-test behavior of interest • Aptitude tests: Assess capability – Most tests are norm-referenced = they describe the behavior in • Achievement tests: Assess degree of accomplishment terms of norms, test results gathered from a large group of • tests: Assess capacity for novelty subjects (the standardization sample) – Some tests are criterion-referenced = the objective is to see if the • Personality tests: Assess traits subject can attain some pre-specified criterion. • Interest inventories: Assess preferences for activities • Behavioral tests: Measure behaviors and their antecedents/consequences • Neuropsychological tests: Measure cognitive, sensory, perceptual, or motor functions

Psychometrics: An intro Psychometrics: An intro

The 10 most commonly used tests Clinical versus actuarial judgment

1.) Wechsler Intelligence Scale for Children (WISC) • Clinical judgment = reaching a decision by processing 2.) Bender Visual-Motor Gestalt Test information in ones head 3.) Wechsler Intelligence Scale (WAIS) • Actuarial judgment = reaching a decision without 4.) Minnesota Multiphasic Personality Inventory (MMPI) employing human judgment, using empirically-established 5.) Rorschach Ink Blot Test relations between and the event of interest 6.) Thematic Apperception Test (TAT) 7.) Sentence Completion – ‘Actuarial’: ad. L. actu [{amac}] ri-us, a keeper of 8.) Goodenough Draw-A-Person Test accounts 9.) House-Tree-Person Test – Note that some of the data in an actuarial judgment may 10.) Stanford-Binet Intelligence Scale be qualitative clinical , allowing a mixture of methods From Brown & McGuire, 1976

Psychometrics: An intro Psychometrics: An intro

2 Clinical versus actuarial judgment Meehl (1954): Results • Paul Meehl (1954) first addressed the question: • He looked at between 16 and 20 studies Which is better? (depending on inclusion criteria) • His ground rules for comparison: “…it is clear that the dogmatic, complacent assertion sometimes heard from clinicians that ‘naturally’ clinical – Both methods should draw from the same prediction, being based on ‘real ’ is data set (this was relaxed by others, with superior, is simply not justified by the facts to date”. no changes in results) – Cross-validation should be required, to • In all but one case, predictions made by actuarial avoid using variation specific to the data were equal to or better than clinical set methods – There should be explicit prediction of – In a later paper, he changed his mind about the one. success, recidivism, or recovery Psychometrics: An intro Psychometrics: An intro

Thirty years later... Where are clinician’s strengths? I i.) Theory-mediated judgments “There is no controversy in social science that – If the predictor knows the relevant causal shows such a large body of qualitatively influences, can measure them, and has a model diverse studies coming out so uniformly as specific enough to take him/her from theory to this one.” fact Paul Meehl, 1986 – However, are there any to doubt this potential advantage?

Psychometrics: An intro Psychometrics: An intro

Where are clinician’s strengths? II Where are clinician’s strengths? III

ii.) Ability to use rare events iii.) Able to detect predictive cures – If the predictor knows that the current case is an - Humans beings still (for now) are masters at exception to the statistical trend, s/he can use recognizing some complex configurations, such as that information to over-ride the trend facial expressions etc. – it is in theory possible to build these into actuarial methods • Why is it very difficult in practice? • Why might we worry about clinicians ability to incorporate rare events into prediction?

Psychometrics: An intro Psychometrics: An intro

3 Where are clinician’s strengths? IV Where are actuarial strengths? I iv.) Able to re-weight utilities in real-time i.) Immunity from fatigue, forgetfulness, - For ethical, legal, or humanitarian reasons, hang-overs, hostility, prejudice, ignorance, we might decide to do things differently false association, over-confidence, bias, and than usual in particular cases. random fluctuations in judgment.

Psychometrics: An intro Psychometrics: An intro

Where are actuarial strengths? II Where are actuarial strengths? III iii.) Feedback & base-rates ‘built-in’ to the ii.) Consistency & proper weighting system - variables are weighted the same way every - Clinicians rarely know how they are doing time, according to their actual demonstrable because they don’t get immediate feedback and contributions to the criterion of interest because they have imperfect - irrelevant variables are properly weighted - actuarial records constitute perfect of to zero how things came out in similar cases and can include a larger and wider sample than a human can ever hope to see

Psychometrics: An intro Psychometrics: An intro

Where are actuarial strengths? IV iv.) Not overly sensitive to optimal weightings - Even simplistic actuarial judgments often beat human judgments

Psychometrics: An intro

4