Goals Psychometric Theory: Goals II 4
Total Page:16
File Type:pdf, Size:1020Kb
Psychometric Theory • ‘The character which shapes our conduct is Psychology 405: a definite and durable ‘something’, and Psychometric Theory therefore! ... it is reasonable to attempt to measure it. (Galton, 1884) William Revelle • “Whatever exists at all exists in some amount. To know it thoroughly involves knowing its Northwestern University quantity as well as its!quality” (E.L. Thorndike, 1918) Spring, 2003 http://pmc.psych.nwu.edu/revelle/syllabi/405.html Psychometric Theory: Goals Psychometric Theory: Goals II 4. To instill an appreciation of and an interest 1. To acquire the fundamental vocabulary and in the principles and methods of logic of psychometric theory. psychometric theory. 2. To develop your capacity for critical 5. This course is not designed to make you judgment of the adequacy of measures into an accomplished psychometist (one purported to assess psychological constructs. who gives tests) nor is it designed to make 3. To acquaint you with some of the relevant you a skilled psychometrician (one who literature in personality assessment, constructs tests), nor will it give you "hands psychometric theory and practice, and on" experience with psychometric methods of observing and measuring affect, computer programs. behavior, cognition and motivation. Psychometric Theory: Text and Syllabus Requirements • Objective Midterm exam • Nunnally, Jum & Bernstein, Ira (1994) • Objective Final exam Psychometric Theory New York: • Final paper applying principles from the McGraw Hill, 3rd ed.(required: available course to a problem of interest to you. at Norris) • Sporadic applied homework and data sets • Loehlin, John (1998) Latent Variable Models: an introduction to factor, path, and structural analysis . Hillsdale, N.J.: LEA. (recommended) 1 Syllabus: Psychometric Theory: A conceptual Syllabus Overview X1 Y1 X2 L1 Y2 I. Individual Differences and Experimental Psychology X3 L4 II. Models of measurement Y3 III. Test theory X4 Y4 A. Reliability X5 L2 B. Validity (predictive and construct) Y5 C. Structural Models X6 D. Test Construction Y6 X7 L5 IV. Assessment of traits Y7 X8 L3 V. Methods of observation of behavior Y8 X9 Constructs/Latent Variables Theory as organization of constructs L1 L1 L4 L4 L2 L2 L5 L5 L3 L3 Theories as metaphors and analogies-1 Theory development and testing • Physics – Planetary motion • Theories as organizations of observables • Ptolemy • Constructs, latent variables and observables • Galileo • Einstein – Observables – Springs, pendulums, and electrical circuits • Multiple levels of description and abstraction – The Bohr atom • Multiple levels of inference about observables – Latent Variables • Biology • Latent variables as the common theme of a set of observables – Evolutionary theory • Central tendency across time, space, people, situations – Genetic transmission – Constructs as organizations of latent variables and observed variables 2 Theories as metaphors and analogies-2 Examples of psychological constructs and their operationalization as observables • Business competition and evolutionary theory • Anxiety – Business niche – Trait – State – Adaptation to change in niches • Love • Learning, memory, and cognitive psychology • Conformity – Telephone as an example of wiring of connections • Intelligence – Digital computer as information processor • Learning and memory – Parallel processes as distributed information processor – Procedural - memory for how – Episodic -- memory for what • Implicit • explicit Observables/measured variables Models and theory X1 Y1 X2 Y2 • Formal models X3 – Mathematical models Y3 X4 – Dynamic models - simulations Y4 X5 • Conceptual models Y5 – As guides to new research X6 Y6 – As ways of telling a story X7 • Organizational devices Y7 X8 • Shared set of assumptions Y8 X9 Psychometric Theory: A conceptual Syllabus A Theory of Data: What can be measured X1 X1 Y1 X2 L1 What is measured? Y2 Individuals X3 L4 Y3 Objects X4 Y4 X5 L2 What kind of measures are taken? Y5 X6 proximity Y6 order X7 L5 Y7 Comparisons are made on: X8 L3 Y8 Single Dyads or Pairs of Dyads X9 3 Scaling: the mapping between observed and Variance, Covariance, and Correlation latent variables X1 X1 Simple correlation L1 X2 Simple regression X3 Y3 X4 Multiple correlation/regression X5 Y5 Observed X6 Y6 Variable Partial correlation X7 Y7 X8 Y8 Latent Variable X9 Classic Reliability Theory: How well do we Types of Validity: What are we measuring measure what ever we are measuring X1 X1 Face Y1 X2 L1 X2 Concurrent Y2 X3 X3 Predictive Y3 X4 Construct Y4 X5 L2 Y5 X6 Convergent Y6 X7 Discriminant L5 Y7 X8 L3 Y8 X9 Techniques of Data Reduction: Structural Equation Modeling: Combining Factors and Components Measurement and Structural Models X1 X1 Y1 Y1 X2 L1 X2 L1 Y2 Y2 X3 L4 X3 L4 Y3 Y3 X4 X4 Y4 Y4 X5 L2 X5 L2 Y5 Y5 X6 X6 Y6 Y6 X7 L5 X7 L5 Y7 Y7 X8 L3 X8 L3 Y8 Y8 X9 X9 4 Scale Construction: practical and theoretical Traits and States: What is measured? X1 X1 Y1 Y1 X2 L1 X2 BAS Y2 Y2 X3 L4 X3 Affect Y3 Y3 X4 X4 Y4 Y4 X5 L2 X5 BIS Y5 Y5 X6 X6 Y6 Y6 X7 L5 X7 Cog Y7 Y7 X8 L3 X8 IQ Y8 Y8 X9 X9 The data box: measurement across time, Psychometric Theory: A conceptual Syllabus situations, items, and people X1 Y1 X2 L1 Y2 P1 X3 L4 Y3 P2 X4 P3 Y4 P4 X5 L2 . Y5 . X6 Pi Y6 Pj X7 L5 Tn Y7 … … T2 X8 L3 Pn Y8 T1 X1 X2 … Xi Xj … Xn X9 Syllabus: Overview Two Disciplines of Psychological Research (Cronbach, 1957, 1975; Eysenck, 1966, 1997) I. Individual Differences and Experimental Psychology A. Two historic approaches to the study of psychology B=f(Personality) B=f(P*E) B=f(Environment) B. Individual differences and general laws C. The two disciplines reconsidered Darwin II. Models of measurement Galton Weber, Fechner A. Theory of Data B. Issues in scaling Binet, Terman Watson, Thorndike C. Variance, Covariance, and Correlation III. Test theory Allport, Burt Lewin Hull, Tolman A. Reliability B. Validity (predictive and construct) Cattell Atkinson, Spence, Skinner C. Structural Models D. Test Construction Eysenck IV. Assessment of traits Epstein Mischel V. Methods of observation of behavior 5 Two Disciplines of Psychological Research B=f(Person) B=f(Environment) Types of Relationships Method/ Correlational Experimental (Vale and Vale, 1969) Model Observational Causal Biological/field Physical/lab • Behavior = f(Situation) Statistics Variance Mean • Behavior = f1(Situation)+ f2(Personality) Dispersion Central Tendency • Behavior = f1(Situation)+f2(Personality)+ Correlation/ Covariance t-test, F test f3(Situation*Personality) Effects Individuals Situations • Behavior = f1(Situation * Personality) Individual Differences General Laws • Behavior = idiosyncratic B=f(P,E) Effect of individual in an environment Multivariate Experimental Psychology Types of Relationships: Types of Relationships: Behavior = f(Situation) Behavior = f1(Situation)+f2(Person) High ability Output Output Low ability Behavioral Behavioral Environmental Input Environmental Input (income) Neuronal excitation = f(light intensity) Probability of college = f1(income) + f2(ability) Types of Relationships: Types of Relationships: Behavior = f1(Situation)+f2(Personality)+ f3(Situation*Personality) Behavior = f(Situation*Person) Low Output Output High Behavioral Behavioral High Low Environmental Input Avoidance = f1(shock intensity)+f2(anxiety) + f3(shock*anxiety) Environmental Input Reading = f1(sesame street) = f2(ability) + f3(ss * ability) Eating = f(preload * restraint) GRE = f(caffeine * impulsivity) 6 Types of Relationships: Persons, Situations, and Theory Behavior = f(Situation*Person) Observed relationship Low High Performance Output External stimulation-> Individual Difference General Law Behavioral Arousal Environmental Input Performance External stimulation-> Arousal-> GRE = f(caffeine * impulsivity) Theory and Theory Testing I: Theory and Theory Testing II: Theory Experimental manipulation Construct 2 Construct 2 Construct 1 Construct 1 rc1c2 rmo Fm Manipulation 1 Observation 1 Theory and Theory Testing III: Theory and Theory Testing IV: Correlational inference Correlational inference Construct X ? ? Construct 1 Construct 2 Construct 1 Construct 2 ? ? rmo rmo Observation 1 Observation 2 Observation 1 Observation 2 7 Theory and Theory Testing V: Alternative Explanations Individual differences and general laws ? Impulsivity Construct 1 Construct 2 Attention Reaction Time Arousal GREs Working Memory Memory Span Observation 1 Observation 2 Caffeine Theory and Theory Testing VI: Psychometric Theory: A conceptual Syllabus Eliminate Alternative Explanations X1 Y1 X2 L1 Y2 X3 L4 Y3 Construct 1 Construct 2 X4 Y4 X5 L2 Y5 X6 Y6 X7 L5 Observation 1 Observation 2 Y7 X8 L3 Y8 X9 A Theory of Data: What can be measured X1 Coombs: A theory of Data What is measured? Individuals • O = {Stimulus Objects} S={Subjects} Objects • O = {o1, o2, …, oi, …, on} • S = {s1, s2, …, si, …, sm} What kind of measures are taken? • S x O = {(s1, o1 ) ,(si,oj), … , (sm, on)} proximity • O x O = {(o1, o1 ) ,(oi,oj), … , (on, on)} order • Types of Comparisons: Comparisons are made on: – Order si <oj (aptitudes or amounts) – Proximities |s -o | < d (preferences ) Single Dyads or Pairs of Dyads i j 8 Coombs typology of data Metric spaces and the axioms of Single Dyads Pairs of Dyads a distance measure Measurement (S*O) s <o i j |si -oj | < • A metric space is a set of points with a distance Stimuli (Abilities) |s -o | < d |sk -ol | function, D, which meets the following properties i j Unfolding (S*O)*(S*O) (attitudes) Single Preferential choice • Distance is symmetric, positive definite, and satisfies Scaling of stimuli the triangle inequality: (O * O) MDS – D(X,Y)>= 0 (non negativity) Stimuli oi <oj (O*O)*(O*O) – D(X,Y) = 0 iff X=Y (D(X,X)=0 reflexive) of |oi -oj | < |ok -ol | – D(X,Y) = D(Y,X) (symmetric) Pairs – D(X,Y) + D(Y,Z) => D(X,Z) (triangle inequality) Individual differences in Multidimensional Scaling S * (O*O) * (O*O) Scaling of Stimuli (O*O) Thurstonian Scaling of Stimuli • Finding a distance metric for a set of stimuli • What is scale location of objects I and J on an attribute dimension D? – Sports teams (wins and losses) • Assume that object I has mean value mi with some – Severity of crimes (judgments of severity) variability.