Psychological Measurement mark.hurlstone @uwa.edu.au Validity Validity: Theoretical Basis Importance of Validity Classic & Contemporary Approaches PSYC3302: Psychological Measurement and Its Trinitarian View Unitary View Applications Test Content Internal Structure Response Processes Mark Hurlstone Associations With Univeristy of Western Australia Other Variables Consequences of Testing Other Week 5 Perspectives Reliability & Validity [email protected] Psychological Measurement Learning Objectives Psychological Measurement mark.hurlstone @uwa.edu.au Validity • Introduction to the concept of validity Importance of Validity • Overview of the theoretical basis of validity: Classic & Contemporary 1 Trinitarian view of validity Approaches Trinitarian View 2 Unitary view of validity Unitary View Test Content • Alternative perspectives on validity Internal Structure Response Processes • Contrasting validity and reliability Associations With Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Everyday Usage Psychological Measurement mark.hurlstone @uwa.edu.au • In everyday language, we say something is valid if it is Validity sound, meaningful, or supported by evidence Importance of Validity • e.g., we may speak of a valid theory, a valid argument, Classic & or a valid reason Contemporary Approaches • In legal terminology, lawyers say something is valid if it is Trinitarian View Unitary View "executed with the proper formalities"—such as a valid Test Content Internal Structure contract or will Response Processes • Associations With In psychometrics, validity is a term used to refer to the Other Variables Consequences of meaningfulness of a test score—what the test score truly Testing means Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity • A basic definition of validity is "how well a test measures Importance of what it claims to measure" Validity Classic & • This definition is very common, but it is an oversimplification Contemporary Approaches • Trinitarian View A better definition is that validity is "the degree to which Unitary View evidence and theory support the interpretations of test Test Content Internal Structure scores entailed by the proposed uses" of a test Response Processes Associations With • This definition has at least four important implications Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity Importance of Validity 1 Validity concerns interpretations and uses of scores Classic & 2 Validity is not a property of the test itself Contemporary Approaches Trinitarian View 3 Validity is a matter of degree Unitary View Test Content 4 Validity is based on theory and evidence Internal Structure Response Processes Associations With Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity Importance of Validity 1 Validity concerns interpretations and uses of scores Classic & 2 Validity is not a property of the test itself Contemporary Approaches Trinitarian View 3 Validity is a matter of degree Unitary View Test Content 4 Validity is based on theory and evidence Internal Structure Response Processes Associations With Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement 1. Validity Concerns Interpretations and Uses of Scores Psychological Measurement mark.hurlstone @uwa.edu.au • A measure itself is neither valid nor invalid Validity • The issue of validity concerns the interpretations and uses of Importance of Validity a measures scores Classic & Contemporary • For example, a persons’s Operation Span can be validly Approaches Trinitarian View interpreted as a measure of her or his working memory Unitary View Test Content • It would be less valid to interpret a persons’s Operation Span Internal Structure Response as a measure of her or his short-term memory Processes Associations With Other Variables • It would be totally invalid to interpret a person’s Operation Consequences of Testing Span as a measure of her or his self-esteem Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity Importance of Validity 1 Validity concerns interpretations and uses of scores Classic & 2 Validity is not a property of the test itself Contemporary Approaches Trinitarian View 3 Validity is a matter of degree Unitary View Test Content 4 Validity is based on theory and evidence Internal Structure Response Processes Associations With Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity Importance of Validity 1 Validity concerns interpretations and uses of scores Classic & 2 Validity is not a property of the test itself Contemporary Approaches Trinitarian View 3 Validity is a matter of degree Unitary View Test Content 4 Validity is based on theory and evidence Internal Structure Response Processes Associations With Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement 2. Validity Is Not a Property of The Test Itself Psychological Measurement mark.hurlstone @uwa.edu.au • As a short-hand, test users sometimes refer to a particular Validity test as a "valid test" Importance of Validity • For example, someone might say that the "Operation Span Classic & task is valid" Contemporary Approaches • Trinitarian View However, what is really meant is that the test has been Unitary View shown to be valid for a particular use, with a particular Test Content Internal Structure population of people, at a particular time Response Processes Associations With • Validity is not a property of the test itself Other Variables Consequences of Testing • It is a property of the interpretation and uses of test scores Other Perspectives Reliability & Validity [email protected] Psychological Measurement 2. Validity Is Not a Property of The Test Itself Psychological Measurement mark.hurlstone @uwa.edu.au • Validity No test is "universally valid" for all time, for all uses, with all Importance of types of test-taker populations Validity • Rather, tests may be shown to be valid within reasonable Classic & Contemporary boundaries of a contemplated usage Approaches Trinitarian View • Unitary View If those boundaries are exceeded, the validity of the test may Test Content be called into question Internal Structure Response Processes • The validity of a test may have to be re-established with the Associations With Other Variables passage of time or changes in culture Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity Importance of Validity 1 Validity concerns interpretations and uses of scores Classic & 2 Validity is not a property of the test itself Contemporary Approaches Trinitarian View 3 Validity is a matter of degree Unitary View Test Content 4 Validity is based on theory and evidence Internal Structure Response Processes Associations With Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity Importance of Validity 1 Validity concerns interpretations and uses of scores Classic & 2 Validity is not a property of the test itself Contemporary Approaches Trinitarian View 3 Validity is a matter of degree Unitary View Test Content 4 Validity is based on theory and evidence Internal Structure Response Processes Associations With Other Variables Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement 3. Validity is A Matter of Degree Psychological Measurement mark.hurlstone @uwa.edu.au • Validity Like reliability, validity is not an all-or-nothing issue—it exists Importance of on a continuum Validity • Validity is conceived in terms of strong versus weak, rather Classic & Contemporary than simply valid or invalid Approaches Trinitarian View • Unitary View For test users, validity should be a deciding factor in their Test Content choice of psychological test Internal Structure Response Processes • A test should be selected only if there is strong enough Associations With Other Variables evidence supporting its intended use and interpretation Consequences of Testing Other Perspectives Reliability & Validity [email protected] Psychological Measurement Validity in Psychometrics Psychological Measurement mark.hurlstone @uwa.edu.au Validity Importance of Validity 1 Validity concerns interpretations and uses of scores Classic & 2 Validity is not a property of the test itself Contemporary Approaches Trinitarian View 3 Validity is a matter of degree Unitary View Test Content 4 Validity is based