Individual Differences and Correlation

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions Individual Differences and Correlation Variability and Distributions Central Tendency Variability Distribution Shapes PSYC3302: Psychological Measurement and Its Covariability Covariance Applications Correlation

Composite Mark Hurlstone Variables Univeristy of Western Australia Interpreting Test Scores Z Scores T Scores Week 2 Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Learning Objectives

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions • Foundations of psychological measurement: Central Tendency Variability • Distribution Shapes variability

Covariability • covariability Covariance • interpreting test scores Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Individual Differences

Psychological Measurement mark.hurlstone • Individual differences are the currency of psychometric @uwa.edu.au analysis Distributions • It is assumed that individual differences in psychological Variability and Distributions attributes exist between people, and that these differences Central Tendency can be quantiﬁed Variability Distribution Shapes • To measure individual differences on some psychological Covariability Covariance attribute, we administer a psychological test or measure that Correlation is assumed to tap that attribute to a group of people Composite Variables • The resulting collection of measurement or test scores Interpreting Test Scores constitutes a distribution of scores Z Scores T Scores • Variability is the term used to describe the differences Percentile Ranks Normalised Scores among the scores in a distribution

Test Norms

References [email protected] Psychological Measurement Individual Differences

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency Variability • Before we look at the concept of variability, let’s ﬁrst talk a bit Distribution Shapes more about distributions of scores Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Frequency Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • A teacher administers a 100-item Distributions multiple choice exam to the 25 Variability and students in her class Distributions Central Tendency • Variability The table on the right shows each Distribution Shapes student’s raw score (number Covariability correct) on the test Covariance Correlation • The collection of scores constitute Composite Variables a distribution of scores Interpreting Test Scores • The scores can be distributed in Z Scores various ways to help summarise T Scores Percentile Ranks the data Normalised Scores

Test Norms

References [email protected] Psychological Measurement Frequency Distributions

Psychological Measurement mark.hurlstone • One way to distribute the scores is @uwa.edu.au according to the frequency with Distributions which they occur Variability and Distributions • In a simple frequency distribution, Central Tendency all scores are listed alongside the Variability Distribution Shapes number of times each score Covariability occurred Covariance Correlation • The scores could be listed in a Composite Variables table or illustrated graphically Interpreting • Test Scores This approach is useful if we have Z Scores a large number of scores T Scores Percentile Ranks (hundreds or thousands) but less Normalised Scores useful with only 25 responses Test Norms

References [email protected] Psychological Measurement Frequency Distributions

Psychological Measurement mark.hurlstone • Another kind of frequency @uwa.edu.au distribution is a grouped Distributions frequency distribution Variability and Distributions • In such distributions, test score Central Tendency intervals, known as class Variability Distribution Shapes intervals, replace the raw test Covariability scores Covariance Correlation • In the distribution on the right, Composite Variables the test scores have been

Interpreting grouped into 12 class intervals Test Scores each equal to 5 points Z Scores The width of the class intervals used T Scores is a decision for the test user Percentile Ranks • The data are now starting to Normalised Scores look much more meaningful Test Norms

References [email protected] Psychological Measurement Frequency Distributions

Psychological Measurement • Frequency distributions are more interpretable when mark.hurlstone conveyed graphically @uwa.edu.au • Two types of graphs that are useful for illustrating frequency Distributions distributions are histograms and frequency polygons Variability and Distributions Frequency Histogram Frequency Polygon Central Tendency Variability Distribution Shapes

Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Frequency Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • Frequency distributions can take on many different shapes Distributions

Variability and • One type of distribution—which you are already familiar Distributions with—that is of particular interest to measurement Central Tendency Variability researchers is the normal distribution Distribution Shapes

Covariability • notice that the distribution on the previous slide Covariance Correlation approximates a normal distribution (although it is

Composite somewhat skewed) Variables • We will talk more about the normal distribution and other Interpreting Test Scores types of distributions shortly Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variability and Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • Many key ideas in psychological measurement depend upon our ability to detect and describe distributions of test scores Distributions

Variability and • A key goal of statistics is to describe distributions of scores Distributions in meaningful ways Central Tendency Variability Distribution Shapes • There are at least three kinds of information that are used to Covariability do this Covariance Correlation 1 central tendency Composite 2 Variables variability 3 shape Interpreting Test Scores Z Scores • A good understanding of these concepts is crucial for our T Scores Percentile Ranks future discussion of reliability and validity concepts Normalised Scores

Test Norms

References [email protected] Psychological Measurement Central Tendency

Psychological Measurement mark.hurlstone • One way to describe a distribution is through a measure of @uwa.edu.au central tendency Distributions • This is a statistic that describes the “typical” or average Variability and Distributions score between the extreme scores in a distribution Central Tendency Variability • Distribution Shapes There are various measures of central tendency (median and

Covariability mode) but the mean, denoted as X, is the most common: Covariance Correlation Composite P X Variables Mean = X = , (1) N Interpreting Test Scores Z Scores • P T Scores where means to sum each individual score X in the Percentile Ranks distribution, and N represents the total number of scores Normalised Scores

Test Norms

References [email protected] Psychological Measurement Central Tendency

Psychological Measurement mark.hurlstone @uwa.edu.au Example: calculate the mean of the 25 raw class exam

Distributions scores presented in the table presented earlier Variability and Distributions Central Tendency Variability 78 + 67 + 69 + 63 + 85 + ··· + 79 Distribution Shapes X = , 25 Covariability Covariance Correlation 1803 Composite X = Variables 25

Interpreting Test Scores Z Scores X = 72.12 T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variability

Psychological Measurement • The mean is a useful way of describing a distribution but we mark.hurlstone are more interested in quantifying the extent to which people @uwa.edu.au in a distribution differ from one another—a concept known as Distributions variability Variability and • Distributions Variability is an indication of how scores in a distribution are Central Tendency scattered or dispersed Variability Distribution Shapes • Two distributions can have the same mean but very different Covariability dispersions of scores: Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variability

Psychological Measurement mark.hurlstone @uwa.edu.au • There are various statistical approaches to representing the Distributions variability in a distribution of scores (range, interquartile and Variability and Distributions semi-interquartile ranges, average deviation) Central Tendency Variability • Distribution Shapes We will focus on the variance and standard deviation since

Covariability they are the most common metrics of variability, and they lie Covariance at the heart of psychometric analysis Correlation Composite • The variance and standard deviation reﬂect variability as the Variables degree to which scores in a distribution deviate (i.e., differ) Interpreting Test Scores from the mean of that distribution Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variance

Psychological Measurement • The variance is the mean of the squared deviations between mark.hurlstone @uwa.edu.au the scores in a distribution and their mean. To compute the variance we: Distributions

Variability and 1 calculate the deviation of each score from the mean X-X Distributions 2 square each deviation (X-X)2 Central Tendency Variability 3 sum the squared deviations and divide by the total Distribution Shapes number of scores in the distribution Covariability Covariance • Correlation Formally, the variance is given by:

Composite Variables P (X − X)2 Interpreting Variance = s2 = , (2) Test Scores N Z Scores T Scores Percentile Ranks Normalised Scores • It represents the average degree to which people differ from

Test Norms each other

References [email protected] Psychological Measurement Variance

Psychological Measurement mark.hurlstone @uwa.edu.au Example: calculate the variance of the 25 raw class exam scores presented in the table presented earlier Distributions

Variability and Distributions 2 2 2 2 Central Tendency 2 [78 − 72.12] + [67 − 72.12] + [69 − 72.12] + ··· + [79 − 72.12] Variability s = Distribution Shapes 25 Covariability Covariance 4972.64 Correlation s2 = Composite 25 Variables Interpreting s2 = 198.91 Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Deviation

Psychological Measurement mark.hurlstone • @uwa.edu.au The standard deviation is closely related to the variance

Distributions • It is equal to the square root of the "average of the squared

Variability and deviations about the mean"—viz. the variance Distributions Central Tendency • Hence, the standard deviation is simply the square root of Variability Distribution Shapes the variance: Covariability Covariance Correlation s √ P (X − X)2 Composite Standard deviation = σ = s2 = , (3) Variables N Interpreting Test Scores Z Scores • Note that I am using σ to denote the standard deviation but T Scores Percentile Ranks the textbook denotes it as s Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Deviation

Psychological Measurement mark.hurlstone • @uwa.edu.au The standard deviation is closely related to the variance

Distributions • It is equal to the square root of the "average of the squared

Test Norms

References [email protected] Psychological Measurement Standard Deviation

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions Variability and Example: calculate the standard deviation of the 25 raw class Distributions Central Tendency exam scores presented in the table presented earlier Variability Distribution Shapes √ √ 2 Covariability σ = s = 198.91 = 14.1 Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Binary Items

Psychological Measurement mark.hurlstone @uwa.edu.au • Sometimes psychological tests are based on dichotomous responses: Distributions Variability and • people may have to give yes or no answers to questions Distributions • Central Tendency people may have to agree or disagree with statements Variability • people’s responses to test items may be classiﬁed as Distribution Shapes correct or incorrect Covariability Covariance Correlation • Negatively valenced responses (e.g., no, disagree, incorrect) Composite are coded with a 0, whereas positively valenced responses Variables (e.g., yes, agree, correct) are coded with a 1 Interpreting Test Scores • Like continuous items, we can calculate the mean, variance, Z Scores T Scores and standard deviation for binary items Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Binary Items

Psychological Measurement mark.hurlstone @uwa.edu.au • The mean of a binary item is calculated in the same way as Distributions for a continuous item (see earlier), but we will denote the Variability and mean as p rather than X Distributions Central Tendency • For example, suppose that 10 people are asked a single Variability Distribution Shapes question and eight people answer correct, whilst two answer Covariability incorrect Covariance Correlation • The mean is the proportion p of positively valenced (correct) Composite Variables responses: 8 / 10 = .80 Interpreting • Test Scores The proportion of negatively valenced (incorrect) responses, Z Scores denoted as q, is therefore equal to 1–p: 1 – .80 = .20 T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Binary Items

Psychological Measurement • The variance for a binary item is calculated as: mark.hurlstone @uwa.edu.au s2 = pq (4) Distributions Variability and • For our example: Distributions Central Tendency Variability Distribution Shapes s2 = (.80)(.20) Covariability Covariance Correlation s2 = .16 Composite Variables • The standard deviation is the square root of the variance: Interpreting Test Scores Z Scores √ T Scores σ = . Percentile Ranks 16 Normalised Scores Test Norms σ = .40 References [email protected] Psychological Measurement Variance vs. Standard Deviation

Psychological Measurement mark.hurlstone • @uwa.edu.au Both the variance and standard deviation are important measures of variability in psychological measurement Distributions • However, the standard deviation tends to be preferred on Variability and Distributions account that it is more "intuitive" Central Tendency Variability • Distribution Shapes This is because it reﬂects variability in terms of the raw

Covariability deviation scores, whereas the variance reﬂects variability in Covariance terms of squared deviation scores Correlation Composite • it makes intuitive sense to say that the students differed Variables on average by 14 test score points Interpreting Test Scores • it does not make intuitive sense to say that the students Z Scores differed from one another on average by 198 squared T Scores Percentile Ranks deviation points Normalised Scores

Test Norms

References [email protected] Psychological Measurement Factors Affecting The Variance and Standard Deviation

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions • The size of the variance—and therefore the standard Central Tendency deviation—is affected by two things: Variability Distribution Shapes 1 the degree to which scores in a distribution differ from Covariability Covariance each other Correlation 2 the metric of the scores in the distribution Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement 1. The Degree to Which Scores in a Distribution Differ

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions • All else being equal, a larger variance and standard deviation Central Tendency Variability indicates greater variability within a distribution of scores Distribution Shapes

Covariability • However, all else is not equal because the metric of scores in Covariance a distribution will also affect these variability indices Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement 2. The Metric of the Scores in the Distribution

Psychological Measurement mark.hurlstone @uwa.edu.au • By "metric of scores", we mean whether the scores in a Distributions distribution are derived from a "large-scale" or a Variability and Distributions "small-scale" measure Central Tendency Variability • Distribution Shapes The Scholastic Achievement Test (SAT) is an example of a

Covariability large-scale measure—it produces large test scores that Covariance average about 1000 Correlation Composite • Grade Point Average (GPA) is an example of a small-scale Variables measure—it produces test scores that range between 0 and Interpreting Test Scores 4 Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement 2. The Metric of the Scores in the Distribution

Psychological Measurement • Suppose we have two distributions of scores—one derived mark.hurlstone @uwa.edu.au from a large-scale measure (e.g., SAT) and one from a small-scale measure (e.g., GPA) Distributions

Variability and • Suppose that the scores in each distribution differ from each Distributions Central Tendency other to exactly the same degree Variability Distribution Shapes • The variance and standard deviation will nonetheless be Covariability larger for the large-scale than the small-scale measure Covariance Correlation • This is because the former measure produces larger test Composite Variables scores that inﬂate these variability indices

Interpreting Test Scores Z Scores Note: T Scores Percentile Ranks Normalised Scores • Caution should be exercised when comparing the variability

Test Norms indices of distributions of scores based on different metrics

References [email protected] Psychological Measurement 2. The Metric of the Scores in the Distribution

Interpreting Test Scores Z Scores Note: T Scores Percentile Ranks Normalised Scores • Caution should be exercised when comparing the variability

Test Norms indices of distributions of scores based on different metrics

References [email protected] Psychological Measurement Interlude: The N vs. N – 1 Controversy

Psychological Measurement mark.hurlstone @uwa.edu.au • There is some controversy in the statistics literature about

Distributions whether the denominator in the calculation of the variance

Variability and and standard deviation should be N or N – 1 (see p.45 of the Distributions textbook) Central Tendency Variability Distribution Shapes • The convention used in the textbook—and most other

Covariability psychometrics texts—is the N convention, and that is what Covariance we will use Correlation Composite • The key thing you need to know is that SPSS uses the N – 1 Variables convention, so bear in mind that it will produce slightly Interpreting Test Scores different values for the variance and standard deviation than Z Scores T Scores hand calculations performed using the N convention Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • A distribution of scores is usually graphically represented by

Distributions a curve Variability and • The x-axis gives the test score values and the y-axis Distributions Central Tendency represents the frequency or proportion of those values in the Variability Distribution Shapes distribution Covariability • Covariance Distributions come in many different shapes Correlation • Composite One distribution that is fundamental to many statistical ideas Variables and concepts is the normal distribution Interpreting Test Scores • Many statistical procedures are based on the assumption Z Scores T Scores that test scores are (approximately) normally distributed Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement • The normal curve is a bell shaped smooth curve that is mark.hurlstone @uwa.edu.au highest at its centre

Distributions • The curve is perfectly symmetrical, with no skewness

Variability and Distributions Central Tendency Variability Distribution Shapes

Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency • To illustrate the key properties of a normal distribution, let’s Variability Distribution Shapes consider a distribution of National Spelling Test Scores with Covariability X = 50 and σ = 15 Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency Variability Distribution Shapes

Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • 50% of the scores occur above the mean and 50% occur

Distributions below the mean Variability and • Distributions Approximately 34% of all scores occur between the mean Central Tendency and +1 standard deviation Variability Distribution Shapes • Approximately 34% of all scores occur between the mean Covariability Covariance and –1 standard deviation Correlation • Composite Approximately 68% of all scores occur between ±1 standard Variables deviations Interpreting Test Scores • Approximately 95% of all scores occur between ±2 standard Z Scores T Scores deviations Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • 50% of the scores occur above the mean and 50% occur

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • 50% of the scores occur above the mean and 50% occur

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • 50% of the scores occur above the mean and 50% occur

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • 50% of the scores occur above the mean and 50% occur

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • The normal distribution is important because it helps simplify Distributions the interpretation of test scores Variability and Distributions • For example, if a child obtains a raw spelling test score of Central Tendency Variability 35—exactly 1 standard deviation beneath the mean—we Distribution Shapes know that only 16% of children have lower spelling test Covariability Covariance scores than this child Correlation

Composite • This tells us that the child’s spelling ability is "relatively" poor Variables • Interpreting The characteristics of the normal distribution provide a Test Scores convenient model for score interpretation Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement • mark.hurlstone Few, if any, psychological tests yield "precisely" normal @uwa.edu.au distributions of test scores

Distributions • Distributions of actual test scores are typically skewed to Variability and some degree Distributions Central Tendency Variability Distribution Shapes

Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Distribution Shapes and Normal Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • Despite this, scores on many psychological tests are often Distributions approximately normally distributed Variability and Distributions • As a general rule (with many exceptions), the larger the Central Tendency Variability sample size and the wider the range of abilities measured by Distribution Shapes a test, the more the curve of the test scores will approximate Covariability a normal curve Covariance Correlation • A classic example of a normally distributed psychological Composite Variables characteristic is intelligence Interpreting • Test Scores Many other psychological characteristics are approximately Z Scores normal in distribution T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Quantifying the Association Between Distributions

Psychological Measurement mark.hurlstone @uwa.edu.au • Variability in measurement is an important concept Distributions • However, equally important is the concept of covariability Variability and Distributions • This refers to the degree to which two distributions of scores Central Tendency Variability vary in a like manner Distribution Shapes

Covariability • Questions of covariability require that a person contribute Covariance Correlation scores on at least two variables Composite • Variables For example, if we wanted to examine the association between GPA and IQ scores, we would need a sample in Interpreting Test Scores which each participant had taken an IQ test and obtained a Z Scores T Scores GPA Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Quantifying the Association Between Distributions

Psychological Measurement • mark.hurlstone To be useful, a measure of the covariability between two @uwa.edu.au variables (distributions of scores) must:

Distributions 1 indicate the direction of the association Variability and 2 indicate the magnitude of the association Distributions Central Tendency • Variability With regards direction, we want to know if: Distribution Shapes • relatively high scores on one variable are associated Covariability Covariance with relatively high scores on the second variable Correlation (positive association) or if ... Composite Variables • ... relatively high scores on one variable are associated

Interpreting with relatively low scores on the second variable Test Scores (negative association) Z Scores T Scores Percentile Ranks • With regards magnitude, we want to known the strength of Normalised Scores the association between two variables Test Norms

References [email protected] Psychological Measurement Covariance

Psychological Measurement • A basic measure of covariability between two distributions of mark.hurlstone scores is the covariance. To calculate the covariance we: @uwa.edu.au 1 compute the deviation of each score from the mean for Distributions each distribution Variability and Distributions 2 calculate the "cross-products" (X-X)(Y-Y) of the Central Tendency Variability deviation scores Distribution Shapes 3 calculate the mean of the cross-products Covariability Covariance • Formally, the covariance is given by: Correlation

Composite Variables P (X − X)(Y − Y) Interpreting cxy = , (5) Test Scores N Z Scores T Scores Percentile Ranks Normalised Scores • Where X is the mean of the scores in variable X, and Y is the Test Norms mean of the scores in variable Y References [email protected] Psychological Measurement Covariance

Psychological Example: calculate the covariance of the following two Measurement distributions of scores mark.hurlstone @uwa.edu.au IQ (X): 80 110 120 90 130 100 (X = 105) Distributions GPA (Y): 2.5 2.8 3.2 2.9 3.6 3 (Y = 3) Variability and Distributions Central Tendency Variability Distribution Shapes (−25)(−0.5) + (−5)(−0.2) + (−15)(0.2) + ··· + (−5)(0) cxy = Covariability 6 Covariance Correlation 12.5 + 1 + −3 + ··· + 0 Composite cxy = Variables 6 Interpreting Test Scores 31 Z Scores cxy = T Scores 6 Percentile Ranks Normalised Scores cxy = 5.17 Test Norms

References [email protected] Psychological Measurement Covariance

Psychological Measurement mark.hurlstone @uwa.edu.au • The covariance provides clear information about the Distributions direction of an association between two variables Variability and Distributions • If the covariance is a positive value—as in the example on Central Tendency Variability the previous slide—then the association is a positive one Distribution Shapes • Covariability If the covariance is a negative value then the association is a Covariance negative one Correlation Composite • Unfortunately, the covariance does not provide clear Variables information about the magnitude of the association Interpreting Test Scores • Z Scores Why so? T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Covariance

Psychological Measurement • The size of the covariance is inﬂuenced by the strength of mark.hurlstone @uwa.edu.au the association between two variables

Distributions • However, it is also inﬂuenced by the metrics of the two Variability and variables Distributions Central Tendency • Variability The covariance between two variables that produce large Distribution Shapes scores ("large-scale" variables) will tend to be larger than a Covariability covariance that involves one or more variables that produce Covariance Correlation small scores ("small-scale" variables) Composite Variables • Thus, a larger covariance between IQ and SAT scores than

Interpreting between IQ and GPA does not imply that the former Test Scores Z Scores association is stronger than the latter T Scores Percentile Ranks • This is because IQ and SAT scores are both large-scale Normalised Scores measures, whereas GPA is a small-scale measure Test Norms

References [email protected] Psychological Measurement Correlation

Psychological Measurement mark.hurlstone @uwa.edu.au • The correlation coefﬁcient provides a clear representation of Distributions both the direction and the magnitude of an association Variability and Distributions between two variables Central Tendency Variability • Correlation coefﬁcients are bounded within a range of –1 Distribution Shapes and +1 Covariability Covariance • Correlation A correlation with a value between 0 and +1 indicates a

Composite positive association between the two variables Variables • Interpreting A correlation with a value between 0 and –1 indicates a Test Scores negative association between the two variables Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Correlation

Psychological Measurement mark.hurlstone • The main benefit of a correlation coefficient is that it clearly @uwa.edu.au reflects the magnitude of the association between two

Distributions variables Variability and • Distributions The "boundedness" of correlation coefficients eliminates the Central Tendency influence of metric effects of the two variables on the Variability Distribution Shapes strength of the association between them Covariability • Covariance A correlation coefficient of a specific absolute value (e.g., rxy Correlation = –.30 or rxy = .30) represents the same magnitude of Composite Variables association, irrespective of the variables on which the correlation is based Interpreting Test Scores Z Scores • A large correlation coefficient reflects a strong association, T Scores Percentile Ranks whereas a small correlation coefficient reflects a weak Normalised Scores association Test Norms

References [email protected] Psychological Measurement Positive Correlations

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency Variability Distribution Shapes

Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Negative Correlations

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency Variability Distribution Shapes

Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Correlation

Psychological Measurement • The correlation coefﬁcient is calculated by dividing the mark.hurlstone covariance cxy by the product of the standard deviations of @uwa.edu.au variables X (σx) and Y (σy): Distributions

Variability and cxy Distributions rxy = , (6) Central Tendency σxσy Variability Distribution Shapes • For our earlier example involving IQ and GPA, the correlation Covariability Covariance coefﬁcient is: Correlation

Composite Variables 5.17 rxy = , Interpreting (17.08)(.34) Test Scores Z Scores 5.17 T Scores rxy = , Percentile Ranks 5.81 Normalised Scores

Test Norms rxy = 0.89 References [email protected] Psychological Measurement Variance for "Composite Variables"

Psychological Measurement mark.hurlstone @uwa.edu.au • Test scores are usually based on the sum of two or more Distributions items Variability and Distributions • For example, the Beck Depression Inventory (BDI) consists Central Tendency Variability of 21 items Distribution Shapes • Covariability An individual’s response to each item is scored on a scale Covariance from 0 to 3 Correlation Composite • An individual’s total score on the BDI is the sum of his/her Variables scores across the 21 items—a score ranging from 0 to 63 Interpreting Test Scores • Z Scores Such "summative scores" are known as composite scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variance for "Composite Variables"

Psychological Measurement mark.hurlstone @uwa.edu.au • The variance of a composite variable can be calculated in Distributions the way as shown previously Variability and Distributions • We simply sum the items to create our composite variable Central Tendency Variability and then calculate the variance of the composite using Distribution Shapes equation 2 Covariability Covariance Correlation • However, you should know that the variance of a composite

Composite variable is determined by: Variables 1 the variance of each item within the composite, and Interpreting Test Scores 2 the correlations among the items Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variance for "Composite Variables"

Psychological Measurement mark.hurlstone @uwa.edu.au • 2 Formally, the variance of a composite variable scomposite

Distributions containing two items summed together is computed as:

Variability and Distributions Central Tendency s2 = s2 + s2 + 2 r σ σ , (7) Variability composite i j ij i j Distribution Shapes Covariability • 2 2 Covariance Where si is the variance of item i, and sj is the variance of Correlation item j Composite Variables • rij is the correlation between the two items Interpreting Test Scores • σi is the standard deviation of item i, and σj is the standard Z Scores T Scores deviation of item j Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variance for "Composite Variables"

Psychological Measurement Example: calculate the variance of a composite variable made up mark.hurlstone of the following two items @uwa.edu.au

Distributions 2 Item i: 1 2 1 2 3 5 (si = 1.889; σi = 1.374) 2 Variability and Item j: 1 4 1 2 4 5 (sj = 2.472; σj = 1.572) Distributions Central Tendency r Variability ij = 0.874 Distribution Shapes

Covariability Covariance 2 Correlation scomposite = 1.889 + 2.472 + 2(.874)(1.374)(1.572) Composite Variables 2 Interpreting scomposite = 4.361 + 3.776 Test Scores Z Scores T Scores 2 Percentile Ranks scomposite = 8.137 Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variance for "Composite Variables"

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions • The important feature of this equation is that the total test Variability and score variance depends on item variability and the Distributions Central Tendency correlation between item pairs Variability Distribution Shapes • As the correlation between the items increases (and is Covariability positive), the magnitude of the corresponding composite Covariance Correlation score variance also increases

Composite Variables • If the two items are uncorrelated, the variance of the

Interpreting composite is simply the sum of the two items’ variances Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Variance for "Composite Variables"

Psychological Measurement mark.hurlstone @uwa.edu.au • The preceding formula is used to calculate the variance of a Distributions composite variable created by summing only two items Variability and Distributions • When we have more than two items, the variance for a Central Tendency Variability composite variable is calculated by summing the item Distribution Shapes variance and inter-item covariance matrix Covariability Covariance Correlation • It sounds horrible, but it’s very straightforward—I’ll show you

Composite how to do this in our lecture on reliability when we discuss a Variables metric known as coefﬁcient α Interpreting Test Scores • Just focus on the calculation for the two item case for now Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Interpreting Test Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • On most psychological tests, the raw scores on the test are not meaningful and interpretable Distributions

Variability and • Raw scores do not tell us if a particular score is relatively Distributions low, medium, or high Central Tendency Variability Distribution Shapes • For example, suppose a student obtains a score of 80 out of Covariability 100 on a class exam Covariance Correlation • Ostensibly, this is a high score, but what if I told you that all Composite Variables the other students obtained scores > 90—now a score of 80

Interpreting looks low, rather than, high Test Scores Z Scores • The key point is that a person’s test score only makes sense T Scores Percentile Ranks relative to the test scores of other people Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • A standard score is a raw score that has been converted Distributions from one scale to another scale Variability and Distributions • The new scale has an arbitrarily set mean and standard Central Tendency Variability deviation Distribution Shapes Covariability • Raw scores are converted to standard scores because Covariance Correlation standard scores are more easily interpretable than raw

Composite scores Variables • Interpreting With a standard score, the position of a test-taker’s Test Scores performance relative to other test-takers is readily apparent Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Scores: z Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • This type of standard scale score is based on a zero plus or minus one scale Distributions

Variability and • This is because it has a mean of 0 and a standard deviation Distributions Central Tendency of 1 Variability Distribution Shapes • A z score indicates how many standard deviation units the Covariability raw score is below or above the mean of a distribution Covariance Correlation • A z score is equal to the difference between a given raw Composite Variables score and the mean divided by the standard deviation:

Interpreting Test Scores Z Scores X − X T Scores z = , (8) Percentile Ranks σ Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Scores: z Scores

Psychological Measurement mark.hurlstone • Let’s use the example of the normally distributed spelling @uwa.edu.au data presented earlier and convert a raw score of 65 to a z

Distributions score:

Variability and Distributions Central Tendency 65 − 50 Variability z = Distribution Shapes 15 Covariability Covariance 15 Correlation z = 15 Composite Variables

Interpreting z = 1 Test Scores Z Scores T Scores • Percentile Ranks A raw score of 65 is therefore equal to a z score of +1 Normalised Scores standard deviations above the mean Test Norms

References [email protected] Psychological Measurement Standard Scores: z Scores

Psychological Measurement • Knowing the z-score provides context and meaning that is mark.hurlstone @uwa.edu.au missing from the raw score

Distributions • For example, given our knowledge of areas under the normal Variability and curve, we know that only 16% of the other test-takers Distributions Central Tendency obtained higher scores, with 84% of test-takers obtaining Variability lower scores Distribution Shapes Covariability • As well as providing a context for interpreting scores on the Covariance Correlation same test, z scores provide a context for interpreting scores Composite on different tests Variables

Interpreting • Suppose a child obtains a score of 42 on an arithmetic test Test Scores and 24 on a reading test Z Scores T Scores Percentile Ranks • On the face of it, the child performed better on the arithmetic Normalised Scores test than on the reading test Test Norms

References [email protected] Psychological Measurement Standard Scores: z Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • However, suppose the child’s z scores for the arithmetic and reading tests are –0.75 and +1.32, respectively Distributions Variability and • Although the child has a higher raw score on the arithmetic Distributions Central Tendency test than the reading test, relative to the other students in his Variability class he performed below average on the arithmetic test, Distribution Shapes

Covariability and above average on the reading test Covariance Correlation • With z scores, we can determine how much better the child Composite performed on the reading test, and how much worse the Variables child performed on the arithmetic test Interpreting Test Scores • I will show you how to do this shortly when we discuss Z Scores T Scores percentile ranks Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Scores: z Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • Although z scores are incredibly useful for imposing meaning Distributions on test scores, they may be difﬁcult to comprehend for Variability and Distributions test-users and test-takers Central Tendency Variability • As a test-taker, how would you stomach being told that your Distribution Shapes score on your PSYC3302 exam was –1.5? Covariability Covariance • Correlation It is confusing because not only is the score negative, it is

Composite expressed in decimal places Variables • Interpreting We can get around this problem by converting z scores into Test Scores T scores, which are more intuitive to understand Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Scores: T Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • This type of standard scale score is based on a ﬁfty plus or minus ten scale Distributions

Variability and • This is because it has a mean of 50 and a standard deviation Distributions Central Tendency of 10 Variability Distribution Shapes • The calculation of a T score is a two-step process: Covariability Covariance 1 convert an individual’s raw score into a z score Correlation 2 convert an individual’s z score using the following Composite Variables formula:

Interpreting Test Scores Z Scores T = Z(10) + 50, (9) T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Scores: T Scores

Psychological Measurement mark.hurlstone • Returning to our spelling score example, an individual with a @uwa.edu.au score of 65 has a z score of +1 Distributions • This individual’s T score would therefore be: Variability and Distributions Central Tendency Variability T = (1)(10) + 50 Distribution Shapes

Covariability Covariance Correlation T = 10 + 50 Composite Variables

Interpreting T = 60 Test Scores Z Scores T Scores • Percentile Ranks The T score tells us that the individual is 1 standard Normalised Scores deviation above the mean on the spelling proﬁciency test Test Norms

References [email protected] Psychological Measurement Standard Scores: T Scores

Psychological Measurement mark.hurlstone • @uwa.edu.au The scale used for computing T scores ranges from 5 standard deviations below the mean to 5 standard deviations Distributions above the mean Variability and Distributions • a raw score that falls exactly at –5 standard deviations Central Tendency Variability is equal to a T score of 0 Distribution Shapes • a raw score that falls exactly at the mean is equal to a T Covariability score of 50 Covariance Correlation • a raw score that falls exactly at +5 standard deviations Composite is equal to a T score of 100 Variables Interpreting • The scale is therefore bounded between 0 and 100 Test Scores Z Scores • Thus, unlike z scores, T scores cannot take on negative T Scores Percentile Ranks values, which makes them more intuitive for test-takers Normalised Scores

Test Norms

References [email protected] Psychological Measurement Standard Score Equivalents

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency Variability Distribution Shapes

Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Percentile Ranks

Psychological Measurement mark.hurlstone @uwa.edu.au • Another common way of interpreting test scores is in terms of percentile ranks Distributions

Variability and • Percentile ranks indicate the percentage of scores that fall Distributions below and above a speciﬁc test score Central Tendency Variability Distribution Shapes • There are two ways to determine the percentile rank for an Covariability individual’s test score Covariance Correlation • Suppose we have access to the entire distribution of test Composite Variables scores Interpreting • An individual’s percentile rank is the number of scores in the Test Scores Z Scores distribution that are lower than the individual’s score divided T Scores Percentile Ranks by the total number of scores Normalised Scores

Test Norms

References [email protected] Psychological Measurement Percentile Ranks

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions • For example, in the class exam data presented at the outset Variability and Distributions Barbara obtained an exam score of 82 Central Tendency Variability • Barbara’s score is higher than 14 out of 25 people’s exam Distribution Shapes

Covariability scores Covariance Correlation • Her percentile rank is therefore: (14/25)(100) = 56% Composite Variables • Thus, Barbara scored at the 56th percentile

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Percentile Ranks

Psychological Measurement mark.hurlstone • Sometimes we do not have access to the entire distribution @uwa.edu.au of scores Distributions • However, if we know the mean and standard deviation—and Variability and Distributions that the underlying distribution of scores is relatively Central Tendency Variability normal—we could compute a z score for an individual and Distribution Shapes link it to a percentile Covariability Covariance • We can look up the z score in a standard normal distribution Correlation table detailing distances under the normal curve (look at the Composite Variables back of any good statistics textbook) Interpreting Test Scores • The table tells us the percentage of cases that could be Z Scores T Scores expected to fall above or below a particular standard Percentile Ranks deviation point (or z score) Normalised Scores

Test Norms

References [email protected] Psychological Measurement Percentile Ranks

Psychological Measurement mark.hurlstone @uwa.edu.au • Let’s calculate the percentile rank for a raw score with a z

Distributions score of +1.5 Variability and • The table tells us that the area of the normal curve between Distributions Central Tendency the mean and a z score of +1.5 is 43.32% Variability Distribution Shapes • But remember that 50% of the scores in a normal distribution Covariability Covariance fall between the mean and –3 standard deviations Correlation • Composite The person’s percentile rank is therefore 50% + 43.32% = Variables 93.32% Interpreting Test Scores • This indicates that 93% of other test-takers scores fall below Z Scores T Scores this person Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Percentile Ranks

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions • What if the z score is negative? Variability and Distributions • In this case, we subtract the value obtained from the table Central Tendency from 50% Variability Distribution Shapes • Suppose the z score is –1.5 rather than +1.5 Covariability Covariance Correlation • The percentile would be 50% – 43.32% = 6.68%

Composite Variables • This tells us that only 6.68% of the scores in a distribution fall

Interpreting below this person Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Normalised Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • When a test developer produces a new test—e.g., of intelligence—it is desirable that the test yield normally Distributions distributed measurements Variability and Distributions • Central Tendency Often this is to provide norms (interpretive guides) for test Variability users Distribution Shapes Covariability • These norms are based on a large reference sample—also Covariance Correlation known as a normative sample—thought to be representative Composite of some population Variables

Interpreting • During test development, test constructors administer their Test Scores test to the reference sample to obtain scores that can serve Z Scores T Scores as an interpretative "frame of reference" Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Normalised Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • Distributions What happens if the reference sample data is skewed? Variability and • One option is to normalise the distribution Distributions Central Tendency Variability • This involves "stretching" the skewed curve into the shape of Distribution Shapes a normal distribution and creating a corresponding scale of Covariability Covariance standard scores (e.g., T scores)—known as a normalised Correlation standard score scale Composite Variables • The textbook describes one (of several) procedures for Interpreting normalising a distribution Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Normalised Scores

Psychological Measurement mark.hurlstone @uwa.edu.au • However, it is generally preferable to "ﬁne-tune" a test to Distributions yield approximately normally distributed measurements than Variability and to normalise distributions Distributions Central Tendency • This is because there are technical cautions to observe Variability Distribution Shapes before normalising a skewed distribution Covariability Covariance • For example, transformations should only be attempted when Correlation the test sample is large enough and representative enough Composite Variables • If doubts exist about sample size and representativeness, Interpreting Test Scores failure to obtain normally distributed scores may not be the Z Scores fault of the measuring instrument T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement A Final Note on Score Conversions

Psychological Measurement mark.hurlstone @uwa.edu.au • You might be wondering whether it is legitimate to convert Distributions test scores using the procedures just described (z, T, and Variability and Distributions normalised scores, percentile ranks) Central Tendency Variability • These raw score conversion methods are based on sound Distribution Shapes statistical logic Covariability Covariance • Correlation They do not represent attempts to "cook" the data—to paint

Composite a distorted picture of test scores Variables • On the contrary, the aim is to make the data more readily Interpreting Test Scores interpretable and meaningful for test-users and test-takers Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement Test Norms

Psychological Measurement mark.hurlstone @uwa.edu.au • Earlier, we touched on the notion of a reference sample

Distributions • The point was made that the reference sample for a test

Variability and must be representative of the population of interest Distributions Central Tendency • For example, in IQ testing, you would want a reference Variability Distribution Shapes sample that consists of people from a wide range of walks of Covariability life—e.g., gender, education, socioeconomic status, etc. Covariance Correlation • Good IQ tests have reference samples that correspond to Composite Variables that countries bureau of statistics information Interpreting • If about 20% of the adult population has a university degree, Test Scores Z Scores then the test developer will want a reference sample with T Scores Percentile Ranks about 20% of people with a university degree Normalised Scores

Test Norms

References [email protected] Psychological Measurement Probability vs. Non-Probability Sampling

Psychological Measurement mark.hurlstone @uwa.edu.au • Probability sampling uses procedures that ensure a representative sample Distributions

Variability and • Random sampling is a type of probability sample, as you Distributions would expect a random sample from the population to be Central Tendency Variability representative of the population Distribution Shapes Covariability • However, random sampling rarely ever happens in practice Covariance Correlation • Many test norms are based on non-probability samples Composite Variables • e.g., the normative sample may consist of all the people Interpreting who have ever taken the test Test Scores Z Scores • obviously, there would be some self-selection bias in T Scores Percentile Ranks such a process Normalised Scores

Test Norms

References [email protected] Psychological Measurement In Next Week’s Lab ...

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency • Gentle introduction to SPSS Variability Distribution Shapes • Read the Andy Field textbook chapter "SPSS Environment" Covariability before attending your lab (located in the week 3 lab folder) Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement In Next Week’s Lecture ...

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions • Central Tendency Introduction to reliability concepts: Variability Distribution Shapes 1 Theoretical basis of reliability Covariability Covariance 2 Empirical estimates of reliability Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement References

Psychological Measurement mark.hurlstone @uwa.edu.au

Distributions

Variability and Distributions Central Tendency Variability Furr, M. R., & Bacharach, V. R. (2014; Chapter 3). Distribution Shapes Psychometrics: An Introduction (second edition). Sage. Covariability Covariance Correlation

Composite Variables

Interpreting Test Scores Z Scores T Scores Percentile Ranks Normalised Scores

Test Norms

References [email protected] Psychological Measurement