Psychological Measurement mark.hurlstone @uwa.edu.au
Response Psychometric Barriers: Response Biases Bias
Types of Response Biases Acquiescence Bias Extreme and PSYC3302: Psychological Measurement and Its Moderate Responding Applications Social Desirability Malingering Random Responding Guessing Mark Hurlstone Univeristy of Western Australia Methods For Coping With Response Bias Week 7 Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Learning Objectives
Psychological Measurement mark.hurlstone @uwa.edu.au • Introduce problem of response biases: Response Bias • Acquiescence Bias Types of • Extreme and Moderate Responding Response Biases • Social Desirability Acquiescence Bias • Extreme and Malingering Moderate Responding • Random Responding Social Desirability Malingering • Guessing Random Responding Guessing • Methods that have been used to detect the problem Methods For Coping With • Methods that have been used to deal with the problem Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • Imagine that you have taken a personality test as part of a job interview and you were asked to respond to these Response Bias questions: Types of 1 "Have you every stolen anything from an employer?" Response Biases 2 "Do you always tell the truth to others?" Acquiescence Bias Extreme and Moderate • The true answer to 1 may be "yes" and the true answer to 2 Responding Social Desirability may be "no" Malingering Random Responding • Guessing But your actual responses might be biased by your desire to
Methods For impress the employer Coping With Response • Such a bias would affect the validity of interpretations of your Bias Managing Testing questionnaire responses as reflecting honesty or integrity Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • We want to interpret responses on psychological measures as being an accurate reflection of individuals’ true Response Bias psychological characteristics Types of • However, responses to psychological measures can be Response Biases systematically biased for a variety of reasons Acquiescence Bias Extreme and Moderate • These biases are important because they can harm the Responding Social Desirability psychometric quality of psychological tests Malingering Random Responding • Guessing Specifically, they can diminish test reliability and the validity
Methods For of test score interpretations Coping With Response • Response biases can therefore damage our ability to use Bias Managing Testing psychological measures in a meaningful way Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • There are various different types of response biases Types of • Response They can be affected by: Biases Acquiescence Bias • the testing content Extreme and Moderate • the testing format Responding Social Desirability • the testing context Malingering • Random Responding respondent’s conscious efforts to distort "reality" Guessing • unconscious factors that bias responses Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • Acquiescence bias occurs when an individual agrees with statements without regard for the meaning of those Response Bias statements Types of • Response For example, an inventory may have items such as: Biases Acquiescence Bias • "I enjoy my job" Extreme and Moderate • "I dislike my job" Responding Social Desirability Malingering • In its extreme form, people who engage in acquiescence will Random Responding Guessing respond "strongly agree" to both of these items even though
Methods For they are polar opposites of the same construct Coping With Response • These are people who will agree or say "yes" to just about Bias Managing Testing everything Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement • Imagine that an organisational psychologist is interested in mark.hurlstone @uwa.edu.au the association between job satisfaction and perceived prestige Response Bias • Suppose job satisfaction is measured with the following Types of items: Response Biases 1 "I really enjoy my work" Acquiescence Bias Extreme and 2 "I find my work personally fulfilling" Moderate Responding 3 Social Desirability "In general, I am satisfied with the day to day aspects of Malingering my job" Random Responding Guessing 4 "There is very little I would change about my job" Methods For Coping With • Response are made on a 7-point scale (1 = strongly Response Bias disagree; 7 = strongly agree) Managing Testing Context • Scores on a each item are simply summed to give an overall Managing Test Content measure of job satisfaction Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • The items’ phrasing is an important issue in the job Response Bias satisfaction scale Types of • Note that all items are "positively keyed"—a positive Response Biases (agreement) response to each item reflects a relatively high Acquiescence Bias Extreme and level of the construct (job satisfaction) being assessed Moderate Responding Social Desirability • This renders the questionnaire susceptible to the effects of Malingering Random Responding an acquiescence response bias Guessing • Methods For Consider a hypothetical "all-seeing" example in which we Coping With know which respondents exhibited an acquiescence bias Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing JS = job satisfaction; PP = perceived prestige Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • Response In this "all-seeing" example, we know that participants 1 and Bias 4 were acquiescent during responding Types of Response • Both participants agreed strongly with all four items even Biases Acquiescence Bias though they honestly might not be satisfied with their jobs Extreme and Moderate Responding • By contrast, participant 2 also agreed with all four items Social Desirability Malingering because she is genuinely satisfied with her job Random Responding Guessing • In practice, we would not be able to distinguish the Methods For acquiescent responders from the valid responders Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing JS = job satisfaction; PP = perceived prestige Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • Response In this "all-seeing" example, we know that participants 1 and Bias 4 were acquiescent during responding Types of Response • Both participants agreed strongly with all four items even Biases Acquiescence Bias though they honestly might not be satisfied with their jobs Extreme and Moderate Responding • By contrast, participant 2 also agreed with all four items Social Desirability Malingering because she is genuinely satisfied with her job Random Responding Guessing • In practice, we would not be able to distinguish the Methods For acquiescent responders from the valid responders Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Interpreting Individual Test Scores
Psychological Measurement mark.hurlstone @uwa.edu.au • Acquiescence bias can make it difficult to use test scores to Response identify individuals who have a high level of the construct Bias being assessed Types of Response • Biases Imagine, for example, that a human resources director Acquiescence Bias administers a Conscientiousness scale to a group of job Extreme and Moderate applicants Responding Social Desirability Malingering • Suppose that he is unaware of the problem of acquiescence Random Responding Guessing bias
Methods For Coping With • He could end up employing an acquiescent responder, Response instead of someone who is a truly conscientious worker Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Spurious Correlations
Psychological Measurement mark.hurlstone • If multiple tests are "contaminated" by acquiescence bias the @uwa.edu.au tests will be more strongly correlated than the underlying Response constructs Bias Types of • If respondents are acquiescent on one test they are likely to Response Biases be so on another test, giving rise to relatively high scores on Acquiescence Bias both Extreme and Moderate Responding • Social Desirability Returning to the job satisfaction example, suppose that the Malingering organisational psychologist also administered a four-item Random Responding Guessing measure of perceived prestige answered on a 5-point scale Methods For (1 = strongly disagree; 5 = strongly agree) Coping With Response Bias • Suppose also that participants 1 and 4 once again exhibit an Managing Testing Context acquiescence bias Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Spurious Correlations
Psychological Measurement mark.hurlstone @uwa.edu.au • Across all six participants, the correlation between job satisfaction and perceived prestige is r = .43 Response Bias • However, in our "all-seeing" state, we can also compute the Types of Response correlation between the two measures for the four Biases participants who responded validly Acquiescence Bias Extreme and Moderate • The valid responder correlation is much weaker at r = –.09 Responding Social Desirability Malingering • In practice, the researcher only has access to the original Random Responding Guessing correlation—she can’t distinguish "valid" from "invalid"
Methods For responders Coping With Response • Thus, she would incorrectly conclude that there is a link Bias Managing Testing between the two constructs Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Spurious Correlations
Psychological Measurement mark.hurlstone @uwa.edu.au • Across all six participants, the correlation between job satisfaction and perceived prestige is r = .43 Response Bias • However, in our "all-seeing" state, we can also compute the Types of Response correlation between the two measures for the four Biases participants who responded validly Acquiescence Bias Extreme and Moderate • The valid responder correlation is much weaker at r = –.09 Responding Social Desirability Malingering • In practice, the researcher only has access to the original Random Responding Guessing correlation—she can’t distinguish "valid" from "invalid"
Methods For responders Coping With Response • Thus, she would incorrectly conclude that there is a link Bias Managing Testing between the two constructs Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing JS = job satisfaction; PP = perceived prestige Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Spurious Correlations
Psychological Measurement mark.hurlstone @uwa.edu.au • Across all six participants, the correlation between job satisfaction and perceived prestige is r = .43 Response Bias • However, in our "all-seeing" state, we can also compute the Types of Response correlation between the two measures for the four Biases participants who responded validly Acquiescence Bias Extreme and Moderate • The valid responder correlation is much weaker at r = –.09 Responding Social Desirability Malingering • In practice, the researcher only has access to the original Random Responding Guessing correlation—she can’t distinguish "valid" from "invalid"
Methods For responders Coping With Response • Thus, she would incorrectly conclude that there is a link Bias Managing Testing between the two constructs Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Spurious Correlations
Psychological Measurement mark.hurlstone @uwa.edu.au • Across all six participants, the correlation between job satisfaction and perceived prestige is r = .43 Response Bias • However, in our "all-seeing" state, we can also compute the Types of Response correlation between the two measures for the four Biases participants who responded validly Acquiescence Bias Extreme and Moderate • The valid responder correlation is much weaker at r = –.09 Responding Social Desirability Malingering • In practice, the researcher only has access to the original Random Responding Guessing correlation—she can’t distinguish "valid" from "invalid"
Methods For responders Coping With Response • Thus, she would incorrectly conclude that there is a link Bias Managing Testing between the two constructs Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing JS = job satisfaction; PP = perceived prestige Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Spurious Correlations
Psychological Measurement mark.hurlstone @uwa.edu.au • Across all six participants, the correlation between job satisfaction and perceived prestige is r = .43 Response Bias • However, in our "all-seeing" state, we can also compute the Types of Response correlation between the two measures for the four Biases participants who responded validly Acquiescence Bias Extreme and Moderate • The valid responder correlation is much weaker at r = –.09 Responding Social Desirability Malingering • In practice, the researcher only has access to the original Random Responding Guessing correlation—she can’t distinguish "valid" from "invalid"
Methods For responders Coping With Response • Thus, she would incorrectly conclude that there is a link Bias Managing Testing between the two constructs Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: "Nay-Saying"
Psychological Measurement mark.hurlstone @uwa.edu.au • We have focused on acquiescence or "yea-saying" Response bias—where an individual agrees with whatever is presented Bias
Types of • However, it can also take the form on nonacquiescence Response Biases bias—"nay-saying", where an individual disagrees with Acquiescence Bias whatever is presented Extreme and Moderate Responding • A nay-saying bias can have similar consequences to a Social Desirability Malingering yea-saying bias Random Responding Guessing • Respondents who obtain low scores on one test will obtain Methods For Coping With low scores on another test, causing correlations that are Response artificially higher than they should be Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Acquiescence Bias: Factors that Increase Acquiescence
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Acquiescence bias is a threat to the psychometric quality of Bias a test Types of Response • Factors that increase the acquiescence bias include Biases Acquiescence Bias Extreme and 1 ambiguous items Moderate Responding 2 long items Social Desirability Malingering 3 large number of items Random Responding Guessing • Additionally, some people are naturally more susceptible to Methods For the bias than others Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au • The problem of extreme and moderate response biases Response refers to differences in the tendency to use or avoid extreme Bias response options Types of Response • Biases Some people are simply more likely to endorse the extreme Acquiescence Bias of a response scale (even though they do not possess the Extreme and Moderate attribute to that extreme degree)—known as "extreme Responding Social Desirability responding" Malingering Random Responding Guessing • Conversely, some people will choose a response somewhere
Methods For in the middle to avoid making a strong claim Coping With Response • This is known as "moderate responding" Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement • Suppose a researcher is interested in the association mark.hurlstone @uwa.edu.au between emotional distress and spirituality
Response • Emotional distress is assessed using the the State-Trait Bias Anxiety Inventory: Types of Response • "I lack self-confidence" Biases Acquiescence Bias • 4-point response scale: almost never, sometimes, often, Extreme and Moderate Responding almost always Social Desirability Malingering • Spirituality is assessed using the Spirituality/Religiousness Random Responding Guessing scale: Methods For Coping With • "I am a spiritual person" Response Bias • 5-point response scale: very inaccurate, moderately Managing Testing Context inaccurate, neither inaccurate nor accurate, moderately Managing Test Content accurate, very accurate Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Bias Extremity bias can generate artificial differences amongst
Types of respondents’ test scores Response Biases • For example, participants 1 and 2 have the same "True Acquiescence Bias Extreme and Anxiety" levels (this is another "all-seeing" example) but Moderate Responding participant 1 has a higher observed anxiety score Social Desirability Malingering Random Responding • From our all-seeing vantage point, we can see this is Guessing because participant 1 adopted an "extreme" extremity bias Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Extremity bias can also obscure true differences amongst Bias respondents’ construct levels Types of Response • Consider this time participants 2 and 3 Biases Acquiescence Bias Extreme and • These participants have different "True Anxiety" levels but Moderate Responding identical observed anxiety scores Social Desirability Malingering • Random Responding This is because participant 2 is reluctant to use the more Guessing extreme response option required to warrant her true anxiety Methods For level Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au • These types of problems can produce results that lead to Response Bias inaccurate research conclusions Types of • Response For example, from our all-knowing vantage point we know Biases that the correlation between "True Anxiety" levels and "True Acquiescence Bias Extreme and Spirituality" levels is r = –.04 Moderate Responding Social Desirability • However, based on the data (Total STAI vs. Total S/R) this Malingering Random Responding correlation is r = .36 Guessing
Methods For • Thus, the extremity problem can foster incorrect research Coping With Response conclusions Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au • These types of problems can produce results that lead to Response Bias inaccurate research conclusions Types of • Response For example, from our all-knowing vantage point we know Biases that the correlation between "True Anxiety" levels and "True Acquiescence Bias Extreme and Spirituality" levels is r = –.04 Moderate Responding Social Desirability • However, based on the data (Total STAI vs. Total S/R) this Malingering Random Responding correlation is r = .36 Guessing
Methods For • Thus, the extremity problem can foster incorrect research Coping With Response conclusions Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone @uwa.edu.au • These types of problems can produce results that lead to Response Bias inaccurate research conclusions Types of • Response For example, from our all-knowing vantage point we know Biases that the correlation between "True Anxiety" levels and "True Acquiescence Bias Extreme and Spirituality" levels is r = –.04 Moderate Responding Social Desirability • However, based on the data (Total STAI vs. Total S/R) this Malingering Random Responding correlation is r = .36 Guessing
Methods For • Thus, the extremity problem can foster incorrect research Coping With Response conclusions Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone • Bear in mind that the use of an extreme response option is @uwa.edu.au not in itself a problem Response Bias • We want test takers to use response options that reflect their Types of construct levels—this sometimes requires extreme Response Biases responding Acquiescence Bias Extreme and • Moderate It becomes a problem when: Responding Social Desirability 1 people with identical construct levels differ in their Malingering Random Responding tendency to use moderate and extreme response Guessing options Methods For Coping With 2 when people with different construct levels do not differ Response Bias in their willingness to use moderate and extreme Managing Testing Context response options Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone • Bear in mind that the use of an extreme response option is @uwa.edu.au not in itself a problem Response Bias • We want test takers to use response options that reflect their Types of construct levels—this sometimes requires extreme Response Biases responding Acquiescence Bias Extreme and • Moderate It becomes a problem when: Responding Social Desirability 1 people with identical construct levels differ in their Malingering Random Responding tendency to use moderate and extreme response Guessing options Methods For Coping With 2 when people with different construct levels do not differ Response Bias in their willingness to use moderate and extreme Managing Testing Context response options Managing Test Content Specialised Tests
[email protected] Psychological Measurement Extreme and Moderate Responding
Psychological Measurement mark.hurlstone • Bear in mind that the use of an extreme response option is @uwa.edu.au not in itself a problem Response Bias • We want test takers to use response options that reflect their Types of construct levels—this sometimes requires extreme Response Biases responding Acquiescence Bias Extreme and • Moderate It becomes a problem when: Responding Social Desirability 1 people with identical construct levels differ in their Malingering Random Responding tendency to use moderate and extreme response Guessing options Methods For Coping With 2 when people with different construct levels do not differ Response Bias in their willingness to use moderate and extreme Managing Testing Context response options Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement • Social desirability bias (also known as faking good) is the mark.hurlstone @uwa.edu.au tendency for a person to respond in a way that seems socially appealing, regardless of his or her true Response Bias characteristics Types of Response • Recall the item, "Have you ever stolen anything from your Biases employer?" Acquiescence Bias Extreme and Moderate • Responding If you wanted the job, you would be tempted to provide Social Desirability responses that would appeal to the employer Malingering Random Responding Guessing • That is, you would want to appear to have integrity, high Methods For levels of conscientiousness, and honesty Coping With Response • Bias If you do respond in such a way—and you do not actually Managing Testing possess these characteristics—you are exhibiting a social Context Managing Test desirability response bias Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The textbook says that social desirability bias "can diminish Bias the reliability and validity of the measurement process" Types of Response Biases • Social desirability bias does indeed compromise the validity Acquiescence Bias of test score interpretations Extreme and Moderate Responding • Social Desirability However, it is unlikely to affect reliability Malingering Random Responding • This is because people would be expected to engage in Guessing socially desirable responding across all items, consistently Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • Types of Social desirability bias can be affected by at least three Response sources: Biases Acquiescence Bias Extreme and 1 Test content Moderate Responding 2 Test context Social Desirability Malingering 3 Personality of the respondent Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • Types of Social desirability bias can be affected by at least three Response sources: Biases Acquiescence Bias Extreme and 1 Test content Moderate Responding 2 Test context Social Desirability Malingering 3 Personality of the respondent Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Test Content
Psychological Measurement mark.hurlstone @uwa.edu.au • Some psychological constructs have greater implications for Response social appeal than do others Bias
Types of • For example, personality characteristics such as Response Biases psychological well being (vs. psychological distress) or Acquiescence Bias honesty (vs. deceitfulness) might be closely linked to social Extreme and Moderate desirability Responding Social Desirability Malingering • Well-being and honesty are probably viewed as more Random Responding Guessing socially desirable than distress and deceitfulness Methods For • Coping With Other characteristics—such as extraversion (vs. Response introversion)—might be less affected by social desirability Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • Types of Social desirability bias can be affected by at least three Response sources: Biases Acquiescence Bias Extreme and 1 Test content Moderate Responding 2 Test context Social Desirability Malingering 3 Personality of the respondent Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • Types of Social desirability bias can be affected by at least three Response sources: Biases Acquiescence Bias Extreme and 1 Test content Moderate Responding 2 Test context Social Desirability Malingering 3 Personality of the respondent Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Test Context
Psychological Measurement • Socially desirable responding might be more likely to occur mark.hurlstone @uwa.edu.au in contexts in which respondents can be identified than in contexts in which they are anonymous Response Bias • When respondents can be linked to their responses, they Types of Response might be more likely to provide responses that are socially Biases appealing Acquiescence Bias Extreme and Moderate • Responding Additionally, socially desirable responding may be more likely Social Desirability in testing contexts where important consequences hinge on Malingering Random Responding testing outcomes Guessing Methods For • The hiring example is a case in point Coping With Response • Bias Socially desirable responding is less of a problem when Managing Testing there are no important testing consequences (though it is Context Managing Test still a problem) Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • Types of Social desirability bias can be affected by at least three Response sources: Biases Acquiescence Bias Extreme and 1 Test content Moderate Responding 2 Test context Social Desirability Malingering 3 Personality of the respondent Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • Types of Social desirability bias can be affected by at least three Response sources: Biases Acquiescence Bias Extreme and 1 Test content Moderate Responding 2 Test context Social Desirability Malingering 3 Personality of the respondent Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Personality
Psychological Measurement mark.hurlstone @uwa.edu.au • Another source of socially desirable responding is the personality of the test taker Response Bias • Some people have a personality type that disposes them to Types of engaging in socially desirable responding Response Biases Acquiescence Bias • For example, individual differences in the "need for Extreme and Moderate autonomy" are known to correlate negatively with socially Responding Social Desirability desirable responding Malingering Random Responding • Guessing People high in need for autonomy are less sensitive to social
Methods For disapproval Coping With Response • Thus, they score high in need for autonomy and low in Bias Managing Testing socially desirable responding Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Implications
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response • Social desirability bias has implications for: Biases Acquiescence Bias 1 applied psychology Extreme and Moderate Responding 2 behavioural research Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Implications
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response • Social desirability bias has implications for: Biases Acquiescence Bias 1 applied psychology Extreme and Moderate Responding 2 behavioural research Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Applied Psychology
Psychological Measurement mark.hurlstone @uwa.edu.au • Many personality tests are "transparent" in that it is clear
Response what types of qualities they are measuring (high face validity) Bias • In these tests, it is easy to "fake" having desirable qualities Types of Response Biases • Such faking behaviour can be quite common, especially in Acquiescence Bias Extreme and the context of job applications Moderate Responding Social Desirability • If applicants exaggerate their positive qualities and minimise Malingering Random Responding their negative qualities, then hiring decisions can be Guessing compromised Methods For Coping With • When faking takes place, a person hired under a testing Response Bias based selection process is likely to be a "faker" Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Implications
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response • Social desirability bias has implications for: Biases Acquiescence Bias 1 applied psychology Extreme and Moderate Responding 2 behavioural research Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Implications
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response • Social desirability bias has implications for: Biases Acquiescence Bias 1 applied psychology Extreme and Moderate Responding 2 behavioural research Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Behavioural Research
Psychological Measurement mark.hurlstone • Socially desirable responding can have a similar effect on @uwa.edu.au correlations between scores as was demonstrated for the Response case of acquiescence Bias
Types of • The textbook mentions the example of positive affect and Response Biases relationship quality Acquiescence Bias Extreme and • People who tend to engage in socially desirable responding Moderate Responding will report that they have higher levels of positive affect than Social Desirability Malingering they in fact do Random Responding Guessing • Similarly, they will also rate the quality of their relationships Methods For Coping With as better than they in fact are Response Bias • This causes the correlation between the two scores to be Managing Testing Context higher than it really is (see Table 10.3, p.283) Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: The Nature of Socially Desirable Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Del Paulhus has done much work exploring social Bias desirability bias as an aspect of personality Types of Response • An implication of his research is that there may be multiple Biases Acquiescence Bias forms of social desirability bias Extreme and Moderate Responding • He proposed two processes by which socially desirable Social Desirability Malingering responding occurs: Random Responding Guessing 1 Impression management Methods For 2 Self-deception Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: The Nature of Socially Desirable Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Del Paulhus has done much work exploring social Bias desirability bias as an aspect of personality Types of Response • An implication of his research is that there may be multiple Biases Acquiescence Bias forms of social desirability bias Extreme and Moderate Responding • He proposed two processes by which socially desirable Social Desirability Malingering responding occurs: Random Responding Guessing 1 Impression management Methods For 2 Self-deception Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Impression Management
Psychological Measurement mark.hurlstone @uwa.edu.au • This is a conscious process, where test takers intentionally Response Bias attempt to appear socially desirable Types of • For example, a job applicant might exaggerate their Response Biases desirable qualities and minimise their undesirable qualities Acquiescence Bias Extreme and Moderate • Impression management is thought to be a psychological Responding Social Desirability state that depends on situational demands Malingering Random Responding • Guessing Thus, conscious efforts to appear socially desirable usually
Methods For arise in particular contexts (e.g., completing a personality Coping With inventory for a job interview) Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: The Nature of Socially Desirable Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Del Paulhus has done much work exploring social Bias desirability bias as an aspect of personality Types of Response • An implication of his research is that there may be multiple Biases Acquiescence Bias forms of social desirability bias Extreme and Moderate Responding • He proposed two processes by which socially desirable Social Desirability Malingering responding occurs: Random Responding Guessing 1 Impression management Methods For 2 Self-deception Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: The Nature of Socially Desirable Responding
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Del Paulhus has done much work exploring social Bias desirability bias as an aspect of personality Types of Response • An implication of his research is that there may be multiple Biases Acquiescence Bias forms of social desirability bias Extreme and Moderate Responding • He proposed two processes by which socially desirable Social Desirability Malingering responding occurs: Random Responding Guessing 1 Impression management Methods For 2 Self-deception Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Social Desirability: Self-Deception
Psychological Measurement mark.hurlstone @uwa.edu.au • This is thought to be an unconscious process Response Bias • People who engage in self-deception tend to actually believe Types of their overestimation of their psychological characteristics Response Biases • Acquiescence Bias It is correlated positively with narcissism Extreme and Moderate Responding • Unlike impression management, self-deception is thought to Social Desirability reflect a psychological trait that is stable across time and Malingering Random Responding context Guessing Methods For • Thus, some people are naturally more predisposed toward Coping With Response self-deception than others Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Malingering
Psychological Measurement mark.hurlstone • This is the opposite problem to social desirability @uwa.edu.au • Respondents might sometimes try to exaggerate their Response Bias psychological problems Types of Response • For example, in some applied testing contexts, test takers Biases Acquiescence Bias may be motivated to appear more cognitively impaired or Extreme and Moderate psychologically disturbed than they truly are Responding Social Desirability • This is known as or Malingering malingering faking bad Random Responding Guessing • Malingering is particularly problematic in testing situations Methods For where individuals stand to benefit from their faking behaviour Coping With Response Bias • This includes criminal competency hearings, worker Managing Testing Context compensation claims, and personal injury examinations Managing Test Content Specialised Tests
[email protected] Psychological Measurement Malingering
Psychological Measurement mark.hurlstone • Malingering compromises the quality of psychological @uwa.edu.au assessment Response Bias • For example, suppose an individual involved in a car Types of accident takes a neuropsychological assessment that Response Biases includes measures of his attention and memory Acquiescence Bias Extreme and • Moderate If the individual deliberately tries to perform poorly on these Responding Social Desirability measures—to reap financial benefits—then the accuracy Malingering and fairness of the diagnosis and decisions made on the Random Responding Guessing basis of those measures will be compromised Methods For Coping With • Failure to consider the impact of malingering potentially Response Bias carries high costs for insurers, disability systems, and society Managing Testing Context at large Managing Test Content Specialised Tests
[email protected] Psychological Measurement Malingering
Psychological Measurement mark.hurlstone @uwa.edu.au • Malingering is common in applied testing situations Response Bias • It occurs in between 7% to 27% of general psychological Types of evaluations and between 31% to 45% of forensic evaluations Response Biases (e.g., criminal competence, disability hearings etc.) Acquiescence Bias Extreme and Moderate • There is in fact evidence that lawyers coach clients in Responding Social Desirability methods to detect malingering Malingering Random Responding • Guessing Malingering is therefore a legitimate concern in psychological
Methods For testing, especially in situations where test takers stand to Coping With reap benefits by exaggerating their psychological problems Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Careless or Random Responding
Psychological Measurement mark.hurlstone @uwa.edu.au • Whether due to carelessness or lack of motivation, sometime Response test takers respond randomly or semi-randomly Bias
Types of • For example, an individual taking a test in which the items Response Biases are answered on a 5-point scale (1 = strongly disagree, 5 = Acquiescence Bias strongly agree) might cycle through the response scale: Extreme and Moderate Responding • That is, select strongly disagree for Item 1, disagree for Item Social Desirability Malingering 2, neutral for Item 3, agree for Item 4, strongly agree for Item Random Responding Guessing 5, and then wrap back around for item 6 Methods For • Coping With This produces scores that are meaningless with regards the Response construct being measured Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Types of Response Biases
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • The response biases that will be discussed in this lecture Bias are: Types of Response Biases • Acquiescence Bias Acquiescence Bias • Extreme and Moderate Responding Extreme and Moderate • Responding Social Desirability Social Desirability • Malingering Malingering Random Responding • Random Responding Guessing • Guessing Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Guessing
Psychological Measurement mark.hurlstone @uwa.edu.au
Response • Bias Aptitude and achievement tests are designed so that specific
Types of responses are correct and others incorrect Response Biases • Scores on these tests are often consequential in that they Acquiescence Bias Extreme and are used for some form of selection (e.g., college admission) Moderate Responding Social Desirability • Respondents may be motivated to guess on such tests, Malingering Random Responding especially if they are based on a limited set of options (e.g., Guessing multiple-choice questions) Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Guessing
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • Guessing can compromise the quality of test scores Types of Response • A correct guess artificially inflates a test taker’s score relative Biases Acquiescence Bias to his or her true score Extreme and Moderate Responding • Guessing therefore produces test scores that are Social Desirability Malingering inconsistent with the true score differences, which has an Random Responding adverse effect on test reliability Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Testing Context
Psychological Measurement mark.hurlstone • Perhaps the best way to cope with response biases is to @uwa.edu.au prevent them occurring in the first place Response Bias • The occurrence of response biases can be minimised by
Types of managing the testing context Response Biases • For example, respondents may be less likely to engage in Acquiescence Bias Extreme and socially desirable responding, if they are convinced that they Moderate Responding are responding anonymously Social Desirability Malingering Random Responding • The rationale is that anonymity allows respondents to feel Guessing comfortable in honestly admitting to undesirable attributes Methods For Coping With (about themselves or others) Response Bias • The downside is that although anonymity increases honest Managing Testing Context responding, it may also increase random responding Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Testing Context
Psychological Measurement mark.hurlstone @uwa.edu.au • When people are tired, they may increase biased responding Response • Bias Another strategy therefore is to create a testing situation that minimises respondent fatigue, stress, distraction, or Types of Response frustration Biases Acquiescence Bias • Extreme and A final strategy is known as the "bogus pipeline Moderate Responding technique"—inform participants that false responding can be Social Desirability Malingering detected by the test that is being administered Random Responding Guessing • The mere possibility of detection can convince some Methods For respondents to be more honest and unbiased Coping With Response Bias • This strategy is a potential solution to malingering Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Test Content
Psychological Measurement mark.hurlstone @uwa.edu.au • Test developers sometimes choose specific kinds of items or
Response specific kinds of response formats to help minimize some Bias biases: Types of Response 1 Write clear, concise, and unambiguous items Biases Acquiescence Bias • this reduces frustration and distraction that might cause Extreme and Moderate low motivation and biased responding Responding Social Desirability 2 Write items that are neutral in terms of social Malingering Random Responding desirability: Guessing • instead of "I am a hostile person" Methods For Coping With • use "I am sometimes less friendly than other people" Response Bias 3 Forced choice-formats Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Test Content
Psychological Measurement mark.hurlstone @uwa.edu.au • Test developers sometimes choose specific kinds of items or
Response specific kinds of response formats to help minimize some Bias biases: Types of Response 1 Write clear, concise, and unambiguous items Biases Acquiescence Bias • this reduces frustration and distraction that might cause Extreme and Moderate low motivation and biased responding Responding Social Desirability 2 Write items that are neutral in terms of social Malingering Random Responding desirability: Guessing • instead of "I am a hostile person" Methods For Coping With • use "I am sometimes less friendly than other people" Response Bias 3 Forced choice-formats Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Test Content
Psychological Measurement mark.hurlstone @uwa.edu.au • Test developers sometimes choose specific kinds of items or
Response specific kinds of response formats to help minimize some Bias biases: Types of Response 1 Write clear, concise, and unambiguous items Biases Acquiescence Bias • this reduces frustration and distraction that might cause Extreme and Moderate low motivation and biased responding Responding Social Desirability 2 Write items that are neutral in terms of social Malingering Random Responding desirability: Guessing • instead of "I am a hostile person" Methods For Coping With • use "I am sometimes less friendly than other people" Response Bias 3 Forced choice-formats Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Test Content
Psychological Measurement mark.hurlstone @uwa.edu.au • Test developers sometimes choose specific kinds of items or
Response specific kinds of response formats to help minimize some Bias biases: Types of Response 1 Write clear, concise, and unambiguous items Biases Acquiescence Bias • this reduces frustration and distraction that might cause Extreme and Moderate low motivation and biased responding Responding Social Desirability 2 Write items that are neutral in terms of social Malingering Random Responding desirability: Guessing • instead of "I am a hostile person" Methods For Coping With • use "I am sometimes less friendly than other people" Response Bias 3 Forced choice-formats Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Test Content
Psychological Measurement • Test developers have used "forced-choice" formats to mark.hurlstone @uwa.edu.au minimise the existence of social desirability bias
Response • Forced-choice items are items that present two Bias characteristics and require that respondents endorse one Types of Response and only one of them Biases Acquiescence Bias • For example, an item on a personality test might present the Extreme and Moderate Responding two equally socially desirable characteristics "friendly" and Social Desirability "assertive" and test takers choose the one that best Malingering Random Responding describes them personally Guessing Methods For • Another item might include characteristics that are equally Coping With Response undesirable (e.g., "timid" and "argumentative") Bias Managing Testing • The forced-choice format prevents test takers from simply Context Managing Test picking the more desirable choice Content Specialised Tests
[email protected] Psychological Measurement Minimising the Existence of Bias by Managing Test Content or Scoring
Psychological Measurement mark.hurlstone @uwa.edu.au • Response Test developers can also design tests to minimise the extent Bias of the extreme responding problem Types of Response • For example, they can provide only two responses for each Biases Acquiescence Bias item Extreme and Moderate Responding • On a personality test, a test taker might be asked to respond Social Desirability Malingering "Yes" or "No" to whether they consider themselves "friendly" Random Responding Guessing • Such a format eliminates extremity bias by removing any Methods For "extreme" response options altogether Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Effects of Bias by Managing Test Content or Scoring
Psychological Measurement mark.hurlstone @uwa.edu.au • Response In reality, it is very difficult to eliminate the existence of Bias response bias Types of Response • Instead, test developers typically try to minimise the effects Biases Acquiescence Bias of response bias Extreme and Moderate Responding • A common method used to help minimise the effects of Social Desirability Malingering acquiescence bias is to create balanced scales Random Responding Guessing • A balanced scale consists of an equal number of positively Methods For keyed and negatively keyed items Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Effects of Bias by Managing Test Content or Scoring
Psychological Measurement mark.hurlstone @uwa.edu.au • Recall the job satisfaction scale presented in the discussion Response Bias of acquiescence bias: Types of Response 1 "I really enjoy my work" Biases 2 "I find my work personally fulfilling" Acquiescence Bias Extreme and 3 "In general, I am satisfied with the day-to-day aspects of Moderate Responding my job Social Desirability Malingering 4 "There is very little that I would change about my job" Random Responding Guessing • All four items are positively keyed, which might encourage Methods For Coping With acquiescence bias Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Effects of Bias by Managing Test Content or Scoring
Psychological Measurement • To minimise the potential effects of acquiescence bias, we mark.hurlstone @uwa.edu.au might use the following revised set of items: 1 "I really enjoy my work" Response Bias 2 "I do not find my work personally fulfilling" Types of 3 "In general, I am satisfied with the day-to-day aspects of Response Biases my job Acquiescence Bias 4 "There is much I would change about my job" Extreme and Moderate Responding • Social Desirability The revised scale is balanced—it contains equal numbers of Malingering positively keyed and negatively keyed statements Random Responding Guessing • Balanced scales allow a test user to identify an acquiescent Methods For Coping With responder Response Bias • Such a responder would agree with all four items, despite Managing Testing Context the fact that some agreements indicate a high level of Managing Test Content satisfaction, but others indicate a low level Specialised Tests
[email protected] Psychological Measurement Minimising the Effects of Bias by Managing Test Content or Scoring
Psychological Measurement mark.hurlstone @uwa.edu.au • Note that there are some practicalities when it comes to
Response analysing balanced scales Bias • Before you can do any analyses, you must "reverse-score" Types of Response each of the negatively keyed items Biases Acquiescence Bias • Extreme and That is, you want high scores to be indicative of the attribute Moderate Responding of interest Social Desirability Malingering • Random Responding In this example, someone who responded a "strongly Guessing disagree" to Item 2 would get a score of 7 for that item Methods For (rather than 1) Coping With Response Bias • The same goes for item 4 Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Minimising the Effects of Bias by Managing Test Content or Scoring
Psychological Measurement mark.hurlstone @uwa.edu.au • Test users can also use specialised scoring procedures to Response minimise guessing Bias Types of • Achievement tests like the SAT adjust test scores to account Response Biases for guessing Acquiescence Bias Extreme and • Moderate Specifically, in the SAT an item answered incorrectly results Responding Social Desirability in a subtraction of .25 of a point Malingering Random Responding • An item left unanswered results in no such points deduction Guessing Methods For • This differential scoring procedure discourages guessing Coping With Response behaviour amongst SAT test takers Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Managing Test Content to Detect Bias and Intervene
Psychological Measurement mark.hurlstone @uwa.edu.au • Some inventories incorporate so-called validity scales
Response • Validity scales are designed to specifically measure the Bias degree to which people may be engaging in a particular type Types of of response bias Response Biases Acquiescence Bias • Perhaps the most famous self-report inventory is the Millon Extreme and Moderate Clinical Multiaxial Inventory (MMPI) Responding Social Desirability Malingering • The MMPI is used by psychiatrists and psychologists to Random Responding Guessing measure personality (but more along the psychopathological
Methods For ends of the spectrum) Coping With Response • To help determine whether the respondent’s scores are Bias Managing Testing interpretable, they consult the validity scales Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Managing Test Content to Detect Bias and Intervene
Psychological Measurement • The two main validity scales within the MMPI are the L scale mark.hurlstone @uwa.edu.au and the F scale • Response The L(ie) scale: Bias • consists of 15-items that describe "minor flaws and Types of Response weaknesses to which most people are willing to admit" Biases • Acquiescence Bias a high score represents denial of these common Extreme and Moderate characteristics which is indicative of social desirability Responding Social Desirability bias Malingering Random Responding • The F (Infrequency) scale: Guessing Methods For • consists of 64-items that are endorsed by very few Coping With Response respondents Bias • a high score represents some form of deviant Managing Testing Context responding—random responding, malingering, Managing Test Content acquiescence, or serious psychopathology Specialised Tests
[email protected] Psychological Measurement Managing Test Content to Detect Bias and Intervene
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias • The MMPI gives instructions one what to do if someone has
Types of elevated scores on the validity scales: Response Biases • In some cases, you might totally exclude any Acquiescence Bias Extreme and interpretation of the scores Moderate Responding • In others, you might interpret them cautiously Social Desirability Malingering • Random Responding Alternatively, you might use statistical procedures to Guessing "correct" the original scores into more valid scores Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Methods For Coping With Response Bias
Psychological Measurement mark.hurlstone @uwa.edu.au • There are at least three general kinds of strategies: Response Bias 1 Manage the testing context Types of 2 Manage the test content and/or scoring Response Biases 3 Use specially designed "bias" tests Acquiescence Bias Extreme and Moderate • Additionally, there at least three goals that these strategies Responding Social Desirability are intended to accomplish: Malingering Random Responding 1 Minimize the existence of response biases Guessing 2 Methods For Minimize the effects of responses biases Coping With 3 Detect bias and intervene in some way Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Using Specialised Tests to Detect Bias and Intervene
Psychological Measurement mark.hurlstone • @uwa.edu.au Unlike the validity scales found in a test such as the MMPI, there are also tests dedicated to the measurement of socially Response desirable responding Bias Types of • Two of the most famous tests designed for this purpose are Response Biases the Marlowe-Crowne Social Desirability Scale and the Acquiescence Bias Extreme and Balanced Inventory for Desirable Responding (see the Moderate Responding textbook for a description p.297) Social Desirability Malingering • These scales exist independently of any other scale, so they Random Responding Guessing can be used in a variety of contexts (applied or research) Methods For Coping With • They can be used in conjunction with other tests to establish Response Bias the degree to which social desirability bias may have Managing Testing Context affected scores on those tests Managing Test Content Specialised Tests
[email protected] Psychological Measurement Using Specialised Tests to Detect Bias and Intervene
Psychological Measurement mark.hurlstone @uwa.edu.au • The Dot Counting Test is an example of test specifically designed to detect malingered cognitive impairment Response Bias • Consists of 12 cards with half of them containing dots Types of presented in a grouped fashion and the other half presented Response Biases in a random fashion Acquiescence Bias Extreme and Moderate • Malingering is suspected when it takes people just as long to Responding Social Desirability count the grouped dots as the random dots Malingering Random Responding • Guessing Note that the cognitive competencies required to perform
Methods For this task are so primitive they should be present in all but the Coping With most severely brain injured patients Response Bias Managing Testing • This is why it constitutes a measure of malingering Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Using Specialised Tests to Detect Bias and Intervene
Psychological Measurement mark.hurlstone @uwa.edu.au • The Dot Counting Test is an example of test specifically designed to detect malingered cognitive impairment Response Bias • Consists of 12 cards with half of them containing dots Types of presented in a grouped fashion and the other half presented Response Biases in a random fashion Acquiescence Bias Extreme and Moderate • Malingering is suspected when it takes people just as long to Responding Social Desirability count the grouped dots as the random dots Malingering Random Responding • Guessing Note that the cognitive competencies required to perform
Methods For this task are so primitive they should be present in all but the Coping With most severely brain injured patients Response Bias Managing Testing • This is why it constitutes a measure of malingering Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Using Specialised Tests to Detect Bias and Intervene
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Acquiescence Bias Extreme and Moderate Responding Social Desirability Malingering Random Responding Tese types of cards Guessing should take less time to Methods For Coping With count Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Using Specialised Tests to Detect Bias and Intervene
Psychological Measurement mark.hurlstone @uwa.edu.au • The Dot Counting Test is an example of test specifically designed to detect malingered cognitive impairment Response Bias • Consists of 12 cards with half of them containing dots Types of presented in a grouped fashion and the other half presented Response Biases in a random fashion Acquiescence Bias Extreme and Moderate • Malingering is suspected when it takes people just as long to Responding Social Desirability count the grouped dots as the random dots Malingering Random Responding • Guessing Note that the cognitive competencies required to perform
Methods For this task are so primitive they should be present in all but the Coping With most severely brain injured patients Response Bias Managing Testing • This is why it constitutes a measure of malingering Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement Terminology: Response Sets vs. Response Styles
Psychological Measurement mark.hurlstone @uwa.edu.au • Response Sets: Response • Are temporary and due to factors such as Bias circumstances surrounding the testing or the test itself Types of Response Biases • Response Styles: Acquiescence Bias Extreme and • Are considered relatively stable and enduring traits that Moderate Responding are observed across tests and testing situations Social Desirability Malingering • Random Responding Some people are more concerned about appearing Guessing socially desirable than others Methods For Coping With • Response biases can arise from response sets and Response Bias response styles Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement References
Psychological Measurement mark.hurlstone @uwa.edu.au
Response Bias
Types of Response Biases Furr, M. R., & Bacharach, V. R. (2014; Chapter 10). Acquiescence Bias Extreme and Psychometrics: An Introduction (second edition). Sage. Moderate Responding Social Desirability Malingering Random Responding Guessing
Methods For Coping With Response Bias Managing Testing Context Managing Test Content Specialised Tests
[email protected] Psychological Measurement