C O N F I D E N T I A L Review of and Recommendations Regarding Pittsburgh Bureau of Police Entry-Level Hiring Practices Prepared by: Leaetta Hough, Ph.D. The Dunnette Group, Ltd. 370 Summit Avenue St. Paul, MN 55102 651-227-4888 [email protected] Submitted to: City of Pittsburgh & Plaintiffs’ Counsel [Pennsylvania Civil Liberties Union (PACLU) & Feinstein Doyle Payne & Kravec, LLC (FDPK]) February 19, 2014 Table of Contents Topic Page No. Chapter 1: Circumstances and Content of Report ........................................................1 Chapter 2: Sources of Information .................................................................................5 Chapter 3: Required Applicant Characteristics – What Should Be Measured in the Pittsburgh BOP Entry-Level Police Officer Selection System? ..........17 BOP Mission Statement, Values Statement, and Standard Field Evaluation Guidelines ..................................................................................................19 More Formal, Empirical Job Analyses ..........................................................22 Development of Preliminary Duty/Task List .............................................23 Final Duty/Task List ..................................................................................24 Importance Ratings of Job Duties/Tasks. ..................................................25 Identification of Physical Requirements of Police Work ..........................26 Identification of Required Cognitive Abilities ..........................................26 Identification of Required Personality Characteristics ..............................30 Relative Weight of Characteristics ............................................................33 Overall Summary of Concerns of the Formal Job Analysis ......................34 Criterion-related Validity Research .................................................................35 Cognitive Abilities .....................................................................................36 Personality Characteristics .........................................................................37 Summary ..........................................................................................................40 Chapter 4: Quality of the Measurement System – How Well Does the Current Selection System Measure What It Purports to Measure and What Is Its Impact on African Americans? ....................................................................43 Applicant Preparation Materials .....................................................................44 Written Exam – Law Enforcement Applicant Battery (LEAB) ......................45 LEAB Cognitive Ability Test (CAT) ........................................................45 Impact of LEAB Cognitive Ability Test (CAT) on Protected Classes ......50 LEAB Non-cognitive Measure: Life Experiences Survey (LES) .............57 Impact of LEAB LES on African Americans ............................................63 LEAB Non-cognitive Measure: Work Styles Questionnaire ....................67 Impact of LEAB WSQ on African Americans ..........................................71 LEAB Composite (aka Written Exam) ......................................................75 Oral Board Exam ............................................................................................77 i Oral Board Examiner Training ..................................................................82 Impact of the Oral Board on African Americans .......................................84 LEAB & ORAL Board Composite Score .......................................................85 Description .................................................................................................85 Impact of LEAB/ORAL Board Composite on African Americans ...........87 MPOETC Fitness and Reading Tests .............................................................89 Nelson-Denny Reading Test ......................................................................89 Physical Fitness Training ...........................................................................90 Impact on Protected Classes ......................................................................91 Background Investigation Phase .....................................................................91 Pre-Polygraph Procedures ..........................................................................92 Polygraph Procedures ................................................................................93 Drug Testing ..............................................................................................94 Background Interviews ..............................................................................94 Background File Preparation .....................................................................95 Impact on Protected Classes ......................................................................96 Chief’s Roundtable .........................................................................................97 Impact of Chief’s Roundtable on Protected Classes ..................................99 Psychological Exam Conducted by Licensed Psychologist .........................100 Medical Exam ...............................................................................................101 Analysis of Overall Impact of Selection System on Protected Classes ........101 Chapter 5: Job Relatedness (Validity) of the Current Selection System ..................105 LEAB Written Exam ....................................................................................105 Oral Board ....................................................................................................107 MPOETC Physical Fitness and Reading Tests .............................................108 Background Investigation .............................................................................108 Chief’s Roundtable .......................................................................................108 Summary .......................................................................................................109 Chapter 6: Summary of Evaluation and Recommendations .....................................111 Update the Job Analysis ...............................................................................112 Improve Applicant Preparation Materials and Classes .................................112 Develop a Construct-oriented Measurement Plan for Entire Selection Process .....................................................................................................113 Improve the Tests .........................................................................................114 Improve the Rating Processes .......................................................................115 Conclusion ....................................................................................................116 ii Tables Table 1: Documents Examined ........................................................................................7 Table 2: List of Interviewees and Their Roles ...............................................................15 Table 3: Ethnic Background and Gender of Interviewees .............................................16 Table 4: Field Training Officer Criteria for Evaluating Probationary Officers .............20 Table 5: Absolute and Relative Importance of Job Duties for Overall Job Performance..................................................................................25 Table 6: Relative Importance of Physical Abilities for Overall Job Performance .........27 Table 7: Relative Importance of Cognitive Abilities for Overall Job Performance ......29 Table 8: Relative Importance of Personality Characteristics for Overall Job Performance .....................................................................................................31 Table 9: Relative Importance of Cognitive Abilities, Physical Abilities, and Personality Characteristics for Overall Job Performance ................................34 Table 10: Meta-analytic Evidence of the Usefulness of Personality Variables for Predicting Important Police Officer Outcomes................................................39 Table 11: Sample LEAB Cognitive Ability Test (CAT) Item .........................................48 Table 12: Adverse Impact Analysis of LEAB Cognitive Ability Test (CAT) – 2011 Test Administration Data ........................................................................51 Table 13: Adverse Impact Analysis of LEAB Cognitive Ability Test (CAT) – 2008 Test Administration Data ........................................................................52 Table 14: Adverse Impact Analysis of LEAB Cognitive Ability Test (CAT) – 2006 Test Administration Data ........................................................................53 Table 15: Mean Score Differences between Whites and African Americans on Specific CAT Ability Tests and the Effect Sizes of Those Differences ........................56 Table 16: LEAB Life Experience Survey (LES) Initial Scales, Definitions, and Reliabilities ......................................................................................................58 Table 17: Comparison of LEAB Life Experience Survey (LES) Initial and Current Scales ...............................................................................................................64 Table 18: Mean Score Differences between Whites and African Americans on LES Scales & Effect Sizes of Those Differences–2013