Technical Manual for the Praxis® Tests and Related Assessments October 2020 Technical Manual for the Praxis® Tests and Related Assessments Copyright © 2020 by Educational Testing Service. All rights reserved. E T S, the E T S logo, PRAXIS, e-rater, and MEASURING THE POWER OF LEARNING are registered trademarks of Educational Testing Service. c-rater is a trademark of Educational Testing Service. All other trademarks (and service marks) are the property of their respective owners. Technical Manual for the Praxis® Tests and Related Assessments Table of Contents Preface ...................................................................................................................................................... 6 Purpose of This Manual ........................................................................................................................ 6 Audience ............................................................................................................................................... 6 Purpose of the Praxis® Assessments ......................................................................................................... 7 Overview ............................................................................................................................................... 7 The Praxis Core Academic Skills for Educators Tests ......................................................................... 8 The Praxis Subject Assessments — Subject Knowledge and Pedagogical Knowledge Related to Teaching ................................................................................................................................................ 8 The School Leadership Series Assessments ......................................................................................... 8 How the Praxis Assessments Address States’ Needs ........................................................................... 9 Assessment Development ....................................................................................................................... 10 Fairness in Test Development ............................................................................................................. 10 Test Development Standards .............................................................................................................. 10 Validity ............................................................................................................................................... 11 The Nature of Validity Evidence .................................................................................................. 11 Content-related Validity Evidence. ............................................................................................... 12 Validity Maintenance .................................................................................................................... 13 Test Development Process .................................................................................................................. 14 Development of Test Specifications ............................................................................................. 16 Facilitate Committee Meetings ..................................................................................................... 16 Development of Test Items and Reviews ..................................................................................... 16 Assembly of Test Forms and Review ........................................................................................... 16 Administer the Test ....................................................................................................................... 16 Perform Statistical Analysis .......................................................................................................... 17 Review Processes ................................................................................................................................ 17 ETS Standards for Quality and Fairness ....................................................................................... 17 ETS Fairness Review .................................................................................................................... 17 Test Adoption Process ............................................................................................................................ 18 Process Overview................................................................................................................................ 18 The Praxis® Core Academic Skills for Educators Tests ............................................................... 18 The Praxis® Subject Assessments ................................................................................................. 18 3 Technical Manual for the Praxis® Tests and Related Assessments Analysis of States’ Needs ................................................................................................................... 21 Standard-Setting Studies ..................................................................................................................... 21 Panel Formation ............................................................................................................................ 21 Typical Standard Setting Methods ................................................................................................ 22 Standard-Setting Reports .............................................................................................................. 22 Psychometric Properties ......................................................................................................................... 23 Introduction ......................................................................................................................................... 23 Test-Scoring Process ........................................................................................................................... 23 Item Analyses...................................................................................................................................... 24 Classical Item Analyses ................................................................................................................ 24 Speededness .................................................................................................................................. 26 Differential Item Functioning (DIF) Analyses ................................................................................... 27 DIF Statistics ................................................................................................................................. 28 Test-Form Equating ............................................................................................................................ 29 Overview ....................................................................................................................................... 29 Scaling........................................................................................................................................... 29 Equating ........................................................................................................................................ 30 The NEAT Design ........................................................................................................................ 30 The Equivalent Groups Design ..................................................................................................... 31 The Single Group Design.............................................................................................................. 31 The SiGNET Design ..................................................................................................................... 32 The ISD Design............................................................................................................................. 33 Equating Methodology Summary ................................................................................................. 34 Test Statistics ...................................................................................................................................... 35 Reliability ...................................................................................................................................... 35 Standard Error of Measurement .................................................................................................... 36 Reliability of Classification .......................................................................................................... 37 Reliability of Scoring .................................................................................................................... 37 Scoring Methodology ............................................................................................................................. 38 Scoring ................................................................................................................................................ 38 Scoring Methodology for Constructed-Response Items ..................................................................... 38 Content Category Information ............................................................................................................ 40 Quality Assurance Measures..............................................................................................................