ACT Aspire® Technical Manual

2019 VERSION 5

Summative Technical Manual

discoveractaspire.org Preface

The ACT Aspire Technical Manual contains detailed technical information about the ACT Aspire® summative assessment. The principal purpose of the manual is to document technical characteristics of the ACT Aspire assessment in light of its intended purposes. The ACT Aspire Technical Manual documents the collection of validity evidence that supports appropriate interpretations of test scores and describes various content and psychometric aspects of the ACT Aspire. Multiple test design and development processes are articulated documenting how ACT attends to building the assessment in line with the validity argument and how concepts like construct validity, fairness, and accessibility are attended to throughout the process. Also described are routine analyses designed to support ongoing and continuous improvement and research intended to assure the program remains psychometrically sound.

ACT endorses and is committed to industry standards and criteria. ACT endorses and is committed to complying with The Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014). ACT also endorses the Code of Fair Testing Practices in Education (Joint Committee on Testing Practices, 2004), which is a statement of the obligations to test takers of those who develop, administer, or use educational tests and test data in the following four areas: developing and selecting appropriate tests, administering and scoring tests, reporting and interpreting test results, and informing test takers. ACT endorses and is committed to complying with the Code of Professional Responsibilities in Educational Measurement (NCME Ad Hoc Committee on the Development of a Code of Ethics, 1995), which is a statement of professional responsibilities for those involved with various aspects of assessments, including development, marketing, interpretation, and use.

We encourage individuals who want more detailed information on a topic discussed in this manual, or on a related topic, to contact ACT.

Please direct comments or inquiries to the address below: Research Services ACT, Inc. 500 ACT Drive Iowa City, Iowa 52243-0168.

© 2019 by ACT, Inc. All rights reserved. ACT Aspire®, the ACT® test, ACT Explore®, ACT Plan®, EPAS ®, and ACT NCRC® are registered trademarks, and ACT National Career Readiness Certificate™ isa trademark of ACT, Inc. OPS1262 Special Note Regarding ACT Aspire Design Changes and Technical Manual Corrections

ACT Aspire has undergone some test design changes since its release. Readers should be aware that this Technical Manual describes these specific updates in the Appendix of this Manual. Test description and analyses contained in the main body of the manual reflect the state of the forms at the time corresponding data was collected, and so may not be consistent with the updated form.

Please keep the following in mind when consulting this Technical Manual:

• The four chapters at the end of the appendix of this Technical Manual (entitled 2016 Updates, 2017 Updates, 2018 Updates and 2019 Updates respectively) contain updated information for ACT Aspire. • The updates include changes to norms and summary statistics, timing, inclusion of new benchmarks, updated reliabilities, and changes to the testing program for 2016, 2017, 2018 and 2019. • The information in these chapters are the latest changes to the ACT Aspire testing program and reflect the current ACT Aspire test design. Information in the main body of the manual accurately describes prior ACT Aspire test design. • A future full revision of the ACT Aspire Technical Manual will incorporate these updates.

i ii Contents

1 General Description of ACT Aspire Assessments and Standards � 1.1 1.1 Overview �� 1.1 1.2 Purposes, Claims, Interpretations, and Uses of ACT Aspire �� 1.1 1.3 ACT Aspire: Performance Level Descriptors and College and Career Readiness Standards �� 1.4 1.4 ACT Aspire and Standards Alignment �� 1.5

2 ACT Aspire Test Development �� 2.1 2.1 Assessment Design Elements and Construct Coherence �� 2.1 2.1.1 ACT Aspire Item types �� 2.2 2.1.2 Cognitive Complexity and Depth of Knowledge (DOK) �� 2.2 2.2 ACT Aspire Test Development Processes �� 2.3 2.2.1 Selecting and Training Item Writers �� 2.3 2.2.2 Designing Items that Elicit Student Evidence �� 2.3 2.2.3 Item Review �� 2.4 2.2.4 Field-testing �� 2.4

3 Assessment Specifications �� 3.1 3.1 Overview �� 3.1 3.2 ACT Aspire Assessment Support Materials �� 3.1 3.2.1 ACT Knowledge and Skills Map �� 3.1 3.2.2 ACT Aspire Exemplar Items �� 3.2 3.3 English Language Arts Assessments Overview �� 3.2 3.3.1 English Test �� 3.2 3.3.1.1. English Framework �� 3.2 3.3.1.2 English Reporting Categories �� 3.3 3.3.1.3 English Item Types �� 3.4 3.3.1.4 English Test Blueprints �� 3.4

iii Contents

3.3.2 Reading Test �� 3.6 3.3.2.1 Reading Framework �� 3.6 3.3.2.2 Reading Reporting Categories �� 3.7 3.3.2.3 Reading Test Complexity and Types of Texts �� 3.7 3.3.2.4 Reading Test Blueprints �� 3.9 3.3.3 Writing Test �� 3.11 3.3.3.1 Writing Task Types by Grade �� 3.11 3.3.3.2 Writing Scoring Rubrics and Reporting Categories �� 3.12 3.4 Mathematics Test �� 3.13 3.4.1 Mathematics Reporting Categories �� 3.14 3.4.2 Calculator Policy �� 3.23 3.4.3 Mathematics Item Types, Tasks, Stimulus �� 3.24 3.4.4 Mathematics Test Blueprints �� 3.25 3.5 Science Test �� 3.30 3.5.1 Science Reporting Categories �� 3.30 3.5.2 Continuum of Complexity �� 3.31 3.5.3 Science Item Types �� 3.32 3.5.4 Science Test Blueprints �� 3.37

4 Item and Task Scoring �� 4.1 4.1 Selected-Response Item Scoring �� 4.1 4.2 Technology-Enhanced Items Scoring �� 4.1 4.3 Constructed-Response Tasks �� 4.1 4.3.1 Performance Scoring Quality Control �� 4.2 4.3.2 Automated Scoring �� 4.4

5 Accessibility �� 5.1 5.1 Development of the ACT Aspire Accessibility Support System �� 5.1 5.2 Test Administration and Accessibility Levels of Support �� 5.3 5.2.1 Understanding Levels of Accessibility Support �� 5.3 5.2.2 Support Level 1: Default Embedded System Tools �� 5.5 5.2.3 Support Level 2: Open Access Tools �� 5.6 5.2.4 Support Level 3: Accommodations �� 5.7 5.2.5 Support Level 4: Modifications �� 5.9 5.3 Accommodations, Open Access and Embedded Tools �� 5.9 5.4 2016 Paper Summative Testing �� 5.10 5.5 2016 Online Summative Testing �� 5.13

6 Test Administration �� 6.1 6.1 Policies and Procedures �� 6.1 6.2 Standardized Procedures �� 6.1 iv Contents

6.3 Selecting and Training Testing Staff �� 6.2 6.3.1 Room Supervisors �� 6.3 6.3.2 Responsibilities of Other Testing Staff �� 6.4 6.3.3 Staff Training Sessions �� 6.5

7 Test Security �� 7.1 7.1 Data Security �� 7.1 7.1.1 Personally Identifiable Information �� 7.2 7.1.2 Security Features of Data Systems �� 7.2 7.1.3 Data Breaches and Remedies �� 7.5 7.1.4 Data Privacy and Use of Student Data �� 7.6 7.2 Test Security and Administration Irregularities �� 7.8

8 ACT Aspire Reporting �� 8.1

9 ACT Aspire Scores �� 9.1 9.1 ACT Aspire Subject Scores �� 9.1 9.2 ACT Aspire Composite Score �� 9.1 9.3 ACT Aspire ELA Score �� 9.3 9.4 ACT Aspire STEM Score �� 9.3 9.5 Reporting Category Scores �� 9.4 9.6 Progress with Text Complexity �� 9.5 9.7 Progress toward Career Readiness Indicator �� 9.5

10 ACT Aspire Score Scale �� 10.1 10.1 ACT Aspire Scaling Philosophy �� 10.1 10.2 Scaling Study �� 10.2 10.2.1 Scaling Study Design �� 10.2 10.2.2 Construction of Scaling Tests �� 10.2 10.2.3 Data Collection Design �� 10.3 10.2.4 Creation of Vertical Scales �� 10.5 10.3 Evaluation of the Vertical Scales �� 10.11 10.4 Results �� 10.12 10.4.1 Scaling Test Raw Scores �� 10.12 10.4.2 Evaluating On-the-Same-Scale Property �� 10.12 10.4.3 Evaluating Constant Conditional Standard Error of Measurement Property �� 10.21 10.4.4 Growth Patterns �� 10.26 10.5 Scaling ACT Aspire Writing �� 10.31

11 ACT Aspire Norms �� 11.1

v Contents

12 EPAS® to ACT Aspire Concordance �� 12.1 12.1 Method �� 12.2 12.2 Results �� 12.3 12.3 Summary �� 12.16

13 ACT Readiness Benchmarks �� 13.1 13.1 Introduction �� 13.1 13.2 ACT Readiness Benchmarks for English, Mathematics, Reading, and Science �� 13.2 13.2.1 Grades 8–10 �� 13.2 13.2.2 Grades 3–7 �� 13.3 13.3 ACT Readiness Benchmark for Writing �� 13.4 13.4 ACT Readiness Benchmarks for ELA and STEM �� 13.5 13.5 ACT Readiness Levels �� 13.5 13.6 Readiness Ranges for Reporting Categories �� 13.8

14 ACT Aspire Growth �� 14.1 14.1 Predicted Score Paths �� 14.1 14.1.1 Student Progress Reports �� 14.2 14.1.2 Aggregate Progress Reports �� 14.2 14.1.3 Samples Used to Develop the Predicted Paths �� 14.3 14.1.4 Statistical Models Used to Develop the Predicted Paths �� 14.4 14.1.5 Limitations of the Predicted Paths �� 14.5 14.2 Student Growth Percentiles �� 14.5 14.2.1 Consistency of SGPs across Subject Areas �� 14.5 14.3 ACT Aspire Gain Scores �� 14.6 14.3.1 Interpreting ACT Aspire Gain Scores �� 14.7 14.4 Growth-to-Standards Models with ACT Aspire �� 14.8 14.5 Measurement Error of Growth Scores �� 14.9 14.6 Aggregate Growth Scores for Research and Evaluation �� 14.9 14.6.1 Normative Data for School Mean SGPs �� 14.10 14.6.2 Mean SGP Differences across Student Subgroups �� 14.12 14.7 Additional Growth Modeling Resources �� 14.14

15 Progress toward Career Readiness �� 15.1 15.1 Introduction �� 15.1 15.2 Link ACT NCRC Levels to the ACT Composite Scores �� 15.3 15.3 Link the EPAS Composite Scores to ACT Aspire Composite Scores �� 15.4 15.4 Identify the ACT Aspire Composite Scores Corresponding to Each ACT NCRC Level �� 15.5 15.5 Report Progress toward Career Readiness using ACT Aspire Cut Scores �� 15.6

vi Contents

16 ACT Aspire Reliability �� 16.1 16.1 Introduction �� 16.1 16.2 Raw Score Reliability �� 16.2 16.3 Scale Score Reliability and Conditional Standard Error of Measurement �� 16.6

17 ACT Aspire Validity �� 17.1 17.1 Interpretations of ACT Aspire Scores �� 17.2 17.2 Validity Evidence �� 17.3 17.2.1 Content-Oriented Evidence �� 17.3 17.2.2 Evidence Regarding Cognitive Processes �� 17.5 17.2.3 Evidence Regarding Internal Structure �� 17.6 17.2.4 Evidence Regarding Relationships with Conceptually Related Constructs 17.33 17.2.5 Evidence Regarding Relationships with Criteria �� 17.64 17.2.6 Evidence Based on Consequences of Tests �� 17.64 17.3 Summary �� 17.65

18 Fairness �� 18.1 18.1 Overview �� 18.1 18.2 Fairness During the Testing Process �� 18.2 18.3 Fairness in Test Accessibility �� 18.2 18.4 Fairness in Score Interpretations for Intended Uses �� 18.2

19 ACT Aspire Mode Comparability Study �� 19.1 19.1 Introduction �� 19.1 19.2 Method and Results �� 19.2 19.3 Spring 2013 Comparability Study Demographic Summary Tables �� 19.4 19.4 Mode Effects for Raw Scores on Common Items �� 19.9 19.5 Item-level Performance across Modes �� 19.26 19.6 Comparisons of Scale Scores across Modes �� 19.44 19.7 Summary �� 19.62

20 ACT Aspire Equating �� 20.1 20.1 Methods and Procedures �� 20.1 20.2 Summary from the Spring 2014 Administration �� 20.2 20.2.1 Results �� 20.3 20.2.2 Standard Error of Equating �� 20.3

Appendices �� A.1 Appendix A: Illustration of the ACT Aspire Mathematics Assessments �� A.1 Appendix B: The ACT with Constructed-Response Items �� B.1 Appendix C: Spring 2013 Scaling Study Demographic Summary Tables �� C.1 Appendix D: Descriptive Statistics and Reliabilities of Reporting Category Raw Scores �� D.1

vii Contents

Appendix E: ACT Aspire ELA Content Framework and Alignment: English �� E.1 Appendix F: ACT Aspire ELA Content Framework and Alignment: Reading �� F.1

References �� R.1

2016 Updates �� U.1 Section 1: Updating ACT Aspire Norms �� U.1 ACT Aspire Norm Samples �� U.1 Weighting Methodology for ACT Aspire Norms �� U.2 Norm Tables and Summary Statistics �� U.7 Section 2: Reliability for Spring 2015 ACT Aspire Forms �� U.30 Section 3: Changes to the Program for 2016 �� U.32

2017 Updates �� V.1 Section 1: Updating ACT Aspire Norms �� V.1 ACT Aspire Norm Samples �� V.1 Norm Tables and Summary Statistics �� V.2 Section 2: Reliability for Spring 2016 ACT Aspire Forms �� V.23 Section 3: Changes to the Program for 2017 �� V.24

2018 Updates �� W.1 Section 1: Updating ACT Aspire Norms �� W.1 ACT Aspire Norm Samples �� W.1 Weighting methodology for ACT Aspire Norms �� W.2 Norm Tables and Summary Statistics �� W.10 Section 2: Updating ELA and STEM Benchmarks �� W.40 Section 3: Reliability for Spring 2017 ACT Aspire Forms �� W.42 Section 4: Program Updates �� W.43 Reducing Constructed Response (CR) Items and Replacing with Multiple Choice (MC) Items �� W.43 Science �� W.43 Reading �� W.44 Mathematics �� W.44 Specification Tables �� W.44 Science �� W.44 Reading �� W.44 Mathematics �� W.44 English �� W.45 Reporting for Writing �� W.48

viii Contents

2019 Updates �� X.1 Section 1: ACT Aspire Spring Norms �� X.1 ACT Aspire Spring Norm Samples �� X.2 Weighting methodology for ACT Aspire Spring Norms �� X.2 Spring Norm Tables and Summary Statistics �� X.10 Section 2: ACT Aspire Fall Norms �� X.40 ACT Aspire Fall Norm Samples �� X.40 Fall Norm Tables and Summary Statistics �� X.40 Section 3: Reliability for Spring 2018 ACT Aspire Standard Forms �� X.56 Section 4: Character Limits for Student Responses on the ACT Aspire Writing Test �� X.57

ix Tables

Table 3.1. Specification Ranges by Item Type and Reporting Category for Grade 3 �� 3.4 Table 3.2. Specification Ranges by Item Type and Reporting Category for Grades 4–5 �� 3.5 Table 3.3. Specification Ranges by Item Type and Reporting Category for Grades 6–8 �� 3.5 Table 3.4. Specification Ranges by Item Type and Reporting Category for Grade EHS �� 3.6 Table 3.5. Passage Coverage by Text Complexity for the ACT Aspire Reading Test Grade Level �� 3.8 Table 3.6. Specification Ranges by Item Type and Reporting Category for Grades 3–7 �� 3.10 Table 3.7. Specification Ranges by Item Type and Reporting Category for Grades 8–EHS �� 3.10 Table 3.8. Items by Passage Type for the ACT Aspire Reading Test �� 3.11 Table 3.9. Tasks by Writing Mode for Each Grade of the Writing Test �� 3.11 Table 3.10. Writing Test Scoring Rubric Domains and Associated Reporting Categories �� 3.12 Table 3.11. Points by Content Category for the Writing Test �� 3.13 Table 3.12. Number of Justification & Explanation Tasks, Raw-Score Points (and Percentage Weight) in Reporting Categories by Grade �� 3.22 Table 3.13. Modeling Minimum Number (and Percentage) of Raw-Score Points by Reporting Category and Grade �� 3.23 Table 3.14. Modeling Actual Average Number of Raw-Score Points for 2 Forms by Grade �� 3.23 Table 3.15. Modeling Actual Average Percentage of Raw-Score Points for 2 Forms by Grade �� 3.23 Table 3.16. Target Percentage of Raw-Score Points by Item Type and Grade �� 3.24 Table 3.17. Target for Raw-Score Points (Percentage of Test) [Percentage of Reporting Category] by DOK and Grade �� 3.25 Table 3.18. Specification Ranges by Item Type and Reporting Category for Grades 3-5 �� 3.26 Table 3.19. Specification Ranges by Item Type and Reporting Category for Grades 6-7 �� 3.27 Table 3.20. Specification Ranges by Item Type and Reporting Category for Grade 8 �� 3.28 Table 3.21. Specification Ranges by Item Type and Reporting Category for EHS �� 3.29 Table 3.22. Stimulus Modes used on the ACT Aspire Science Test �� 3.33 x Table 3.23. ACT Aspire Science College and Career Readiness Knowledge and Skill Domain �� 3.34 Tables

Table 3.24. Specification Ranges by Item Type and Reporting Category for Grades 3–5 �� 3.37 Table 3.25. Specification Ranges by Item Type and Reporting Category for Grades 6–7 �� 3.38 Table 3.26. Specification Ranges by Item Type and Reporting Category for Grades 8–EHS � 3.39 Table 4.1. Scorer Performance Reports �� 4.4 Table 5.1. Paper Summative Testing Presentation Supports �� 5.10 Table 5.2. Paper Summative Testing Interaction and Navigation Supports �� 5.12 Table 5.3. Paper Summative Testing Response Supports �� 5.12 Table 5.4. Paper Summative Testing General Test Condition Supports �� 5.13 Table 5.5. Online Summative Testing Presentation Supports �� 5.13 Table 5.6. Online Summative Testing Interaction and Navigation Supports �� 5.15 Table 5.7. Online Summative Testing Response Supports �� 5.16 Table 5.8. Online Summative Testing General Test Condition Supports �� 5.16 Table 5.9. Frequency of Use of Accommodated Forms, ACT Aspire, Spring 2015 �� 5.17 Table 9.1. Composite Score Descriptive Statistics Based on the 2013 Scaling Study �� 9.1 Table 9.2. Composite Score Effective Weights-Equal Nominal Weights �� 9.2 Table 9.3. ELA Score Effective Weights-Equal Nominal Weights �� 9.3 Table 9.4. STEM Score Effective Weights-Equal Nominal Weights �� 9.4 Table 9.5. Reporting Categories within each Subject Area �� 9.4 Table 10.1. Data Collection Design of the Scaling Study �� 10.5 Table 10.2. Raw Scores of On-Grade Tests for Two Groups Taking Lower/Upper Level Scaling Tests �� 10.7 Table 10.3. Raw Scores on Common Items for Two Groups Taking Lower/Upper level Scaling Tests �� 10.8 Table 10.4. Mean and Standard Deviation of Raw Scores on English Scaling Test (ST) by Grade Specific Items �� 10.13 Table 10.5. Mean and Standard Deviation of Raw Scores on Mathematics Scaling Test (ST) by Grade Specific Items �� 10.14 Table 10.6. Mean and Standard Deviation of Raw Scores on Reading Scaling Test (ST) by Grade Specific Items �� 10.15 Table 10.7. Mean and Standard Deviation of Raw Scores on Science Scaling Test (ST) by Grade Specific Items �� 10.16 Table 10.8. English Scale Score Descriptive Statistics Based on Scaling Study by Grade . . 10.28 Table 10.9. Mathematics Scale Score Descriptive Statistics Based on Scaling Study by Grade �� 10.28 Table 10.10. Reading Scale Score Descriptive Statistics Based on Scaling Study by Grade 10.29 Table 10.11. Science Scale Score Descriptive Statistics Based on Scaling Study by Grade 10.29 Table 10.12. Writing Scale Score Descriptive Statistics Based on Scaling Study by Grade �� 10.29 Table 10.13. Lowest Obtainable Scale Scores (LOSS) and Highest Obtainable Scale Scores (HOSS) �� 10.30 Table 11.1. Percentage of Students Meeting Subject Benchmark in the 2014 Norm Sample 11.2 Table 11.2. 2014 ACT Aspire English Norm Group Demographics �� 11.2 Table 11.3. 2014 ACT Aspire Mathematics Norm Group Demographics �� 11.4 xi Tables

Table 11.4. 2014 ACT Aspire Reading Norm Group Demographics �� 11.6 Table 11.5. 2014 ACT Aspire Science Norm Group Demographics �� 11.8 Table 11.6. 2014 ACT Aspire Writing Norm Group Demographics �� 11.10 Table 11.7. 2014 ACT Aspire English Norms: Percentage of Students at or below Each Scale Score �� 11.12 Table 11.8. 2014 ACT Aspire Mathematics Norms: Percentage of Students at or below Each Scale Score �� 11.14 Table 11.9. 2014 ACT Aspire Reading Norms: Percentage of Students at or below Each Scale Score �� 11.16 Table 11.10. 2014 ACT Aspire Science Norms: Percentage of Students at or below Each Scale Score �� 11.18 Table 11.11. 2014 ACT Aspire Writing Norms: Percentage of Students at or below Each Scale Score �� 11.20 Table 12.1. Descriptive Statistics of the Sample for Concordance Analysis �� 12.3 Table 12.2. Demographic Information of the Sample for Concordance Analysis �� 12.4 Table 12.3. Descriptive Statistics of ACT Explore and Concorded ACT Aspire Scores in Cross-Sectional Evaluation Sample �� 12.7 Table 12.4. Descriptive Statistics of ACT Plan and Concorded ACT Aspire Scores in Cross-Sectional Evaluation Sample �� 12.7 Table 12.5. Descriptive Statistics of ACT Explore Grade 8 and ACT Plan Grade 10 Scale Scores in Longitudinal Data �� 12.16 Table 12.6. Descriptive Statistics of Concorded ACT Aspire Scale Scores in Longitudinal Data �� 12.16 Table 12.7. EPAS to ACT Aspire Concordance �� 12.17 Table 13.1. ACT College Readiness Benchmarks and ACT Readiness Benchmarks for Grades 8–10 �� 13.3 Table 13.2. Classification Agreement Rate between ACT Aspire and EPAS Benchmarks �� 13.3 Table 13.3. ACT Readiness Benchmarks for Grades 3–7 �� 13.4 Table 13.4. ACT Readiness Benchmarks in ELA and STEM for Grades 3–10 �� 13.5 Table 13.5. Benchmarks, Low Cuts, and High Cuts for ACT Readiness Benchmarks by Subject and Grade �� 13.7 Table 14.1. Longitudinal Samples Used to Develop Predicted Paths, Predicted ACT Score Ranges, and Student Growth Percentiles for Spring 2016 Reporting �� 14.4 Table 14.2. Correlations of SGPs across Subject Areas �� 14.6 Table 14.3. ACT Aspire Gain Score Means and Standard Deviations �� 14.7 Table 14.4. Distributions of School MGPs, by Subject Area �� 14.11 Table 14.5. SGP Sample Size by Subject Area and Student Subgroup �� 14.13 Table 14.6. Mean SGP by Subject Area and Student Subgroup �� 14.13 Table 15.1. Descriptive Statistics of ACT Composite Scores by ACT NCRC Level �� 15.3 Table 15.2. ACT Composite Scores Corresponding to a 50% Chance of Obtaining Different ACT NCRC Levels �� 15.4 Table 15.3. Descriptive Statistics of the Analysis Sample (N=13,528) �� 15.4 Table 15.4. Grade 11 EPAS and ACT Aspire Composite Scores Corresponding to xii ACT NCRC Levels �� 15.5 Tables

Table 15.5. ACT Aspire Composite Scores Corresponding to a 50% Chance of Obtaining Different ACT NCRC Levels �� 15.5 Table 16.1. Raw and Scale Score Reliability Coefficient Ranges by Grade for Subject Tests and Composite Score from Spring 2013 Special Studies Data �� 16.2 Table 16.2. Raw Score Reliability Coefficient Ranges by Grade for Subject Tests from Spring 2014 Operational Data �� 16.3 Table 16.3. Percentages of Students for Writing Rater Analysis in the Spring 2013 Special Studies by Grade and Demographics �� 16.4 Table 16.4. Writing Test Correlations between Rater 1 and Rater 2 by Domain and by Form from Spring 2013 Special Studies Data �� 16.5 Table 16.5. Writing Test Reliability Coefficients Based on Four Domain Scores from Spring 2013 Special Studies Data �� 16.5 Table 16.6. Scale Score Reliability Coefficient and Standard Error of Measurement by Grade and Subject from Spring 2014 Operational Data �� 16.8 Table 17.1. English ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentage of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study �� 17.18 Table 17.2. Mathematics ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentages of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study �� 17.20 Table 17.3. Reading ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentages of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study �� 17.22 Table 17.4. Science ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentages of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study �� 17.23 Table 17.5. English Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study �� 17.25 Table 17.6. Mathematics Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study �� 17.26 Table 17.7. Reading Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study �� 17.27 Table 17.8. Science Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study �� 17.28 Table 17.9. Two-Factor Exploratory Factor Model Loadings for Four ACT Aspire Forms Identified as Showing Evidence of Two Factors Based on Analysis of Fit Statistics . . 17.29 Table 17.10. Summary Statistics for ACT Aspire Scale Scores by Grade �� 17.30 Table 17.11. Correlations among ACT Aspire Scale Scores by Grade �� 17.31 Table 17.12. Percentages of Students in the ACT Explore, ACT Plan, and ACT Aspire Matched Sample by Subject, Grade and Demographics �� 17.34 Table 17.13. Descriptive Statistics for ACT Explore, ACT Plan and ACT Aspire Scale Scores �� 17.35 Table 17.14. Correlations between ACT Explore/ACT Plan Scale Scores and ACT Aspire Scale Scores �� 17.36 xiii Tables

Table 17.15. Disattenuated Correlations between ACT Explore/ACT Plan Scale Scores and ACT Aspire Scale Scores �� 17.36 Table 17.16. Scale Score Reliabilities �� 17.37 Table 17.17. Multitrait-Multimethod Matrices for ACT Explore/Plan and ACT Aspire Scale Scores by Grade Level �� 17.46 Table 17.18. Sample Sizes by Grade and Subject for the Sample of Students with Scores on ARMT+ and ACT Aspire in Spring 2013 �� 17.50 Table 17.19. Descriptive Statistics for ARMT+ and ACT Aspire Scale Scores �� 17.50 Table 17.20. Correlations between ARMT+ Scale Scores and ACT Aspire Scale Scores �� 17.51 Table 17.21. Disattenuated Correlations between ARMT+ Scale Scores and ACT Aspire Scale Scores �� 17.51 Table 17.22. Multitrait-Multimethod Matrices for ARMT+ and ACT Aspire Scale Scores by Grade Level �� 17.60 Table 17.23. Disattenuated Correlations between ARMT+ and ACT Aspire Scale Scores by Grade Level �� 17.63 Table 19.1. Percentage of Items Different Across Online and Paper Forms �� 19.3 Table 19.2. Sample Sizes by Grade and Subject �� 19.3 Table 19.3. Percentages of Students Taking English Tests in the Comparability Study by Grade, Mode and Demographics �� 19.4 Table 19.4. Percentages of Students Taking Mathematics Tests in the Comparability Study by Grade, Mode and Demographics �� 19.5 Table 19.5. Percentages of Students Taking Reading Tests in the Comparability Study by Grade, Mode and Demographics �� 19.6 Table 19.6. Percentages of Students Taking Science Tests in the Comparability Study by Grade, Mode and Demographics �� 19.7 Table 19.7. Percentages of Students Taking Writing Tests in the Comparability Study by Grade, Mode and Demographics �� 19.8 Table 19.8. English Common-Item Raw Score Summary for Online and Paper Modes �� 19.10 Table 19.9. Mathematics Common-Item Raw Score Summary for Online and Paper Modes 19.11 Table 19.10. Reading Common-Item Raw Score Summary for Online and Paper Modes �� 19.12 Table 19.11. Science Common-Item Raw Score Summary for Online and Paper Modes �� 19.13 Table 19.12. Writing Common-Item Raw Score Summary for Online and Paper Modes �� 19.14 Table 19.13. ACT Aspire Spring 2013 Comparability Study P-values for English Early High School Items Flagged in Delta Plots �� 19.43 Table 19.14. English Scale Score Summary for Online and Paper Modes �� 19.45 Table 19.15. Mathematics Scale Score Summary for Online and Paper Modes �� 19.46 Table 19.16. Reading Scale Score Summary for Online and Paper Modes �� 19.47 Table 19.17. Science Scale Score Summary for Online and Paper Modes �� 19.48 Table 19.18. Writing Scale Score Summary for Online and Paper Modes �� 19.49 Table 20.1. Percentages of Students Taking English Online Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.5 Table 20.2. Percentages of Students Taking English Paper Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.6 xiv Tables

Table 20.3. Percentages of Students Taking Mathematics Online Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.7 Table 20.4. Percentages of Students Taking Mathematics Paper Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.8 Table 20.5. Percentages of Students Taking Reading Online Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.9 Table 20.6. Percentages of Students Taking Reading Paper Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.10 Table 20.7. Percentages of Students Taking Science Online Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.11 Table 20.8. Percentages of Students Taking Science Paper Forms in the Spring 2014 Equating Study by Grade and Demographics �� 20.12 Table B.1. Number of Constructed-Response Items and Number of Score Points Included in the On-Grade ACT with Constructed-Response Test Forms in the Scaling Study . . B.1 Table C.1. Percentages of Students Taking English Tests in the Scaling Study by Grade and Demographics �� C.1 Table C.2. Percentages of Students Taking Mathematics Tests in the Scaling Study by Grade and Demographics �� C.2 Table C.3. Percentages of Students Taking Reading Tests in the Scaling Study by Grade and Demographics �� C.3 Table C.4. Percentages of Students Taking Science Tests in the Scaling Study by Grade and Demographics �� C.4 Table C.5. Percentages of Students Taking Writing Tests in the Scaling Study by Grade and Demographics �� C.5 Table D.1. English Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration �� D.1 Table D.2. Mathematics Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration �� D.3 Table D.3. Reading Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration �� D.7 Table D.4. Science Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration �� D.9 Table U.1. Sample Sizes Used for Norm Establishment �� U.2 Table U.2. Correlation of District Mean ACT Composite Scores with District Mean ACT Aspire Score �� U.3 Table U.3. Target Public School Population and ACT Public School Population �� U.5 Table U.4. One Example of ACT Aspire Norm Sample Weighting: Grade 5 English �� U.6 Table U.5. 2015 ACT Aspire Weighted English Norm Group Demographics �� U.9 Table U.6. 2015 ACT Aspire Weighted Mathematics Norm Group Demographics �� U.11 Table U.7. 2015 ACT Aspire Weighted Reading Norm Group Demographics �� U.13 Table U.8. 2015 ACT Aspire Weighted Science Norm Group Demographics �� U.15 Table U.9. 2015 ACT Aspire Weighted Writing Norm Group Demographics �� U.17 Table U.10. 2015 ACT Aspire English Norms: Percentage of Students at or below Each Scale Score �� U.19 xv Tables

Table U.11. 2015 ACT Aspire Mathematics Norms: Percentage of Students at or below Each Scale Score �� U.21 Table U.12. 2015 ACT Aspire Reading Norms: Percentage of Students at or below Each Scale Score �� U.23 Table U.13. 2015 ACT Aspire Science Norms: Percentage of Students at or below Each Scale Score �� U.25 Table U.14. 2015 ACT Aspire Writing Norms: Percentage of Students at or below Each Scale Score �� U.27 Table U.15. National Average Scale Scores Using 2015 Norm Sample �� U.28 Table U.16. Percentage of Students Who Met Subject Benchmark in the 2015 Norm Sample �� U.29 Table U.17. Cronbach’s Alpha of ACT Aspire Forms Administered in Spring 2015 �� U.30 Table U.18. ACT Aspire Summative Testing Time Changes (in minutes) �� U.32 Table U.19. English: Number of Items by Item Type �� U.34 Table U.20. English: Number of Points by Reporting Category �� U.34 Table U.21. Mathematics: Number of Items by Item Type �� U.34 Table U.22. Mathematics: Number of Points by Reporting Category �� U.35 Table V.1. Sample Sizes Used for Norm Establishment �� V.2 Table V.2. 2016 ACT Aspire Weighted English Norm Group Demographics �� V.3 Table V.3. 2016 ACT Aspire Weighted Mathematics Norm Group Demographics �� V.5 Table V.4. 2016 ACT Aspire Weighted Reading Norm Group Demographics �� V.7 Table V.5. 2016 ACT Aspire Weighted Science Norm Group Demographics �� V.9 Table V.6. 2016 ACT Aspire Weighted Writing Norm Group Demographics �� V.11 Table V.7. 2016 ACT Aspire English Norms: Percent of Students at or below Each Scale Score �� V.13 Table V.8. 2016 ACT Aspire Mathematics Norms: Percent of Students at or below Each Scale Score �� V.15 Table V.9. 2016 ACT Aspire Reading Norms: Percent of Students at or below Each Scale Score �� V.17 Table V.10. 2016 ACT Aspire Science Norms: Percent of Students at or below Each Scale Score �� V.19 Table V.11. 2016 ACT Aspire Writing Norms: Percent of Students at or below Each Scale Score �� V.21 Table V.12. National Average Scale Scores Using 2016 Norm Sample �� V.22 Table V.13. Percentage of Students Who Met Subject Benchmark in the 2016 Norm Sample �� V.22 Table V.14. Ranges of Cronbach’s Alpha of ACT Aspire Forms Administered in Spring 2016 V.23 Table V.15. ACT Aspire Summative Testing Time Changes (in minutes) �� V.24 Table W.1. Sample Sizes Used for Norm Establishment �� W.2 Table W.2. Correlation of District Mean ACT Composite Scores with District Mean ACT Aspire Score �� W.3 Table W.3. Target Public School Population and ACT Public School Population �� W.5 Table W.4. An Example of ACT Aspire Norm Sample Weighting: Grade 5 English �� W.6 xvi Tables

Table W.5. An Example of ACT Aspire Norm Sample Weighting: Grade 9 Mathematics �� W.8 Table W.6. 2017 ACT Aspire Weighted English Norm Group Demographics �� W.11 Table W.7. 2017 ACT Aspire Weighted Mathematics Norm Group Demographics �� W.13 Table W.8. 2017 ACT Aspire Weighted Reading Norm Group Demographics �� W.15 Table W.9. 2017 ACT Aspire Weighted Science Norm Group Demographics �� W.17 Table W.10. 2017 ACT Aspire Weighted Composite Norm Group Demographics �� W.19 Table W.11. 2017 ACT Aspire Weighted ELA Norm Group Demographics �� W.21 Table W.12. 2017 ACT Aspire Weighted STEM Norm Group Demographics �� W.23 Table W.13. 2017 ACT Aspire English Norms: Percent of Students at or Below Each Scale Score �� W.25 Table W.14. 2017 ACT Aspire Mathematics Norms: Percent of Students at or Below Each Scale Score �� W.27 Table W.15. 2017 ACT Aspire Reading Norms: Percent of Students at or Below Each Scale Score �� W.29 Table W.16. 2017 ACT Aspire Science Norms: Percent of Students at or Below Each Scale Score �� W.31 Table W.17. 2017 ACT Aspire Composite Norms: Percent of Students at or Below Each Scale Score �� W.33 Table W.18. 2017 ACT Aspire ELA Norms: Percent of Students at or Below Each Scale Score �� W.35 Table W.19. 2017 ACT Aspire STEM Norms: Percent of Students at or Below Each Scale Score �� W.37 Table W.20. National Average Scale Scores Using 2017 Norm Sample �� W.39 Table W.21. Percentage of Students Who Met Subject Benchmark in the 2017 Norm Sample �� W.39 Table W.22. Updated ELA and STEM Readiness Benchmarks. �� W.41 Table W.23. Ranges of Cronbach’s Alpha of ACT Aspire Forms Administered in Spring 2017. �� W.42 Table W.24. Science: Number of Items by Item Type �� W.45 Table W.25. Science: Number of Points by Reporting Category �� W.45 Table W.26. Reading: Number of Items by Item Type �� W.45 Table W.27. Reading: Number of Points by Reporting Category �� W.46 Table W.28. Mathematics: Number of Items by Item Type �� W.46 Table W.29. Mathematics: Number of Points by Reporting Category �� W.47 Table W.30. English: Changes to stimulus type by grade �� W.47 Table X.1. Sample Sizes Used for Norm Establishment �� X.2 Table X.2. Correlation of District Mean ACT Composite Scores with District Mean ACT Aspire Score �� X.3 Table X.3. Target Public School Population and ACT Public School Population �� X.5 Table X.4. An Example of ACT Aspire Spring Norm Sample Weighting: Grade 5 English �� X.6 Table X.5. An Example of ACT Aspire Spring Norm Sample Weighting: Grade 9 Mathematics � X.8 Table X.6. 2018 ACT Aspire Weighted English Spring Norm Group Demographics �� X.11 Table X.7. 2018 ACT Aspire Weighted Mathematics Spring Norm Group Demographics �� X.13 xvii Tables

Table X.8. 2018 ACT Aspire Weighted Reading Spring Norm Group Demographics �� X.15 Table X.9. 2018 ACT Aspire Weighted Science Spring Norm Group Demographics �� X.17 Table X.10. 2018 ACT Aspire Weighted Composite Spring Norm Group Demographics �� X.19 Table X.11. 2018 ACT Aspire Weighted ELA Spring Norm Group Demographics �� X.21 Table X.12. 2018 ACT Aspire Weighted STEM Spring Norm Group Demographics �� X.23 Table X.13. 2018 ACT Aspire English Spring Norms: Percent of Students at or Below Each Scale Score �� X.25 Table X.14. 2018 ACT Aspire Mathematics Spring Norms: Percent of Students at or Below Each Scale Score �� X.27 Table X.15. 2018 ACT Aspire Reading Spring Norms: Percent of Students at or Below Each Scale Score �� X.29 Table X.16. 2018 ACT Aspire Science Spring Norms: Percent of Students at or Below Each Scale Score �� X.31 Table X.17. 2018 ACT Aspire Composite Spring Norms: Percent of Students at or Below Each Scale Score �� X.33 Table X.18. 2018 ACT Aspire ELA Spring Norms: Percent of Students at or Below Each Scale Score �� X.35 Table X.19. 2018 ACT Aspire STEM Spring Norms: Percent of Students at or Below Each Scale Score �� X.37 Table X.20. National Average Scale Scores Using 2018 Spring Norm Sample �� X.39 Table X.21. Percentage of Students Who Met Subject Benchmark in the 2018 Spring Norm Sample �� X.39 Table X.22. Sample Sizes used for Fall Norm Establishment �� X.40 Table X.23. 2018 ACT Aspire English Fall Norms: Percent of Students at or below Each Scale Score �� X.41 Table X.24. 2018 ACT Aspire Mathematics Fall Norms: Percent of Students at or below Each Scale Score �� X.43 Table X.25. 2018 ACT Aspire Reading Fall Norms: Percent of Students at or below Each Scale Score �� X.45 Table X.26. 2018 ACT Aspire Science Fall Norms: Percent of Students at or below Each Scale Score �� X.47 Table X.27. 2018 ACT Aspire Composite Fall Norms: Percent of Students at or below Each Scale Score �� X.49 Table X.28. 2018 ACT Aspire ELA Fall Norms: Percent of Students at or below Each Scale Score �� X.51 Table X.29. 2018 ACT Aspire STEM Fall Norms: Percent of Students at or below Each Scale Score �� X.53 Table X.30. National Average Scale Scores Using 2018 Fall Norm Sample �� X.55 Table X.31. Percentage of Students Who Met Subject Benchmark in the 2018 Fall Norm Sample �� X.55 Table X.32. Ranges of Coefficient Alpha of ACT Aspire Standard Forms Administered in Spring 2018 �� X.56 Table X.33. Character Limits for Student Responses on the ACT Aspire Writing Test �� X.57 xviii Figures

Figure 1.1. The full picture: evidence and validity. �� 1.2 Figure 2.1. ACT Aspire field testing model �� 2.5 Figure 3.1. ACT Aspire English test reporting with example skill targets for one reporting category. �� 3.3 Figure 3.2. ACT Aspire reading test reporting categories with example skill targets for one reporting category �� 3.7 Figure 3.3. ACT Aspire reading test item types. �� 3.9 Figure 3.4. ACT Aspire Writing test modes and reporting categories �� 3.13 Figure 3.5. Mathematics Domains, and Primary Currents in the Flow Across Grades �� 3.20 Figure 3.6. Foundation and Grade Level Progress (illustrated for Grade 7) �� 3.21 Figure 3.7. ACT and ACT Aspire science content complexity continuum. �� 3.32 Figure 4.1. Components of student writing performance. �� 4.6 Figure 5.1. Accessibility feature mapping process. �� 5.3 Figure 5.2. ACT Aspire Levels of Accessibility �� 5.4 Figure 5.3. Default embedded system tools. �� 5.6 Figure 5.4. Open access tools. �� 5.7 Figure 5.5. Accommodations �� 5.8 Figure 5.6. Modifications �� 5.9 Figure 10.1. Construction of the ACT Aspire scaling tests �� 10.3 Figure 10.2. Effect sizes on individual scaling test raw score scale and whole scaling test raw score scale derived from the 2PL model �� 10.9 Figure 10.3. Scatter plots of average scale scores against scaling test raw scores by grade �� 10.17 Figure 10.4. Conditional standard error of measurement on raw and scale score scales �� 10.22 Figure 10.5. Effect sizes on individual scaling test raw score scale, whole scaling test raw score scale, and ACT Aspire score scale �� 10.26 Figure 12.1. Scatter plots of ACT Aspire scale scores versus EPAS scale scores �� 12.5 Figure 12.2. Distributions of ACT Explore scale scores and concorded ACT Aspire scale scores �� 12.8 xix Figures

Figure 12.3. Distributions of ACT Plan scale scores and concorded ACT Aspire scale scores �� 12.10 Figure 12.4. Box plots of ACT Plan Scores (or concorded ACT Aspire scale scores) conditional on ACT Explore Scores (or concorded ACT Aspire Scores) �� 12.12 Figure 14.1. Prototype of ACT Aspire Student Progress Report �� 14.2 Figure 14.2. Prototype ACT Aspire Aggregate Progress Report �� 14.3 Figure 14.3. ACT Aspire gain score statistics for grade 7–8 reading �� 14.8 Figure 14.4. Distributions of school MGPs, by subject area �� 14.11 Figure 15.1. Sample Progress toward Career Readiness indicator from ACT Aspire report �� 15.2 Figure 17.1. Scree plots of ACT Aspire online and paper forms by subject and grade using data from the spring 2013 comparability study �� 17.10 Figure 17.2. Box plots of ACT Explore or ACT Plan English scale scores conditional on ACT Aspire English scale scores �� 17.38 Figure 17.3. Box plots of ACT Explore or ACT Plan mathematics scale scores conditional on ACT Aspire mathematics scale scores �� 17.39 Figure 17.4. Box plots of ACT Explore or ACT Plan reading scale scores conditional on ACT Aspire reading scale scores �� 17.41 Figure 17.5. Box plots of ACT Explore or ACT Plan science scale scores conditional on ACT Aspire science scale scores �� 17.42 Figure 17.6. Box plots of ACT Explore or ACT Plan Composite scale scores conditional on ACT Aspire Composite scale scores �� 17.44 Figure 17.7. Box plots of ARMT+ mathematics scale scores conditional on ACT Aspire mathematics scale scores �� 17.52 Figure 17.8. Box plots of ARMT+ reading scale scores conditional on ACT Aspire reading scale scores �� 17.55 Figure 17.9. Box plots of ARMT+ science scale scores conditional on ACT Aspire science scale scores �� 17.58 Figure 19.1. Cumulative percentage of students for common item raw scores across modes �� 19.15 Figure 19.2. English common item mean scores for paper and online forms by item type* �� 19.28 Figure 19.3. Mathematics common item mean scores for paper and online forms by item type* �� 19.30 Figure 19.4. Reading common item mean scores for paper and online forms by item type* 19.32 Figure 19.5. Science common item mean scores for paper and online forms by item type* 19.34 Figure 19.6. Delta plots for online and paper common items with predicted delta and one- and two-standard error bands. �� 19.36 Figure 19.7. Cumulative percentage of students for scale scores across modes �� 19.50 Figure 19.8. Box plots of scale scores by grade and mode for each subject area �� 19.60 Figure 20.1. Average English scale scores against common item raw scores �� 20.13 Figure 20.2. Average mathematics scale scores against common item raw scores �� 20.14 Figure 20.3. Average reading scale score against common item raw scores �� 20.15 Figure 20.4. Average science scale scores against common item raw scores �� 20.16 xx Figures

Figure 20.5. Unrounded scale score standard error of equating for selected subjects and grades �� 20.17 Figure A.1. Mathematics test content specifications and flow �� A.1 Figure U.1. Conceptual diagram of chain-linked weighting strategy �� U.8 Figure W.1. Conceptual diagram of chain-linked weighting strategy �� W.10 Figure X.1. Conceptual diagram of chain-linked weighting strategy �� X.10

xxi xxii CHAPTER 1 General Description of ACT Aspire Assessments and Standards

1.1 Overview The ACT Aspire® summative assessment program includes a vertically scaled battery of achievement tests designed to measure student growth in a longitudinal assessment system for grades 3–Early High School (EHS) in English, reading, writing, mathematics, and science. Taken as individual subject tests or as a battery, ACT Aspire can be delivered via computer or as a paper-and-pencil administration. The scale scores are linked to college and career data through scores on the ACT® test and the ACT National Career Readiness Certificate™ (ACT NCRC®) program. To enhance score interpretation, reporting categories for ACT Aspire use the same terminology as The ACT College and Career Readiness Standards (ACT CCRS) and other standards that target college and career readiness, including the standards of many states and the Common Core State Standards (CCSS).

1.2 Purposes, Claims, Interpretations, and Uses of ACT Aspire The purpose of the ACT Aspire assessments is to measure student achievement and progress toward college and career readiness. To identify college and career readiness constructs, ACT uses empirical and performance data to define requisite constructs in the content areas of English, reading, writing, mathematics, and science. Every three years, ACT administers the ACT National Curriculum Survey study to identify what kindergarten through postsecondary teachers, including instructors of

1.1 General Description of ACT Aspire Assessments and Standards

entry-level college and workforce-training courses, expect of their entering students— this includes the most current knowledge and skills students need to demonstrate to be ready for entry-level postsecondary courses and jobs. ACT also collects data about what is actually being taught in elementary, middle, and high school classrooms. Taken altogether, these results support ACT Aspire’s curriculum base and ACT’s ability to identify the key skill targets and knowledge that is most important for students to know to be ready for the next step. ACT uses these and other research results to design assessments and inform test blueprints so that each test targets the most important college and career readiness skills across skill progressions.

Additional validation of the constructs occurs through ACT’s longitudinal research (see figure 1.1). The Full Picture: Evidence and Validity

Identify the Skills that lead to Career College and Career Readiness ACT Success ACT National WorkKeys Curriculum Design tasks that elicit valid Performance Job Survey evidence on these skills Levels Proﬁles Describe what Academic students Design test forms that produce ACT Aspire Research can do reliable measurements Readiness at each Benchmarks College and benchmark Build Forms Career Administer Forms Readiness ACT College College Score Results Standards Readiness Course Benchmarks Grades Instructionally Actionable CONTENT VALIDITY Reporting Categories PREDICTIVE VALIDITY

Figure 1.1. The full picture: evidence and validity. Figure 1.1 shows how ACT uses research including empirical feedback loops to help inform continuous improvement. The ACT Aspire development starts by exhaustively understanding the most important requirements and most desired evidence needed to support inferences about college and career readiness, and tests are designed specifically to elicit the identified evidence. Subject matter experts (SMEs), item writers, and other educators work collaboratively to write items and review them for accuracy, appropriateness, and fairness. Operational forms are put together using pretested items and are built to test specifications. Scores from assessments are calculated, reported, and sent out on various reports designed for each audience. Instructionally relevant reporting categories provide immediate actionable information about how students perform on key skills at a more granular level. Additional psychometric work continues, including analyzing student performance along the Kindergarten to Early High School (K–EHS) continuum. Students who meet readiness score benchmarks at one grade level are compared to students who succeed in the next grade. This information is used to ensure the benchmarks are accurate and

1.2 General Description of ACT Aspire Assessments and Standards

up-to-date. Through research and feedback loops, ACT ensures that key targets are identified and measured appropriately across the continuum.

ACT Aspire was created by employing a theory of action (TOA) approach that fuses content validity (academic research) with predictive validity (empirical data), thus following similar methodologies used to build the ACT. The TOA begins by answering fundamental questions about the purpose of the assessment, such as: Who are the intended users? What are the intended uses of the assessment results? What are the consequences that may result from using the assessment? What are the intended interpretations or claims based on the assessment? What are measurable outcomes from using the assessment? The answers to these questions emerge as a result of rigorous research and data collection that inform and allow for the identification of high-value skill targets in each subject area, resulting in focal points for the development of tasks and test forms. The procedure set forth by the TOA further gives rise to hypothesized mechanisms or processes for bringing about the intended goals of the assessment results. For example, cognitive labs, piloting, and field testing are used to validate results, and iteratively improve the specifications and design of the assessment. Operational results are used to continuously improve the components of the assessment.

Artifacts of the assessment architecture emerge from the research and data collection process to ensure that items and test forms elicit the intended evidence to support the claims made by the assessment. For example, content and item specifications, test blueprints, benchmarks, and performance level descriptors (PLD’s) influence the technical quality and output of test items and forms. These artifacts are informed by several factors, including: • Academic research on skill targets, sequencing of skills, and grade level placement • Data and evidence of student understanding collected from the assessments • The ACT National Curriculum Survey • Survey of standards frameworks—including, but not limited to the ACT CCRS, CCSS, and Next Generation Science Standards • Subject Matter Experts (SME)

The principal claims, interpretations, and uses of ACT Aspire are the following: 1. To measure student readiness on an empirically derived college readiness trajectory. (Note that students taking the ACT Aspire battery will receive scores that can be compared to ACT Readiness Benchmarks that are linked to the ACT.) 2. To measure student readiness on a career readiness trajectory.

The secondary claims, interpretations, and uses of ACT Aspire are the following: 1. To provide instructionally actionable information to educators. Data from the ACT Aspire summative assessment can be used to identify areas of student strength and weakness in content areas at student, classroom, and school levels. Data can inform instruction and facilitate the identification of interventions. 1.3 General Description of ACT Aspire Assessments and Standards

2. To provide empirical data for inferences related to accountability. ACT Aspire data can be one of multiple measures for making inferences about student progress and growth with respect to college and career readiness in reading, language, writing, mathematics, and science, as reported in terms of ACT CCRS. ACT Aspire can also be used for accountability reporting where college and career readiness standards (such as CCSS and others) have been adopted. 3. To provide empirical support for inferences about international comparisons. Building on ACT research that mark ACT College Readiness Benchmarks as internationally competitive, ACT Aspire scores also will be linked to scores on the Programme for International Student Assessment (PISA) for comparisons on an international scale. Results will be available in the near future.

1.3 ACT Aspire: Performance Level Descriptors and College and Career Readiness Standards ACT Aspire assessments are aligned with leading frameworks of content standards that target college and career readiness. ACT Aspire Performance Level Descriptors (PLDs) have been created to provide specific descriptions of student performance within and across grades. The PLDs were developed in a 2016 study run by an independent facilitator; over 90 subject-matter experts (SMEs) from 14 states drafted the PLD statements. The PLDs at each grade are organized in 4 performance levels: In Need of Support, Close, Proficient, and Exceeding. The cut score for the Proficient level is based on the ACT Readiness Benchmark at that grade. The ACT Readiness Benchmarks were derived empirically working backwards from the ACT College and Career Readiness Benchmark scores, which indicate the level of achievement required to be on target for success at the next level (see Chapter 13 for details). In the 2016 PLD study, SMEs reviewed ACT Aspire materials, performance data and administered test items to generate statements that describe what students know and are able to do within each category in each content and grade. PLDs are found at https://www.discoveractaspire.org/performance-level-descriptors/.

In addition, ACT Aspire assessments in grade 8 and Early High School are aligned with the ACT College and Career Readiness Standards (CCRS). The ACT CCRS, developed for each content test, are descriptions of the skills and knowledge that ACT has empirically linked to readiness in postsecondary education and the world of work. Different groups of SMEs developed the ACT CCRS statements by synthesizing the domain-specific knowledge and skills demonstrated by students in particular score bands across thousands of students’ scores. Within each content area, the CCRS are organized by strand, which mirror the reporting categories featured in ACT Aspire, and by score band. The ACT CCRS are organized into six achievement levels; each level is defined by a 3-point score band. The CCRS also indicate the ACT College Readiness Benchmarks in each content area (except on the optional Writing test) that were derived using test performance and college placement data. The ACT College Readiness Benchmark scores are clearly delineated on the CCRS tables (see 1.4 General Description of ACT Aspire Assessments and Standards

the ACT Technical Manual for more information on derivation of ACT CCRS and ACT College Readiness Benchmarks).

Science learning standards currently vary dramatically across states and organizations. ACT Aspire science assessments for grades 3–7 are associated with the draft ACT Aspire Science College and Career Readiness Knowledge and Skill Domain (See Table 3.23). SMEs developed these statements by back-mapping the ACT CCRS down to grade 3. Content and measurement specialists then consulted several sources including ACT’s evidence, research in science education, the National Science Education Standards, NAEP findings, and input from nationally known SMEs in elementary level science education. These statements were then used to develop the ACT Aspire PLDs. The ACT Aspire PLDs provide a greater level of specificity in describing student achievement across grades and at different scale-score levels.

1.4 ACT Aspire and Standards Alignment Many sets of standards target college and career readiness exist including the ACT CCRS, the CCSS, and unique standards from states across the United States. The purpose of these standards is to articulate requisite knowledge and skills needed to prepare students for postsecondary and career success. As previously described, ACT Aspire assessments are designed to measure progress and provide evidence to back up the claim, in this case, that students are on target for college and career readiness.

To demonstrate alignment, evidence must be provided that assessment scores support inferences about student achievement and content understanding identified by a given set of expectations (often articulated by a state’s standards). Some assessments are designed to measure a single set of content standards; alignment is sometimes assumed for those cases. Sometimes alignment is demonstrated by asking SMEs to examine several content test forms and standards and judge whether content standards are covered adequately by the test forms. ACT Aspire has been involved in several SME-based alignment studies with each study demonstrating slightly different results depending on the standards examined and on the SMEs who participated.

Strong empirical evidence supports that ACT Aspire scores can be interpreted as measures of college and career readiness. ACT score scales for English, mathematics, reading, and science have been empirically linked to ACT Aspire scores and the ACT College Readiness Benchmarks, which relate scores on the ACT assessment directly to performance in college courses. ACT Aspire Readiness Benchmark scores have been backmapped from student performance data that support inferences of a student being on target for readiness. An empirical link has also been established between the ACT Aspire Composite score and the ACT National Career Readiness Certificate (ACT NCRC), which provides information about student achievement and employable skills in reading, math, and locating information. These research-based connections to real-world college and real-world career performance contribute direct 1.5 General Description of ACT Aspire Assessments and Standards

evidence of alignment from ACT Aspire assessments to college and career readiness skills and knowledge. Therefore, ACT Aspire assessments exhibit strong alignment with standards that target college and career readiness.

ACT has a historical connection to the CCSS, a set of academic standards adopted by many states. In 2008, The National Governors Association (NGA) and the Council of Chief State School Officers (CCSSO) invited ACT and other partners to participate in the development of a set of research-supported college and career readiness standards. ACT’s role included providing data, empirical research, and staff expertise about what constitutes college and career readiness. Due to some of the shared research upon which the CCSS were based, significant overlap exists between them and the college and career readiness constructs used to guide the development of ACT Aspire.

ACT closely monitors and has contributed to the creation of the Next Generation Science Standards (NGSS) that are being implemented in some states. ACT Science content experts presented ACT research on science curricula and career and college readiness with NGSS developers early in the NGSS standard drafting process. Considerable alignment between ACT Aspire and the NGSS exists, however, the ACT Aspire assessments are not designed specifically to assess the NGSS. As previously described, the ACT Aspire science assessments are based on ACT research on current curricula at the elementary, middle, and high school levels as well as ACT research on college and career readiness.

Some areas of achievement are not measured by ACT Aspire summative assessments. Specifically, any standards that require extended time, advanced uses of technology, active research, peer collaboration, producing evidence of a practice over time, or speaking and listening are not currently assessed on ACT Aspire summative assessments.

Further information on the alignment of ACT assessments can be found in the document “How ACT Assessments Align with State College and Career Readiness Standards,” available at: http://www.discoveractaspire.org/pdf/ACT_Alignment- White-Paper.pdf

1.6 CHAPTER 2 ACT Aspire Test Development

2.1 Assessment Design Elements and Construct Coherence ACT Aspire tests are designed to measure student achievement and progress toward college and career readiness. A principal philosophical basis for a longitudinal system of tests such as ACT Aspire is that readiness is best assessed by measuring, as directly and authentically as possible, the academic knowledge and skills that students will need in order to be successful. Coherence across the learning trajectory of college and career readiness constructs is required to support inferences about individual and aggregate growth. Skills and understanding develop over time, which means that students must learn new material and must also apply what they have previously learned in more sophisticated ways. ACT Aspire assessments are designed to measure key college and career readiness constructs in a way that recognizes that knowledge and skills are not isolated to specific grades, but rather progress across grades. Items and tasks are designed to elicit evidence about constructs learned primarily in a particular grade, and also collect evidence about the skill progression across grades. From a measurement perspective, this design is an important component of supporting a vertical scale thus reinforcing inferences about growth. Using this design is also important to develop actionable reports for students across all achievement levels.

Although sampling constructs across grades occurs in all the subject tests, it can be illustrated most clearly by examining the design of the mathematics test. In each form, certain items contribute to Foundation scores. Foundation items are explicitly designed to elicit evidence about a concept that was introduced in a previous year but that the student must now apply in a more advanced context. An illustration of this for the mathematics test is provided in Appendix A. These items are important 2.1 ACT Aspire Test Development

for understanding where a student in the current grade actually falls on the learning trajectory.

As illustrated in Appendix B, the ACT Aspire assessments are designed to be developmentally and conceptually linked across grade levels. To clearly reflect and help interpret that linkage, the reporting categories are associated across all grade levels. The ACT Aspire scores are on the same vertical score scale for each subject. Higher grade levels of ACT Aspire assessments are taken by students who are closest in age and skill development to students taking the ACT and ACT NCRC, and, therefore, these scores have an even stronger empirical bridge to inferring levels of college and career readiness.

2.1.1 ACT Aspire Item types The goal of assessment is to collect relevant evidence from the student as authentically as possible while sampling enough of the construct to support inferences based on the student's responses. ACT Aspire uses multiple item types to achieve this goal.The ACT Aspire assessments measure student knowledge and abilities in the following areas. • Selected-response/multiple-choice (SR/MC) items require the examinee to select a single correct response among several provided choices. • Constructed-response (CR) tasks require students to generate their own response to the questions. • Technology-enhanced (TE) items and tasks incorporate computer interfaces to ask questions and pose scenarios that are not possible in traditional paper-based formats. TE items may require students to generate their responses, or they may present students with a set of answer options.

Each ACT Aspire test contains a combination of these item types. ACT Aspire English tests across all grade levels feature selected-response and TE items. The constructed response part of English is handled in the ACT Aspire writing tests that feature only constructed-response (based on a single writing prompt). For the reading, mathematics, and science tests, each grade-level exam contains selected-response, constructed-response, and technology enhanced items (with selected response items replacing TE items on paper forms).

2.1.2 Cognitive Complexity and Depth of Knowledge (DOK) The cognitive complexity level of written passages and the cognitive demands of an assessment item are important characteristics to consider when measuring a student’s academic achievement. ACT Aspire assessments reflect expectations that students will need to think, reason, and analyze at high levels of cognitive complexity in order to be college and career ready; items and tasks require the sampling of different levels of cognitive complexity, with most items targeted at upper levels. Due to the wide use of Norman Webb’s depth-of-knowledge terminology, this document describes the

2.2 ACT Aspire Test Development

cognitive complexity of items using language such as depth of knowledge (DOK). Given the various definitions of DOK levels, understanding ACT’s interpretation of them is important to understanding the test design.

Similar to Webb’s definition, DOK levels are assigned to reflect the complexity of the cognitive process required, not the psychometric “difficulty” of the item. Unlike other DOK interpretations, ACT only assigns a DOK level 4 value to describe multiday, potentially collaborative classroom activities and assessments designed for learning purposes. By this definition, DOK assignments on any summative assessment (including ACT Aspire) are limited to values of 1 to 3.

ACT’s DOK level 1 corresponds to Webb’s level 1 where students are primarily actively using knowledge and skills with limited extended processing. ACT’s DOK level 2 extends beyond level 1 and involves applying these cognitive processes to many situations, including real-world situations. Therefore, ACT’s DOK level 2 actually aligns with Webb’s DOK level 2 and some of Webb’s DOK level 3. ACT’s DOK level 3 involves situations where the student must apply high-level, strategic thinking skills to short- and long-term situations. Some of these situations are novel and some require generating something like a graph, but all involve higher-level thinking skills. Given this interpretation, ACT’s DOK level 3 aligns with Webb’s DOK level 3 and DOK level 4.

2.2 ACT Aspire Test Development Processes 2.2.1 Selecting and Training Item Writers Item writers are chosen from applicants who demonstrate adequate qualifications (e.g., practicing teachers, subject specialists, curriculum coordinators, department chairs). Item writers have extensive content and pedagogical knowledge and teach at the grade levels covered by ACT Aspire. These educators are actively engaged in teaching at various levels, at a variety of institutions, from small private schools to large public schools.

ACT recruits item writers who represent the diversity found in the United States with respect to ethnic background, gender, English-language proficiency, and geographic location.

Figure 2.1 illustrates the process flow of item and test form development followed by ACT Aspire. The first 3 steps of this process are described in the following sections of this chapter and steps 4-7 are described in subsequent chapters of this technical manual.

2.2.2 Designing Items that Elicit Student Evidence Item writers are instructed to consider the entire construct when crafting assessment tasks and items. Items are designed to elicit evidence about students’ knowledge and skills that span the full construct, which in many cases, is beyond any specific achievement standard language. Item writers use templates that frame what 2.3 ACT Aspire Test Development

knowledge, skills, and abilities are of greatest interest in construct measurement while calling out unintentional knowledge, skills, and abilities that should not be measured. Items must fulfill task template requirements (e.g., content, DOK, word count, accessibility), reflect diversity, and meet fairness standards.

The goal of crafting high quality items or tasks is to design situations to collect relevant evidence from the student in a manner that is as authentic as possible while sampling enough of the construct to support the inferences based on the student's responses.

2.2.3 Item Review All items undergo rigorous content reviews by internal and external content experts to ensure that they solicit sufficient student evidence, are developmentally appropriate, and that the contents and contexts are error-free. The process includes internal peer reviews, internal senior-level reviews, and external reviews. The content reviews also ensure that each item or task measures what is intended and functions at the intended DOK. External experts participate in fairness reviews to ensure that items and tasks are fair and not biased to one student demographic.

Initial forms are constructed to conform to test specifications. Once constructed, forms are reviewed by ACT staff and external content panelists. These panelists evaluate items for content accuracy and evaluate the test form for context appropriateness and representation. They also confirm that the form will collect the necessary evidence to support the intended inferences from scores.

After forms are finalized, they are administered operationally, and they are equated. Before students receive scores, data is carefully checked to ensure items and forms are working as intended.

2.2.4 Field-testing ACT Aspire field testing follows an embedded model in which field test items are spiraled across operational forms. All field test items are prepared and delivered for administration within the same forms as the operational items. This model allows for field test items to be delivered to some students with accommodations. There are no specific directions provided to students or test coordinators about field test items, with the exception that students and educators are alerted that some items may not count toward their overall score.

2.4 ACT Aspire Test Development

National 1. Administration 6.

Item Equating Writers Administration

ACT’s 2. Test Pre-Tryout Development National Contest Staff Forms Content and and Fairness Fairness Reviews Reviews 5.

Tryout National National Forms Administration Initial Construction 3. 4.

Figure 2.1. ACT Aspire field testing model

2.5

CHAPTER 3 Assessment Specifications

3.1 Overview Each test’s design includes a specified range of item types (i.e., selected-response, constructed-response, technology-enhanced) and a specified range of item difficulties at targeted depths of knowledge organized into content-specific reporting categories. The amount and nature of evidence needed to support an inference about a student’s achievement about a given construct are taken into consideration when selecting the types and number of items developed for each reporting category. These requirements are balanced with maintaining manageable administration conditions. ACT Aspire assessments cover topic progressions from foundational concepts to sophisticated applications. Taken as individual subject tests or as a battery, ACT Aspire assessments can be delivered via computer or as a paper-and-pencil administration.

3.2 ACT Aspire Assessment Support Materials 3.2.1 ACT Knowledge and Skills Map The ACT Knowledge and Skills Map is an interactive, online tool that can be used to learn more about ACT’s college and career readiness domain across grades. Each subject area provides a grade-by-grade overview of knowledge and skills domains that research suggests a student should achieve on the path to college and career readiness. In addition to providing a framework for understanding the path to college and career readiness, the ACT Knowledge and Skills Map provides sample items and research citations. http://skillsmap.actlabs.org/map • username: actlabs • password: actlabs

3.1 Assessment Specifications

3.2.2 ACT Aspire Exemplar Items ACT Aspire has developed two resources designed to help students, parents, educators, and policymakers become familiar with ACT Aspire test presentation and content. These resources, the Student Sandbox and the Exemplar Test Question Booklets, illustrate the different types of test questions and formats found in both paper-based and computer-based testing modes. http://actaspire.pearson.com/exemplars.html http://www.discoveractaspire.org/assessments/test-items/

3.3 English Language Arts Assessments Overview ELA knowledge and skills include overlapping constructs related to English, reading, and writing. ACT Aspire ELA assessments are designed to elicit sufficient evidence to support general inferences about student achievement in ELA as well as student achievement in each of these content areas separately. Inferences about student achievement in reading, writing, and English language skills can be made by analyzing ACT Aspire reading, writing, and English test scores, respectively. ACT Aspire reporting uses language consistent with college and career readiness standards in ELA and literacy so that students, parents, and educators can understand performance on the test in terms that clearly relate to instruction.

Students who take ACT Aspire English, reading, and writing tests receive a Composite ELA score that shows how their overall performance compares to the readiness benchmark at their grade. The benchmark is based on empirical research of students who have been identified as college and career ready. The Composite ELA score is reported in an effort to provide insight about student skills that are integrated across the English Language Arts. This perspective aligns with the integrated approach to literacy education in leading achievement standards frameworks.

3.3.1 English Test The English test puts the student in the position of a writer who makes decisions to revise and edit a text. Short texts and essays in different genres provide a variety of rhetorical situations. Students must use the rich context of the passage to make editorial choices, demonstrating their understanding of grammar, usage, and mechanics conventions. Students must also apply understanding of rhetorical purposes and strategies, structure and organize sentences and paragraphs for effective communication, and maintain a consistent style and tone.

Figure 3.1 shows an example of the construct hierarchy for the English test.

3.3.1.1. English Framework The English Framework articulates constructs measured across the grade levels. Each grade is broken out and each has descriptive statements of knowledge and skills measured within each reporting category. Please see Appendix E for more details. 3.2 Assessment Specifications

3.3.1.2 English Reporting Categories The English test measures student knowledge and skill in the following reporting categories. Production of Writing The items in this category require students to apply their understanding of the rhetorical purpose and focus of a piece of writing to develop a topic effectively and to use various strategies to achieve logical organization, topical unity, and general cohesion. • Topic Development: These items require students to demonstrate an understanding of, and control over, the rhetorical aspects of texts by identifying the purposes of parts of texts, determining whether a text or part of a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Organization, Unity, and Cohesion: These items require students to use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion.

Knowledge of Language The items in this category require students to demonstrate effective language use through ensuring precision and concision in word choice and maintaining consistency in style and tone. Conventions of Standard English The items in this category require students to apply an understanding of the conventions of standard English grammar, usage, and mechanics to revise and edit text. • Punctuation, Usage, and Capitalization Conventions: These items require students to edit text to conform to standard English punctuation, usage, and capitalization. • Sentence Structure and Formation: These items test understanding of relationships between and among clauses, placement of modifiers, and shifts in sentence construction.

Reporting Categories Topic Development Production of Writing Organization ACT Aspire Knowledge of English Language

Conventions of Standard English ACT Skill Targets

Figure 3.1. ACT Aspire English test reporting with example skill targets for one reporting category. 3.3 Assessment Specifications

3.3.1.3 English Item Types The ACT Aspire English Test puts the student in the position of a writer who makes decisions to revise and edit a text. Different passage types provide a variety of rhetorical situations, each accompanied by multiple- choice or constructed-response items.

3.3.1.4 English Test Blueprints The test consist of texts (such as essays, sentences, and paragraphs), each accompanied by selected-response and, on computer-based forms of the test, technology-enhanced test items (on paper forms, technology-enhanced items are replaced by selected-response items). Different essay genres are employed to provide a variety of rhetorical situations. Texts vary in length depending on the grade level. The stimuli are chosen not only for their appropriateness in assessing writing skills, but also to reflect students’ interests and experiences. Some questions refer to underlined or highlighted portions of the essay and offer several alternatives to the portion underlined. These items include “NO CHANGE” to the underlined or highlighted portion in the essay as one of the possible responses. Some questions ask about a section of a paragraph or an essay, or about the paragraph or essay as a whole. The student must decide which choice best answers the question posed. Non-operational items are included in the total number of items in Tables 3.1–3.4 but do not contribute towards raw-score points.

Table 3.1. Specification Ranges by Item Type and Reporting Category for Grade 3 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 21–23 21–23 (84–92%) Technology enhanced 2–4 2–4 (8–16%) Reporting Categories Production of Writing 8–10 8–10 (32–40%) Conventions of Standard English 15–17 15–17 (60–68%) Depth of Knowledge DOK Level 1 13–14 13–14 (52–56%) DOK Level 2 3–7 3–7 (12–28%) DOK Level 3 4–8 4–8 (16–32%) Non-operational Field-test 6–10 TOTAL: 31–35* 25 (100%)* *Total number of items includes field-test items that do not contribute to score points.

3.4 Assessment Specifications

Table 3.2. Specification Ranges by Item Type and Reporting Category for Grades 4–5 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 21–23 21–23 (84–92%) Technology enhanced 2–4 2–4 (8–16%) Reporting Categories Production of Writing 6–8 6–8 (24–32%) Knowledge of Language 2–3 2–3 (8–12%) Conventions of Standard English 15–16 15–16 (60–64%) Depth of Knowledge DOK Level 1 13–14 13–14 (52–56%) DOK Level 2 3–7 3–7 (12–28%) DOK Level 3 4–8 4–8 (16–32%) Non-operational Field-test 6–10 TOTAL: 31–35* 25 (100%)* *Total number of items includes field-test items that do not contribute to score points.

Table 3.3. Specification Ranges by Item Type and Reporting Category for Grades 6–8 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 33–35 33–35 (94–100%) Technology enhanced 0–2 0–2 (0–6%) Reporting Categories Production of Writing 10–12 10–12 (29–45%) Knowledge of Language 3–6 3–6 (29–45%) Conventions of Standard English 19–20 19–20 (55–61%) Depth of Knowledge DOK Level 1 15–17 15–17 (43–49%) DOK Level 2 6–9 6–9 (17–26%) DOK Level 3 11–12 11–12 (31–34%) Non-operational Field-test 9–12 — TOTAL: 44–47* 35 (100%) *Total number of items includes field-test items that do not contribute to score points.

3.5 Assessment Specifications

Table 3.4. Specification Ranges by Item Type and Reporting Category for Grade EHS Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 48–50 48–50 (96–100%) Technology enhanced 0–2 0–2 (0–4%) Reporting Categories Production of Writing 13–14 13–14 (26–28%) Knowledge of Language 6–7 6–7 (12–14%) Conventions of Standard English 30–31 30–31 (60–62%) Depth of Knowledge DOK Level 1 22–24 22–24 (44–48%) DOK Level 2 10–12 10–12 (20–24%) DOK Level 3 15–17 15–17 (30–34%) Non-operational Field-test 9–10 — TOTAL: 59–60* 50 (100%)* *Total number of items includes field-test items that do not contribute to score points.

3.3.2 Reading Test The reading test measures a student’s ability to read closely, reason logically about texts using evidence, and integrate information from multiple sources. Passages in the reading test include both literary narratives, such as prose fiction, memoirs, and personal essays, and informational texts from the natural sciences and social sciences. Across the grades, these texts span a range of complexity levels that lead up to the reading level in challenging first-year college courses. On the test at each grade, passages at different levels of complexity within a range appropriate for the grade present students different opportunities to demonstrate understanding. In addition, reading items assess the student’s ability to complete reading- related tasks at various depth-of-knowledge (DOK) levels, and, as a whole, the tests reflect a range of task difficulty appropriate for the grade level.

Figure 3.2 shows an example of the construct hierarchy for the Reading test.

3.3.2.1 Reading Framework The Reading Framework articulates constructs measured across the grade levels. Each grade is broken out and each has descriptive statements of knowledge and skills measured within each reporting category. Please see Appendix F for more details. 3.6 Assessment Specifications

3.3.2.2 Reading Reporting Categories The Reading Test assesses skills in the following reporting categories. Key Ideas and Details • The items in this category require students to read texts closely; to determine central ideas and themes, and summarize information and ideas accurately; and to understand relationships and to draw logical inferences and conclusions including understanding sequential, comparative, and cause-effect relationships. Craft and Structure • The items in this category require students to determine word and phrase meanings, analyze an author’s word choice rhetorically, analyze text structure, and understand authorial purpose and characters’ points of view. They interpret authorial decisions rhetorically and differentiate between various perspectives and sources of information. Integration of Knowledge and Ideas • The items in this category require students to understand authors’ claims, differentiate between facts and opinions, and use evidence to make connections between different texts that are related by topic. Students read a range of informational and literary texts critically and comparatively, making connections to prior knowledge and integrating information across texts. They analyze how authors construct arguments, evaluating reasoning and evidence from various sources.

Reporting Categories Close Reading

Key Ideas Central Ideas, and Details Themes and Summaries Craft and Structure ACT Aspire Relationships Reading Integration of Knowledge and Ideas

Progress with Skill Text Complexity Targets

Figure 3.2. ACT Aspire reading test reporting categories with example skill targets for one reporting category

3.3.2.3 Reading Test Complexity and Types of Texts During the development of the reading test, ACT evaluates reading passages using both quantitative and qualitative analyses. This information is used to determine the levels of text complexity of passages and to build assessments with passages at a range of text complexity appropriate for each grade level.

Table 3.5 shows ACT Aspire’s five levels of text complexity. These levels were established through ACT’s research and validated through a review process that included ELA teachers. Together with tasks that target important reading skills at each 3.7 Assessment Specifications

grade, these complexity levels ensure a coherent progression across the assessment continuum. At each grade level, the Reading Test includes a range of text complexities as specified in the table. Note that a sixth level of complexity, Highly Complex, is not included on the table because those passages are only administered on the ACT.

ACT research staff has analyzed the effect of text complexity on reading comprehension. Drawing on this research, ACT designed a measure that reports student performance on specific reading tasks that require comprehension and integration of information across texts. In addition to the overall reading score, each student receives this Text Complexity Progress Measure. While the overall reading score provides a measure of important comprehension skills, this score offers a focused measure of the skills that a reader uses to draw inferences, make connections, and integrate meaning across the challenging texts found in educational and workplace settings.

Table 3.5. Passage Coverage by Text Complexity for the ACT Aspire Reading Test Grade Level 3 4 5 6 7 8 EHS

Basic X X X Straightforward X X X X X Somewhat challenging X X X X X More challenging X X Complex X

Performance on the Text Complexity Progress Measure is compared to a readiness level empirically derived from ACT College Readiness Benchmarks. Students who perform above the benchmark level will receive an indication that they are making sufficient progress toward reading the complex texts they will encounter in college and career. Students who perform below the benchmark level will receive recommendations for improvement such as practicing reading appropriately complex texts from a variety of genres, monitoring understanding, and using other strategies to comprehend literary and informational texts.

ACT Aspire Reading passages are drawn from the following range of text types: • Literary Narrative: Literary passages from short stories, novels, memoirs, and personal essays • Social science: Informational passages on topics such as anthropology, archaeology, biography, business, economics, education, environmentalism, geography, history, political science, psychology, and sociology • Natural science: Informational passages on topics such as anatomy, astronomy, biology, botany, chemistry, ecology, geology, medicine, meteorology, microbiology, natural history, physiology, physics, technology, and zoology

3.8 Assessment Specifications

Passage Types

Literary Narrative

ACT Aspire Social Reading Science

Natural Science

Figure 3.3. ACT Aspire reading test item types. The reading test contains selected response, constructed response, and technology enhanced items (on the CBT version).

Examples of constructed-response tasks in the reading test include: • Make claims about a passage and provide support using specific details from the text • Identify similarities and differences between the key ideas of paired passages and provide support using specific details from both texts.

Constructed-response tasks are scored according to rubrics that give students varying amounts of credit for responses that are correct or partially correct, enabling differentiation between multiple skill levels.

3.3.2.4 Reading Test Blueprints The Reading test comprises four sections in grades 3 through 5 and three sections in grades 6 through EHS. Each section contain one prose passage that is representative of the level and kinds of text commonly encountered in school curriculum at that grade. Each passage is accompanied by a set of multiple-choice test items, and, on computer-based forms of the test, some sections include technology-enhanced and constructed-response items (technology-enhanced items are replaced by multiple- choice items on paper forms). These items focus on complementary and mutually supportive skills that readers use to understand texts written for different purposes across a range of subject areas. Items do not test the rote recall of facts from outside the passage or rules of formal logic, nor do they assess vocabulary in isolation from the context. Additionally, every test includes a constructed response item that introduces a second shorter passage on the same topic as the primary passage. These items require the student to synthesize information from across the texts. Non- operational items are included in the total number of items provided in Tables 3.6–3.7, but do not contribute towards raw-score points.

3.9 Assessment Specifications

Table 3.6. Specification Ranges by Item Type and Reporting Category for Grades 3–7 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 18–20 18–20 (62–69%) Technology enhanced 1–3 1–3 (3–10%) Constructed response 3 8 (28%) Reporting Categories Key Ideas and Details 13–15 14–18 (48–62%) Craft and Structure 7–9 7–11 (24–38%) Integration of Knowledge and Ideas 2–3 4–6 (14–21%) Depth of Knowledge DOK Level 1 2–5 2–6 (8–21%) DOK Level 2 12–16 15–21 (50–67%) DOK Level 3 5–8 6–10 (21–33%) Non-operational Field-test 5–7 — TOTAL 29–31* 29 (100%) *Total number of items includes field-test items that do not contribute to score points.

Table 3.7. Specification Ranges by Item Type and Reporting Category for Grades 8–EHS Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 20–21 20–21 (65–68%) Technology enhanced 0–1 0–1 (0–3%) Constructed response 3 10 (32%) Reporting Categories Key Ideas and Details 14–15 17–18 (45–58%) Craft and Structure 6–9 7–11 (23–35%) Integration of Knowledge and Ideas 1–3 4–6 (13–19%) Depth of Knowledge DOK Level 1 5–7 7–9 (23–29%) DOK Level 2 9–13 12–17 (38–54%) DOK Level 3 5–9 7–12 (21–38%) Non-operational Field-test 6–8 — TOTAL 30–32* 31 (100%) *Total number of items includes field-test items that do not contribute to score points. 3.10 Assessment Specifications

Table 3.8. Items by Passage Type for the ACT Aspire Reading Test Grade Level 3 4 5 6 7 8 EHS LN 10–11 10–11 10–11 7 7 7 7 INFO 10–11 10–11 10–11 14 14 14 14 SSC 5–6 5–6 5–6 7 7 7 7 NSC 5–6 5–6 5–6 7 7 7 7 Note. LN = Literary Narrative (prose fiction, memoirs, personal essays); INFO = Informational Text (SSC = Social Science and NSC = Natural Science)

3.3.3 Writing Test The ACT Aspire Writing Test consists of a single 30-minute summative writing task at each grade. The tasks target one of three primary modes of writing: reflective narrative, analytical expository, or persuasive/argumentative. Students respond in essay form to a writing stimulus. The assessments are designed to provide an indication of whether students have the writing skills they will need to succeed at the next level. All writing tasks have been assigned to DOK Level 3.

3.3.3.1 Writing Task Types by Grade Because there is one extended writing task at each grade level, ACT Aspire rotates through the three modes to ensure coverage across the grades.

The reflective narrative mode appears at grades 3 and 6. The analytical expository mode appears at grades 4, 7, and early high school. The persuasive/argumentative mode appears at grades 5 and 8. The ACT Aspire assessments are designed to give students at every grade level an opportunity to display the higher-order thinking skills needed for meaningful reflection, analytical explanation, or persuasive argumentation.

Table 3.9. Tasks by Writing Mode for Each Grade of the Writing Test

Grade Writing Mode

3 Reflective Narrative 4 Analytical Expository 5 Persuasive Argumentative 6 Reflective Narrative 7 Analytical Expository 8 Persuasive Argumentative EHS Analytical Expository ACT Persuasive Argumentative

3.11 Assessment Specifications

3.3.3.2 Writing Scoring Rubrics and Reporting Categories The Writing Test is scored with a four-domain analytic scoring rubric. Each grade level has a unique rubric because the writing tasks assess different writing modes, but the underlying design is the same across grades.

Each of the four rubric domains corresponds to a different trait of the writing sample; traits in the writing sample are evidence of the following writing competencies:

Reflective Narrative/Analysis/Argument The name of the first rubric domain corresponds to the mode of writing assessed at the grade level. Regardless of the mode, this rubric domain is associated with the writer’s generation of ideas. Scores in this domain reflect the ability to generate productive ideas and engage with the writing task. Depending on the mode, writers generate ideas to provide reflection, analysis, or persuasive and reasoned argument. Competent writers understand the topic they are invited to address, the purpose for writing, and the audience. They generate ideas that are relevant to the situation. Development and Support Scores in this domain reflect the writer’s ability to develop ideas. Competent writers explain and explore their ideas, supporting them with reasons, examples, and detailed descriptions. Their support is well integrated with their ideas. They help the reader understand their thinking about the topic. Organization Scores in this domain reflect the writer’s ability to organize ideas with clarity and purpose. Competent writers arrange their writing in a way that clearly shows the relationship between ideas, and they guide the reader through their reflection, analysis, or argument about the topic. Language Use Scores in this domain reflect the writer’s ability to use language to convey their ideas with clarity. Competent writers make use of the conventions of grammar, syntax, and mechanics. Their word choice is precise, and they are also aware of their audience, adjusting voice and tone to enhance their purpose. The reporting categories on the writing test correspond to the rubric domains.

Table 3.10. Writing Test Scoring Rubric Domains and Associated Reporting Categories

Scoring Rubric Domain Reporting Category

Reflective Narrative/Analysis/Argument Ideas and Analysis (corresponds to writing mode) Development Development and Support Organization Organization Language Use Language Use and Conventions 3.12 Assessment Specifications

Ideas and Analysis Reﬂective Narrative Development and Support ACT Aspire Analytical Writing Expository Organization Persuasive Argumentative Language Use and Conventions

Writing Reporting Modes Categories

Figure 3.4. ACT Aspire Writing test modes and reporting categories At grades 3 through 5, the rubrics have five performance levels; at grade 6 and above, the rubrics differentiate among six performance levels. At each grade level and in each domain, a score of 4 is associated with “adequacy,” indicating that a student who achieves this score is on track for success upon entering the next grade level. The five-point rubrics for grades 3 through 5 allow for only one degree of performance above adequate, whereas the six-point rubric allows for two degrees of differentiation above “adequate.” A score of 5 at these grades indicates an advancing level of skill.

The analytic rubrics delineate four dimensions of writing, but traits are scored holistically within each domain. Descriptors within each rubric domain level are not intended to function as independent features assessed separately. Instead, scorers use those elements collectively to form a holistic evaluation of the student’s ability in that domain.

Table 3.11. Points by Content Category for the Writing Test Raw-score points (percentage of test) Content Category Grades 3-5 Grades 6-EHS

Ideas and Analysis 5 (25%) 6 (25%) Development and Support 5 (25%) 6 (25%) Organization 5 (25%) 6 (25%) Language Use and Conventions 5 (25%) 6 (25%) TOTAL 20 (100%) 24 (100%)

Further details about the Writing Rubrics for grades 3–EHS can be found in Appendix F.

3.4 Mathematics Test The mathematics test measures the whole of a student’s mathematical development, breaking it into progress with topics new to the grade and progress with lasting 3.13 Assessment Specifications

topics from previous grades. The mathematics test provides nine reporting categories, facilitating actionable insights.

The ACT Aspire mathematics assessments emphasize quantitative reasoning frequently applied to real-world contexts. The mathematics construct requires making sense of the problem, context, and notation; accessing appropriate mathematical knowledge from memory; incorporating the given information; doing mathematical calculations and manipulations; interpreting; applying thinking skills; providing justification; making decisions based on the mathematics; and communicating results. Knowledge of basic formulas and computational skills are assumed as background for the problems, but memorization of complex formulas and extensive computation are not required.

The mathematical content assessed at a given grade is based on mathematical learning expectations up through that grade that are important to future college and career readiness. Test tasks focus on what students can do with the mathematics they have learned, which encompasses not only content but also mathematical practices.

3.4.1 Mathematics Reporting Categories Sometimes tasks will cut across these domains; they will be assigned to the most appropriate single domain for reporting purposes. All tasks involve some level of mathematical practices. Reporting categories are specific to grade bands indicated in the parentheses following each category name. EHS indicates early high school.

Number & Operations in Base 10 (K–5) Starting from an understanding of the base 10 system involving 1s and 10s and 100s, students extend to 100s, 1,000s, and further, working toward seeing place- value connections between each place and the neighboring places. Students use these place-value connections to extend their number system to include decimals. Students can compute using place-value understanding, flexibly combining and decomposing groups. This thinking undergirds mental mathematics for one’s lifetime, and being able to explain and critique strategies helps cement these important skills. Concurrently, students are seeing the need for efficient computation, and they understand and can explain the procedures they use. By the end of this grade band, students use place-value strategies fluently for mental arithmetic and explain their reasoning. By the end of this grade band, students compute fluently using efficient computation strategies with multidigit whole numbers and decimals to hundredths. Number & Operations—Fractions (3–5) Students develop an understanding of several useful interpretations of fractions. There are connections from unit fractions to equal sharing, and representations of a whole broken into equal-size pieces. But fractions are also numbers, each with a place on the number line and a role in computation. Students compute

3.14 Assessment Specifications

with fractions (up through division with a unit fraction and a whole number) and understand how fraction computation is built on the properties of operations with whole numbers. They can explain why a step in a procedure is necessary. The Number System (6–8) Students coming into this grade band have nearly finished extending whole- number operations to work with fractions, and they complete the task, understanding division of one fraction by another. Then the cycle continues: students extend the four operations to work for all rational numbers. Students understand the order of rational numbers, which now allows them to plot points in all four quadrants of the coordinate plane, and they understand absolute value in terms of distance. Students solve problems and use properties of the rational numbers to aid in computation. They clearly tie their computations into their arguments and draw conclusions from conditional statements. Students can convert rational numbers to decimal and to fraction form. But not all numbers are rational numbers, and by the end of this grade band students understand that a number like √2 is not rational and they can work with approximations. Number & Quantity (EHS) Coming into high school, students have knowledge of the real number system and they have strong understanding and fluency with rational numbers and the four basic operations so they can work with irrational numbers by working with rational numbers that are close. Students are ready to move from integer exponents to rational exponents. And they are ready to probe a little deeper into properties of the real number system. Later, students extend to complex numbers, which offer the solutions to some simple equations that have no real-number solutions, and students learn to compute in this system. Students are invited to go further, exploring properties of complex numbers and therefore learning more about real numbers. Students are invited to explore vectors and matrices and to view them as number systems with properties, operations, and applications. Throughout high school, students are maturing in their understanding of quantity and its connections to measurement. Attending to the types of quantities and units can guide solution strategies and help avoid errors. Students work with derived quantities, and when modeling they choose appropriate quantities to model. Operations & Algebraic Thinking (K–5) Students bring an understanding of addition and subtraction as well as counting by 2s and 5s to this grade band and use it to develop an understanding of multiplication in terms of combining equal-size groups and of division as forming equal-size groups for sharing. Arrays and rectangular area provide models for multiplication and division, and from those connections, students learn about properties of operations, using those properties to aid in computation. Solving problems, working with factors and multiples, and learning in other domains all contribute to fluency with all four operations and the ability to explain reasoning.

3.15 Assessment Specifications

Students use conditional statements (when that is true, then this is true) mainly with specific numbers but working toward general statements. Numerical expressions and equations capture calculations and relationships, and working with number patterns develops operational sense as well as moving toward the concept of function. Expressions & Equations (6–8) This category is closely related to Ratio and Proportional Reasoning/Functions, building a foundation of algebraic understanding. Prior to this grade band, students routinely write numerical expressions. Now students routinely write expressions using a letter to represent a number and are working to attach meaning to expressions and to understand how the properties of numbers are useful for working with expressions. Integer exponents come into play for representing very large and very small quantities, and later, students extend to use rational exponents. Students need to understand what it means for a number to be a solution to an equation, and that solving an equation involves a process of reasoning about solutions, justified by the properties of numbers and operations. Linear equations are the focus, and students use graphs and equations to express relationships. Lines (excepting vertical lines) have a well-defined slope, and the concurrent geometric progress with similar triangles allows students to prove this. Students understand the connections between proportional relationships, lines, and linear equations. Reasoning about the solution of linear equations is applied to solve linear inequalities and to find simultaneous solutions of a pair of linear equations. Ratios & Proportional Reasoning (6–7) For 8th grade, proportional relationships are multiplicative relationships between quantities and are the next step toward general linear relationships. Students coming into this grade band start with strong multiplication skills and expand the ways of working with “how many times as much?” comparisons, including ratios, rates, and percentages. They look at rate-of-change and connect unit rates with the slope of a graph, a pattern in a table, and a multiplier in an equation, using all forms of rational numbers. This builds to general understanding of function as input and output pairs and understanding of proportional functions and then linear functions. Functions are not tied to a specific representation: tables can show representative values, graphs can show behavior in a representative region, equations can represent some types of functions. Linear equations y = mx + b represent all linear functions. The Expressions and Equations domain provides ways of working with these equations. Linear functions are useful in modeling. Functions (8–EHS) Appropriate relationships are multiplicative relationships between quantities and are the next step toward general linear relationships. Students coming into this grade band start with strong multiplication skills and expand the ways of working with “how many times as much?” comparisons, including ratios, rates, and 3.16 Assessment Specifications

percentages. They look at rate-of-change and connect unit rates with the slope of a graph, a pattern in a table, and a multiplier in an equation, using all forms of rational numbers. This builds to general understanding of function as input and output pairs and understanding of proportional functions and then linear functions. Functions are not tied to a specific representation: tables can show representative values, graphs can show behavior in a representative region, equations can represent some types of functions. Linear equations y = mx + b represent all linear functions. The Expressions and Equations domain provides ways of working with these equations. Linear functions are useful in modeling. For EHS, functions have been with students since their early years: consider the counting function that takes an input of “seven” and gives “eight” and an input of “twelve” to give “thirteen.” In grade 8, the concept of function was named and became an object of study. Understanding some general properties of functions will equip students for problem solving with new functions they create over their continued studies and careers. Functions provide a framework for modeling real-world phenomena, and students become adept at interpreting the characteristics of functions in the context of a problem and become attuned to differences between a model and reality. Some functions accept all numbers as inputs, but many accept only some numbers. Function notation gives another way to express functions that highlights general properties and behaviors. Students work with functions that have no equation, functions that follow the pattern of an equation, and functions based on sequences, which can even be recursive. Students investigate particular families of functions—like linear functions, quadratic functions, and exponential functions—in terms of the general function framework: looking at rates of change, algebraic properties, and connections to graphs and tables, and applying these functions in modeling situations. Students also examine a range of functions like those defined in terms of square roots, cube roots, polynomials, and exponentials, and also piecewise-defined functions. Algebra (EHS) Students coming into high school build on their understanding of linear equations to make sense of other kinds of equations and inequalities: what their graphs look like, how to solve them, and what kinds of applications they have for modeling. Students develop fluency with algebra. They continue to make sense of expressions in terms of their parts in order to use their fluency with strategy and to solve problems. Through repeated reasoning, students develop general understanding of solving equations as a process that provides justification that all the solutions will be found, and students extend to quadratic equations, polynomial equations, rational equations, radical equations, and systems, integrating an understanding of solutions in terms of graphs. Families of equations have properties that make them useful for modeling. Solutions of polynomial equations are related to factors of a polynomial. Students recognize relationships in applications and create expressions, equations, and inequalities to represent problems and constraints. 3.17 Assessment Specifications

Geometry (K–EHS) For grades 3–5, students start this grade band recognizing shapes in terms of attributes, which now makes for interrelated categories of shapes that share certain attributes (e.g., the category of parallelograms is made up of all shapes where each of the 4 sides is parallel to the opposite side). Viewed as categories, squares make up a subcategory of parallelograms, and so all the properties of parallelograms apply to squares. In general terms, the attributes of a category apply for all subcategories. Shapes can be made from points, lines or line segments, angles, and other shapes. Shapes can have lines of symmetry. Shapes can be decomposed, which has connections to fractions. All of these concepts can be used to solve problems. Students can find examples and counterexamples. Toward the end of this grade band, students extend number-line concepts to the coordinate plane and can plot points in the first quadrant and make interpretations in the context of solving real-world problems. For grades 6–8, this grade band starts with a focus on area, surface area, and volume. The additive property of area means the area of a polygon can be found by decomposing it into simpler figures the student already can find the area of. Through repeated reasoning, and in conjunction with the Expressions and Equations domain, students bring meaning to formulas for perimeter/ circumference, area, surface area, and volume, and solve problems involving triangles, special quadrilaterals, and polygons, with attention to symmetry, working up to circles, cubes, right prisms, cylinders, cones, and spheres. Students develop the spatial reasoning to describe the cross-section of 3-dimensional figures sliced by a plane. Students address geometric distance. In the coordinate plane, they start plotting polygons and finding the length of horizontal and vertical sides. Later they reach the Pythagorean Theorem and its converse, justifying these relations and using them to solve problems in 2 and 3 dimensions and in a coordinate plane. They understand how to use scale. Students delve into creating figures that meet given conditions based on angle measures and lengths and symmetry, noting when the conditions lead to a unique figure. And this leads to the notion of congruence and then to similarity in terms of dilations, translations, rotations, and reflections. Students learn about these transformations and establish facts about, for example, angle sums in a triangle, and use these facts to write equations and solve problems. For EHS, building on their knowledge from grade 8, students add depth to what they know about transformations, reflections, rotations, and dilations and add precision to their understanding of congruence, similarity, and symmetry. Justification is the glue that holds structure together and lets students answer “Why?” questions, and students justify by using definitions and theorems, tying in calculations and diagrams, considering cases, understanding general versus specific statements, applying counterexamples, and putting statements together into coherent arguments. Students make constructions, solve problems, and model with geometric objects. Informal arguments give a chain of reasoning that leads to formulas for the area of a circle and then on to volume of cylinders, pyramids, and 3.18 Assessment Specifications

cones. Students solve right-triangle problems. All of these results transfer to the coordinate plane, where analytic treatment of distance allows students to derive conditions for parallel and perpendicular lines, to split a line segment into pieces with a given ratio of lengths, to find areas, and to develop equations. Measurement & Data (K–5) Measurement and Data topics in this grade band provide mutual reinforcement with topics from other domains: unit conversions related to scaling and multiplication, whole numbers and fractions related to measurement and operations. Students solve problems that involve units and conversions, starting with time. They have a framework for measurement and can work with measurements as data for line plots and scaled picture graphs, and they can interpret these displays during problem solving. Students develop the concepts of perimeter and area through measurement and relate these to multiplication and addition, moving on to the concept of volume with its relationships to measurement, multiplication, and addition. Students understand angle measure and connect this to fractions of a circle. Statistics & Probability (6–EHS) For grades 6–8, statistics is about distributions of quantities—like the amount of electricity used by each home in Anchorage, Alaska, last month—and how to tell how likely something is—given what is known about the shape of the distribution and being sure to take into account the context of the data and the data-gathering process. Students have already graphed distributions, but now they pay more attention to the shape of the distribution, particularly where it seems to be centered and how spread out it is. Box plots capture some of the differences between distributions in a way that makes comparison surer. Students make informal inferences and learn about randomness in sampling. Probability is a language for conveying likelihood, and students develop uniform probability models and probability models from empirical data. They can use lists, tables, tree diagrams, and simulation results to represent sample spaces and estimate the probability of compound events. For EHS, students add to their understanding of distributions of a single quantity, describing center and spread with statistics and interpreting these in the context of the data. Before high school, students have used 2-way tables and scatter plots to look at relationships between different quantities, and have used linear functions to model relationships that look linear. Now students pay more attention to informal model fit and use other functions to model relationships; they use models for prediction and they interpret characteristics of the model in the context of the data. From 2-way tables, students interpret relative probabilities (including joint, marginal, and conditional relative frequencies but not tied to these terms) and relate these to probabilities. Students look for association and distinguish correlation and causation. 3.19 Assessment Specifications

Randomness unlocks the power of statistics to estimate likelihood, and students learn about the role of randomness in sample surveys, experiments, and observational studies. Students use data to estimate population mean or proportion and make informal inferences based on their maturing judgment of likelihood. They can compare qualities of research reports based on data and can use simulation data to make estimates and inform judgment. Before high school, students have tacitly used independence, but now the idea is developed with a precise definition. Students relate the sample space to events defined in terms of “and,” “or,” and “not,” and calculate probabilities, first using empirical results or independence assumptions. Figure 3.5 shows the general flow of the mathematical domains across the grade levels.

Figure 3.5. Mathematics Domains, and Primary Currents in the Flow Across Grades For each grade, assessment of student progress within the 5 domains of topics new to the grade is complemented by assessment of student progress with lasting topics from previous grades. Not only should students retain what they learned in previous grades, they should actually strengthen and integrate their learning and what they can do with those topics.

3.20 Assessment Specifications

Figure 3.6. Foundation and Grade Level Progress (illustrated for Grade 7) Progress with topics new to a grade is reflected in the Grade Level Progress reporting category. Progress with lasting topics from previous grades is reflected in the Foundation reporting category. Figure 3.6 shows how these components fit together. The sampling within the Grade Level Progress component is done in such a way to create a reporting category around each of the 5 domains of topics new to the grade. This additional layer of detail contributes greater precision to support insights about students.

Mathematical Practices Mathematical Practices highlight cross-cutting mathematical skills and understandings and the complex and vital ways they integrate with content. Test items focus on important mathematics, which includes various levels of using mathematical practices. Selection of items for a form takes mathematical practices into account, providing a balance across content and mathematical practices.

In addition to assessing mathematical practices as a general part of scores, ACT Aspire reports individually on two important mathematical practices: Justification & Explanation and Modeling.

Justification & Explanation Justification & Explanation tasks focus on the whys of mathematics. These are a direct measure, using a constructed-response format, of how well students function with mathematical argument, as appropriate for the given grade. Each task involves 3.21 Assessment Specifications

Justification & Explanation as it applies to some mathematical content. Scores across 4 or 5 Justification & Explanation tasks are reported so that students and teachers can look in more depth at this important mathematical practice and how it is developing in each student. The tasks also contribute to the Grade Level Progress reporting category or to the Foundation reporting category according to the mathematical content involved. The number of Justification & Explanation tasks, raw-score points, and percent of raw-score points is captured in Table 3.12 by grade. Justification & Explanation tasks are each worth 4 raw-score points.

Table 3.12. Number of Justification & Explanation Tasks, Raw-Score Points (and Percentage Weight) in Reporting Categories by Grade

Grade Justification & Explanation (JE) 3–5 6 8–EHS

JE tasks overall 4 4 5 JE weight overall 16 pts (43%) 16 pts (35%) 20 pts (38%) JE tasks in Grade Level Progress 2 2 3 JE weight in Grade Level Progress 8 pts (36%) 8 pts (29%) 12 pts (36%) JE tasks in Foundation 2 2 2 JE weight in Foundation 8 pts (53%) 8 pts (44%) 8 pts (40%) EHS = Early High School

The Justification & Explanation rubric, for all grades, is based on a single progression of Justification & Explanation skills, providing a connected measure of progress from grade to grade. The skills are divided into 3 levels at each grade, but these groupings differ depending on the grade. The more advanced skills are at level 3. Some Justification & Explanation skills at level 3 for grade 3 become level 2 skills in later grades, and some skills at level 2 become level 1 skills in later grades.

Justification & Explanation tasks for a grade are designed to elicit evidence of skills at level 2 or level 3 for the grade, meaning that a successful solution to the task will necessarily demonstrate Justification & Explanation skills at level 2 or level 3, respectively. Most tasks that elicit Justification & Explanation skills at a given level will also elicit Justification & Explanation skills at all lower levels.

Modeling Modeling involves two objects: the actual and the model. A model often helps one predict or understand the actual. The Modeling reporting category represents items that involve producing, interpreting, understanding constraints of, evaluating, and improving models.

There are a minimum number of raw-score points specified for a form for Modeling. Table 3.13 shows the minimum number by grade. Table 3.14 and Table 3.15 show the 3.22 Assessment Specifications

actual average number for 2 forms, and the actual average percentage for those 2 forms, respectively.

Table 3.13. Modeling Minimum Number (and Percentage) of Raw-Score Points by Reporting Category and Grade

Grade Modeling 3–5 6–7 8–EHS Minimum number (percentage) of 7 (19%) 9 (20%) 11 (21%) raw-score points overall Minimum number (percentage) in Grade 5 (22%) 6 (21%) 7 (21%) Level Progress Minimum number (percentage) in 2 (14%) 3 (17%) 4 (20%) Foundation Note. Percentages are rounded

Table 3.14. Modeling Actual Average Number of Raw-Score Points for 2 Forms by Grade Grade Modeling 3 4 5 6 7 8 EHS

Average number of raw-score points 16 13 15 17.5 17.5 21.5 20.5 Average number in Grade Level 10.5 9 8 13 10 12.5 11.5 Progress Average number in Foundation 5.5 4 6 4.5 7.5 9 9

Table 3.15. Modeling Actual Average Percentage of Raw-Score Points for 2 Forms by Grade Grade Modeling 3 4 5 6 7 8 EHS

Average percentage overall 43% 35% 39% 38% 38% 41% 41% Average percentage in Grade Level 46% 39% 29% 46% 36% 38% 35% Progress Average percentage in Foundation 39% 29% 43% 25% 42% 45% 45% Note. Percentages are rounded

3.4.2 Calculator Policy Students taking the mathematics test for grade 6 or higher are encouraged to bring a calculator they are familiar using and can use fluently. A calculator tool is available within the online version. Students are permitted to use most any 4-function, scientific, or graphing calculator. The ACT calculator policy for ACT Aspire is available at www. act.org.

3.23 Assessment Specifications

3.4.3 Mathematics Item Types, Tasks, Stimulus The Mathematics Test uses a specified variety of item types to collect appropriate evidence. Table 3.16 below shows the percentage of raw-score points that come from each item type. These percentages reflect targets and may vary slightly.

Table 3.16. Target Percentage of Raw-Score Points by Item Type and Grade

Grade Item Type 3–5 6–7 8–EHS Technology Enhanced 15% 12% 9% Multiple Choice 42% 53% 53% Constructed Response 43% 35% 38%

Item Sets Some test items are part of a set, presented as a common stimulus used for all the items in the set and test items whose solution must utilize some information from the common stimulus. Test items in the set are independent of one another, meaning that getting the correct answer to one item does not hinge on getting the correct answer to another item.

Each form has approximately 2 item sets, with 2–5 items in each set. Items may be technology enhanced, multiple-choice, or constructed response.

Justification & Explanation Justification & Explanation tasks require students to read and understand a problem, and then generate a response. These tasks are DOK 3. Grade Level Progress has a mix of DOK 3, DOK 2, and DOK 1 items. The Foundation reporting category is looking for growth in what students can do with content from prior years and therefore uses only DOK 2 and DOK 3. Table 3.17 shows the target distribution of DOK by reporting category and grade. Form distributions may vary slightly from these targets. See 2016 Updates on page U.1 for updates to these specifications for 2015–16.

3.24 Assessment Specifications

Table 3.17. Target for Raw-Score Points (Percentage of Test) [Percentage of Reporting Category] by DOK and Grade

Grade Reporting DOK Category Level 3–5 6–7 8–EHS

Overall DOK 3 21 (57%) 24 (52%) 29 (55%) DOK 2 12 (32%) 17 (37%) 18 (34%) DOK 1 4 (11%) 5 (11%) 6 (11%) Grade Level DOK 3 10 (27%) [45%] 12 (26%) [43%] 16 (30%) [48%] Progress DOK 2 8 (22%) [36%] 11 (24%) [39%] 11 (21%) [33%] DOK 1 4 (11%) [18%] 5 (11%) [18%] 6 (11%) [18%] Foundation DOK 3 11 (30%) [73%] 12 (26%) [67%] 13 (25%) [65%] DOK 2 4 (11%) [27%] 6 (13%) [33%] 7 (13%) [35%] DOK 1 none none none Note. Percentages may not sum to 100% due to rounding.

3.4.4 Mathematics Test Blueprints The Mathematics Test is composed of different item types (i.e., multiple choice, technology enhanced, and constructed response). There are some nuances to be aware of across delivery modes (i.e., computer and paper) and within reporting categories.

Numbers of the different item types provided in Table 3.18 show a representative range of numbers of items. The reporting category specifications are based on points, not on numbers of items. Technology Enhanced items are replaced by MC items on paper forms. Item counts for Justification & Explanation and Modeling are also included in the counts of items in Grade Level Progress and Foundation. Non- operational items are included in the total number of items, but do not contribute towards raw-score points.

3.25 Assessment Specifications

Table 3.18. Specification Ranges by Item Type and Reporting Category for Grades 3-5 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 17–20 17–20 (46–54%) Technology enhanced 1–4 1–4 (3–11%) Constructed response 4 16 (43%) Reporting Categories Grade Level Progress 17 23 (62%) Numbers & Operations in Base 10 3–4 3–4 (8–11%) Numbers & Operations—Fractions 3–4 3–4 (8–11%) Operations & Algebraic Thinking 3–4 3–4 (8–11%) Geometry 3–4 3–4 (8–11%) Measurement & Data 3–4 3–4 (8–11%) Foundation 8 14 (38%) Justification & Explanation 4 16 (43%) Modeling ≥12 ≥12 (≥48%) Depth of Knowledge DOK Level 1 3–5 3–5 (8–14%) DOK Level 2 11–13 11–13 (30–35%) DOK Level 3 8–10 20–22 (54–59%) Non-Operational Items Field-test 3–6 – TOTAL 28–31* 37 (100%)

*Total number of items includes field-test items that do not contribute to score points.

3.26 Assessment Specifications

Table 3.19. Specification Ranges by Item Type and Reporting Category for Grades 6-7 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 23–25 23–25 (50–54%) Technology enhanced 5–7 5–7 (11–15%) Constructed response 4 16 (35%) Reporting Categories Grade Level Progress 22 28 (61%) Number System 4–5 4–5 (9–11%) Expressions & Equations 4–5 4–5 (9–11%) Ratio & Proportion Reasoning 4–5 4–5 (9–11%) Geometry 4–5 4–5 (9–11%) Statistics & Probability 4–5 4–5 (9–11%) Foundation 12 18 (39%) Justification & Explanation 4 16 (35%) Modeling ≥14 ≥14 (≥41%) Depth of Knowledge DOK Level 1 4–6 4–6 (9–13%) DOK Level 2 16–18 16–18 (35–39%) DOK Level 3 11–13 23–25 (50–54%) Non-Operational Items Field-test 4–9 – TOTAL 38–43* 46 (100%)

*Total number of items includes field-test items that do not contribute to score points.

3.27 Assessment Specifications

Table 3.20. Specification Ranges by Item Type and Reporting Category for Grade 8 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 29–30 29–30 (55–57%) Technology enhanced 3–4 3–4 (6–8%) Constructed response 5 20 (38%) Reporting Categories Grade Level Progress 23–24 32–33 (60–62%) Number System 2–3 2–3 (4–6%) Expressions & Equations 6–7 6–7 (11–13%) Functions 4–5 4–5 (8–9%) Geometry 5–6 5–6 (9–11%) Statistics & Probability 3–4 3–4 (6–8%) Foundation 14–15 20–21 (38–40%) Justification & Explanation 5 20 (38%) Modeling ≥15 ≥15 (≥39%) Depth of Knowledge DOK Level 1 5–7 5–7 (9–13%) DOK Level 2 17–19 17–19 (32–36%) DOK Level 3 13–15 28–30 (53–57%) Non-Operational Items Field-test 5–8 – TOTAL 43–46* 53 (100%)

*Total number of items includes field-test items that do not contribute to score points.

3.28 Assessment Specifications

Table 3.21. Specification Ranges by Item Type and Reporting Category for EHS Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 29–30 29–30 (55–57%) Technology enhanced 3–4 3–4 (6–8%) Constructed response 5 20 (38%) Reporting Categories Grade Level Progress 23–24 32–33 (60–62%) Number & Quantity 2–3 2–3 (4–6%) Functions 4–5 4–5 (8–9%) Algebra 5–6 5–6 (9–11%) Geometry 5–6 5–6 (9–11%) Statistics & Probability 4–5 4–5 (8–9%) Foundation 14–15 20–21 (38–40%) Justification & Explanation 5 20 (38%) Modeling ≥17 ≥17 (≥45%) Depth of Knowledge DOK Level 1 5–7 5–7 (9–13%) DOK Level 2 17–19 17–19 (32–36%) DOK Level 3 13–15 28–30 (53–57%) Non-Operational Items Field-test 6–8 — TOTAL 44–46* 53 (100%) *Total number of items includes field-test items that do not contribute to score points.

Note that for the EHS level, topics new to Grade 8 are included in the GLP reporting category and not the Foundation reporting category. This reporting organization groups topics together, particularly algebra topics, for more coherence in score interpretation.

3.29 Assessment Specifications

3.5 Science Test The ACT Aspire science assessments are designed based on research evidence about what students need to know and be able to do in order to be on a trajectory where they will be successful in college and career.

Only the ACT Aspire science test questions contribute to students’ science scores— questions on the ACT Aspire math, reading, English, and writing tests are not repurposed to contribute to the science score. Analyses of correlations of other ACT Aspire tests support that the Science test score represents unique achievement information. NCS results also show that educators at all levels, from elementary through postsecondary, report that science is better assessed using a science test rather than a reading, writing, or math test. Testing reading in science is important and aligns with most college and career readiness state standards related to literacy, and because of that, reading in science is assessed on the ACT Aspire reading test.

3.5.1 Science Reporting Categories The science test assesses and reports on science knowledge, skills, and practices across three domains: • Interpretation of Data • Scientific Investigation • Evaluation of Models, Inferences, and Experimental Results

The knowledge, skills, and practices contained in these domains comprise the ACT College and Career Readiness Standards for Science, which link specific skills and knowledge with quantitatively determined score ranges for the ACT science test and a benchmark science score that is predictive of success in science at the postsecondary level. The science test is built on these same skills—skills that students need to learn early, and then continually apply and refine, in order to be on a path to college and career readiness in science. All items on the science test are based on authentic scientific scenarios that are built around important scientific concepts and are designed to mirror the experiences of students and working scientists engaging in real science. Some of the items require that the students have discipline-specific content knowledge (e.g., for the Early High School test, knowledge specific to an introductory middle school biology course), but science content is always assessed in concert with science skills and practices. ACT’s research on science curricula and instruction at the high school and post-secondary levels shows that while science content is important, science skills and practices are more strongly tied to college and career readiness in science.

The content of the science test includes biology (life sciences at the earlier grades), chemistry and physics (physical science at the earlier grades), and Earth/space sciences (e.g., geology, astronomy, and meteorology). Advanced knowledge in these areas is not required, but background knowledge acquired in general, introductory

3.30 Assessment Specifications

science courses may be needed to correctly respond to some of the items in the upper-grade assessments. The assessments do not, however, sample specific content knowledge with enough regularity to make inferences about a student’s attainment of content knowledge in any broad area, or specific part, of the science content domain. The science test stresses science practices over recall of scientific content, complex mathematics skills, and reading ability. When taken with the mathematics test, scores from the two tests are combined to provide a STEM (science, technology, engineering, mathematics) score.

3.5.2 Continuum of Complexity There are a wide variety of scenarios in which students may need to apply their scientific knowledge and reasoning skills. Scenarios are categorized along two dimensions on ACT Aspire: Mode and Complexity. Different modes focus on different aspects of science (e.g., scientific data/results, experimentation, and debate). Complexity is determined by several factors that contribute to how challenging a scientific scenario is likely to be for a student. ACT Aspire focuses on five factors in particular: the topic (e.g., weather, photosynthesis rates), the tone (i.e., formality), the nature of the presented data (i.e., the density, content, and structure of graphs, tables, charts, and diagrams), and the difficulty of the embedded concepts (i.e., the introduced and assumed concepts, symbols, terminology, units of measure, and experimental methods). These elements combine in various proportions to result in scenarios of different complexity, and this complexity affects suitability for different grade levels. This complexity also impacts the difficulty of questions and tasks students are asked to perform in a given context (e.g., an easy question can become much more difficult when posed in a more challenging context).

The science test development process and item design is informed by science education research in learning progressions and misconceptions. This research informs the item task designs and test designs to assure that science knowledge, skills, and practices are tested at developmentally appropriate times and in appropriate ways and at increasing levels of sophistication from grade 3 through EHS. Core skills and practices are similar across the grade continuum, as our research shows that these skills and practices should be taught early, and continually refined, as a student progresses through school. What differs most from grade to grade in the science test is the complexity of the contexts in which science knowledge, skills, and practices are assessed. The contexts used on the tests place the students in rich and authentic scientific scenarios that require that the student apply their knowledge, skills, and practices in science to both familiar and unfamiliar situations. With increasing grade level, the scenarios have increasingly complex scientific graphics and experimental designs and are based on increasingly complex concepts. The students must then apply their knowledge, skills, and practices with increasing levels of sophistication. The amount of assumed discipline-specific science content knowledge a student needs to answer questions on the tests also increases. Figure 3.7 illustrates this continuum.

3.31 Assessment Specifications

Figure 3.7. ACT and ACT Aspire science content complexity continuum. 3.5.3 Science Item Types ACT Aspire science items and tasks require students to apply what they have learned in their science classes to novel, richly contextualized problems. The test is structured to collect evidence about students’ ability to, for example, evaluate the validity of competing scientific arguments or to scrutinize the designs of experiments, and to find relationships in the results of those experiments. Technology-enhanced items and tasks provide various ways to collect unique evidence about how students engage with scientific material. To fully cover this complexity continuum, six different stimulus (passage) modes are employed across the Science Test, which vary from grade 3 through Early High school. These include using modes focused on data interpretation, scientific experimentation, or scientific debate.

Table 3.22 shows how many passages from which stimulus modes are used at each grade level.

Conflicting Viewpoints

A passage format used in grades 8 and EHS in which two or more differing scientific explanations/arguments on the same topic are given, sometimes with one or more graphs, tables, and/or diagrams. Items focus on understanding the explicit and implicit points of each explanation, comparing and contrasting the explanations, drawing conclusions from the arguments, and evaluating the validity of the arguments.

Research Summaries A passage format used for grades 7–EHS involving a detailed experimental procedure along with one or more graphs, tables, and/or diagrams for experimental setups and results. (Technology enhancements involve gathering the data by manipulating variables.) Items focus on aspects of experimental design, accompanying procedures, obtaining and analyzing data, and drawing conclusions from the data. 3.32 Assessment Specifications

Data Representation A passage format used for grades 6–EHS in which a brief description is given for one or more graphs, tables, and/or diagrams. Items focus on obtaining and analyzing data and drawing conclusions from the data. Technology enhancements include videos, animations, and data manipulations (e.g., moving data points to affect a trend).

Student Viewpoints A passage format used for grades 6 and 7 that is analogous to conflicting viewpoints but with one to three brief and simple explanations/arguments that support an observation (e.g., students may have differing interpretations of a data set or teacher demonstration). Items focus on understanding the explicit and implicit points of the argument(s), drawing conclusions from an argument, and evaluating the validity of an argument.

Science Investigations A passage format used for grades 3–7 that is analogous to research summaries but is shorter, simpler, more student-centered experimental procedures along with one to two graphs, tables, and/or diagrams for experimental setups and results. (Technology enhancements involve gathering the data by manipulating variables.) Items are focused on aspects of experimental design, accompanying procedures, obtaining and analyzing data, and drawing conclusions from the data.

Data Presentation A passage format used for grades 3–5 that is analogous to data representation, but data sources are simpler (and rarely more than two per unit) and more related to everyday life, with briefer descriptions. Technology enhancements include videos, animations, and data manipulations (e.g., moving data points to affect a trend).

Table 3.22. Stimulus Modes used on the ACT Aspire Science Test

Grade Level Stimulus Mode 3 4 5 6 7 8 EHS Conflicting Viewpoints 1 1 Research Summaries 1 2 2 Data Representation 1 1 2 2 Student Viewpoints 1 1 Science Investigation 2 2 2 2 1 Data Presentation 2 2 2

3.33 Assessment Specifications

Table 3.23. ACT Aspire Science College and Career Readiness Knowledge and Skill Domain College & Career Reporting Readiness Category Skill Area Skill Code Skill Statement Standards Interpretation Locating and IOD-LU-01 Select one piece of data from a data presentation IOD 201, 401 of Data (IOD) Understanding IOD-LU-02 Find information in text that describes a data IOD 203, 303 presentation IOD-LU-03 Select two or more pieces of data from a data IOD 301, 401 presentation IOD-LU-04 Identify features of a table, graph, or diagram (e.g., axis IOD 202 labels, units of measure) IOD-LU-05 Understand common scientific terminology, symbols, and IOD 302 units of measure Inferring and IOD-IT-01 Translate information into a table, graph, or diagram IOD 403 Translating IOD-IT-02 Determine how the value of a variable changes as the IOD 304, 503 value of another variable changes in a data presentation IOD-IT-03 Compare data from a data presentation (e.g., find the IOD 402, 502 highest/lowest value; order data from a table) IOD-IT-04 Combine data from a data presentation (e.g., sum data IOD 501, from a table) 601, 701 IOD-IT-05 Compare data from two or more data presentations (e.g., IOD 501, compare a value in a table to a value in a graph) 601, 701 IOD-IT-06 Combine data from two or more data presentations (e.g., IOD 501, categorize data from a table using a scale from another 601, 701 table) IOD-IT-07 Determine and/or use a mathematical relationship that exists between data (e.g., averaging data, unit IOD 504, 602 conversions) Extending and IOD-ER-01 Perform an interpolation using data in a table or graph IOD 404, 603 Reevaluating IOD-ER-02 Perform an extrapolation using data in a table or graph IOD 404, 603 IOD-ER-03 Analyze presented data when given new information (e.g., IOD 505, 702 reinterpret a graph when new findings are provided) Scientific Locating and SIN-LC-01 Find information in text that describes an experiment SIN 201, 303 Investigation Comparing (SIN) SIN-LC-02 Identify similarities and differences between experiments SIN 404 SIN-LC-03 Determine which experiments utilized a given tool, SIN 405 method, or aspect of design

3.34 Assessment Specifications

Table 3.23. (continued) College & Career Reporting Readiness Category Skill Area Skill Code Skill Statement Standards

Scientific Designing and SIN-DI-01 Understand the methods, tools, and functions of tools SIN 202, 301, Investigation Implementing used in an experiment 302, 402 (SIN) SIN-DI-02 SIN 401, 403, (continued) Understand an experimental design 501 SIN-DI-03 Determine the scientific question that is the basis for an SIN 601 experiment (e.g., the hypothesis) SIN-DI-04 Evaluate the design or methods of an experiment (e.g., possible flaws or inconsistencies; precision and accuracy SIN 701 issues) Extending and SIN-EI-01 Predict the results of an additional trial or measurement SIN 502 Improving in an experiment SIN-EI-02 Determine the experimental conditions that would SIN 503 produce specified results SIN-EI-03 Determine an alternate method for testing a hypothesis SIN 602 SIN-EI-04 Predict the effects of modifying the design or methods of SIN 702 an experiment SIN-EI-05 Determine which additional trial or experiment could be SIN 703 performed to enhance or evaluate experimental results Evaluation Inferences EMI-IE-01 Determine which hypothesis, prediction, or conclusion is, of Models, and Results: or is not, consistent with a data presentation or piece of EMI 401, 601 Inferences, Evaluating and information in text and Extending EMI-IE-02 Determine which experimental results support or Experimental EMI 505 Results (EMI) contradict a hypothesis, prediction, or conclusion EMI-IE-03 Determine which hypothesis, prediction, or conclusion is, or is not, consistent with two or more data presentations EMI 501, 701 and/or pieces of information in text EMI-IE-04 Explain why a hypothesis, prediction, or conclusion is, or is not, consistent with a data presentation or piece of EMI 401, 601 information in text EMI-IE-05 Explain why presented information, or new information, EMI 502, 702 supports or contradicts a hypothesis or conclusion EMI-IE-06 Explain why a hypothesis, prediction, or conclusion is, or is not, consistent with two or more data presentations EMI 501, 701 and/or pieces of information in text

3.35 Assessment Specifications

Table 3.23. (continued) College & Career Reporting Readiness Category Skill Area Skill Code Skill Statement Standards

Evaluation Models: EMI-MU-01 Find information in a theoretical model (a viewpoint EMI 201 of Models, Understanding proposed to explain scientific observations) Inferences, and Comparing EMI-MU-02 Identify implications and assumptions in a theoretical and (Grade 6 and EMI 301, 402 Experimental higher) model Results (EMI) EMI-MU-03 Determine which theoretical models present or imply EMI 302, 403 (continued) certain information EMI-MU-04 Identify similarities and differences between theoretical EMI 404 models (Grade 6 and Models: EMI-ME-01 Determine which hypothesis, prediction, or conclusion EMI 401, 601 higher) Evaluating and is, or is not, consistent with a theoretical model Extending EMI-ME-02 Determine which hypothesis, prediction, or conclusion is, EMI 501, 701 or is not, consistent with two or more theoretical models EMI-ME-03 Identify the strengths and weaknesses of theoretical EMI 503 models EMI-ME-04 Determine which theoretical models are supported or EMI 504 weakened by new information EMI-ME-05 Determine which theoretical models support or contradict EMI 505 a hypothesis, prediction, or conclusion EMI-ME-06 Use new information to make a prediction based on a EMI 603 theoretical model EMI-ME-07 Explain why presented information, or new information, EMI 602 supports or weakens a theoretical model EMI-ME-08 Explain why a hypothesis, prediction, or conclusion is, or EMI 401, 601 is not, consistent with a theoretical model EMI-ME-09 Explain why a hypothesis, prediction, or conclusion is, or EMI 501, 701 is not, consistent with two or more theoretical models

3.36 Assessment Specifications

3.5.4 Science Test Blueprints The test presents several sets of scientific information, each followed by a number of test items. The scientific information is conveyed in one of three different formats: data representation (graphs, tables, and other schematic forms), research summaries (descriptions of several related experiments), or conflicting viewpoints (expressions of several related hypotheses or views that are inconsistent with one another).

Tables 3.24–3.26 show the makeup of the ACT Aspire Science Tests for each grade level. Technology Enhanced items are replaced by MC items on paper forms. Non- operational items were included in the total number of items, but do not contribute towards raw-score points.

Table 3.24. Specification Ranges by Item Type and Reporting Category for Grades 3–5 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 19–20 19–20 (53–56%) Technology enhanced 3–4 3–4 (6–8%) Constructed response 6 14 (39%) Reporting Categories Interpretation of Data 12–17 14–19 (39–53%) Scientific Investigation 6–8 8–11 (22–31%) Evaluation of Models, Inferences, and 5–8 9–13 (25–36%) Experimental Results Depth of Knowledge DOK Level 1 4–7 4–11 (11–31%) DOK Level 2 13–20 16–27(44–75%) DOK Level 3 3–8 8–15 (22–42%) Non-Operational Items Field-test 5–6 – TOTAL 33–34* 36 (100%)

*Total number of items includes field-test items that do not contribute to score points.

3.37 Assessment Specifications

Table 3.25. Specification Ranges by Item Type and Reporting Category for Grades 6–7 Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 22–23 22–23 (55–58%) Technology enhanced 3–4 3–4 (8–10%) Constructed response 6 14 (35%) Reporting Categories Interpretation of Data 18–19 19–20 (48–50%) Scientific Investigation 6–8 9–11 (23–28%) Evaluation of Models, Inferences, and 5–8 9–12 (23–30%) Experimental Results Depth of Knowledge DOK Level 1 2–8 2–9 (5–23%) DOK Level 2 15–25 19–30 (48–75%) DOK Level 3 4–9 10–18 (25–45%) Non-Operational Items Field-test 5–6 – TOTAL 37–38* 40 (100%)

*Total number of items includes field-test items that do not contribute to score points.

3.38 Assessment Specifications

Table 3.26. Specification Ranges by Item Type and Reporting Category for Grades 8–EHS Raw-score points Operational Items Number of Items (percentage of test) Item Types Multiple choice 23–24 23–24 (58–60%) Technology enhanced 3–4 3–4 (8–10%) Constructed response 5 13 (33%) Reporting Categories Interpretation of Data 12–15 17–19 (43–48%) Scientific Investigation 7–9 9–11 (23–28%) Evaluation of Models, Inferences, and 10–11 11–12 (28–30%) Experimental Results Depth of Knowledge DOK Level 1 2–8 2–9 (5–23%) DOK Level 2 15–25 18–29 (45–73%) DOK Level 3 3–9 11–18 (28–45%) Non-Operational Items Field-test 6–8 –– TOTAL 38–40* 40 (100%)

*Total number of items includes field-test items that do not contribute to score points.

3.39

CHAPTER 4 Item and Task Scoring

4.1 Selected-Response Item Scoring Selected-response items require students to select a correct answer from several alternatives. Each correct selected-response item has a value of 1 point. Incorrect, missing responses (items that a student did not answer) and multiple responses have a value of zero points.

4.2 Technology-Enhanced Items Scoring Technology-enhanced require students to make decisions about how to correctly answer questions. Some TE items require more than one decision or action. All TE items are dichotomously scored (correct response is 1 point; incorrect or missing responses scored as zero points).

4.3 Constructed-Response Tasks Student responses to constructed-response tasks are read and scored by trained raters. Training materials for field-testing of constructed-response tasks were created during range finding. Range finding involves test development specialists, content experts, and expert raters previewing student responses to determine whether the content-specific scoring criteria for each task accurately reflect and encompass all of the acceptable student responses. The initial round of range finding also includes analysis and validation of the scoring rubrics used across tasks. Once the tryout tasks are scored, they undergo statistical analysis focusing on difficulty, validity, and accessibility to determine whether they are suitable for operational use. During range finding, a selection of student responses is individually rated by multiple raters using the scoring criteria and the appropriate rubric. Responses that do not receive the

4.1 Item and Task Scoring

same score by all raters are discussed by the entire group and a consensus score is reached. Additions and clarifications to the scoring criteria can be made at this time, if necessary.

After test development specialists and expert raters have completed range finding, responses are sorted and put together into training sets. Training materials for the qualification item include an anchor set, multiple practice sets, and at least two qualification sets that prospective raters must score accurately in order to be eligible to score field-test responses.

Responses chosen for the anchor set represent clear examples of each score point. Practice set and qualification set responses also include clear examples as well as responses that are not as perfectly aligned to the scoring criteria; these frequently occurring responses are less straightforward and fall either slightly high or low within each score point, making them challenging to score.

Each response included in the training materials is analyzed during range finding. An articulation (rationale) explaining why a particular score was assigned accompanies the response. The articulation explains how the rubric and scoring criteria were used to determine the score; citations from the exemplar response are included, where appropriate, to illustrate the claims made in the articulation.

4.3.1 Performance Scoring Quality Control 4.3.1.1 Scorer Qualifications and Experience Scoring quality starts with the recruitment process and extends through screening and placement (assigning scorers to items based on their skills and experience), training, qualification, and scoring.

Priority is given to scorers with previous experience in scoring similar assessments. In most cases, these professional scorers have specialized educational and professional experience, including valuable experience in performance scoring. All scorers have, at a minimum, a four-year college degree.

The pool of scorers typically reflects a cross section in terms of age, ethnicity, and gender although placement and retention of scorers is based upon their qualifications and the quality and accuracy of their scoring.

4.3.1.2 Managing Scoring Quality Scorer exception processing is used at defined intervals to check scorer accuracy. Scorers who fall below predetermined standards receive automatically generated messages that interrupt their scoring and direct them to work with a scoring supervisor to review anchor papers or take other steps to improve their scoring accuracy.

4.2 Item and Task Scoring

Validity responses are prescored responses strategically interspersed in the pool of live responses in order to gauge scorer accuracy. These responses are not distinguishable from live responses, but scores are compared to the true scores assigned by scoring supervisors and do not factor into final student scores in any way. The validity mechanism provides an objective and systematic check of accuracy. It verifies that scorers are applying the same standards throughout the project and, therefore, guards against scorer drift and ultimately group drift.

Backreading is the primary tool for proactively guarding against scorer drift. Scoring supervisory staff review these responses to confirm that the scores were correctly assigned and to give customized feedback and remedial training to individual scorers. Backreading scores will override the first score.

Frequency distribution reports show a breakdown of score points assigned on a given task. Expressed in percentages, data in these reports show how often scorers, individually and as a group, assign each score point. A project may have general expectations for frequency distribution ranges, which are typically based on historical data.

Ten percent of ACT Aspire CR student responses will be read and scored independently by two different scorers. Second scoring allows scoring supervisors to closely monitor the performance of scorers, though these scores do not contribute to a student’s final reported score. Supervisors will use the second scores to provide scoring inter-rater reliability statistics and monitor scorer performance.

Inter-rater reliability is the agreement between the first and second scores assigned to student responses. Inter-rater reliability measurements include exact, adjacent, and nonadjacent agreement. Scoring supervisors use inter-rater reliability statistics as one factor in determining the needs for continuing training and intervention on both individual and group levels.

The electronic scoring system is capable of purging the scores assigned by a scorer whose work is deemed substandard. Scoring supervisors can reset scores by individual scorer, date range, or item, whereby the scores assigned by an individual are cleared from the database and the affected responses are reset. The responses are then randomly rerouted to qualified scorers and rescored according to the original scoring design.

Reports used to monitor quality and project completion status are generated and updated automatically and are available to scoring supervisors at any time via the digital scoring system. The reports give daily and cumulative statistics and provide individual and group average agreement percentages. Table 4.1 summarizes the reports.

4.3 Item and Task Scoring

Calibration sets are used to reinforce range finding standards, communicate scoring decisions, or correct scoring issues and trends. The primary goal of calibration is to continue training and reinforce the scoring standards. Calibration set responses may be “on the line” between score points, or they may contain unusual examples that are challenging to score, and therefore useful for reinforcing the scoring rubric.

Table 4.1. Scorer Performance Reports

Report Name Description Daily/Cumulative Inter-Rater Group-level summary of both daily and cumulative inter- Reliability Summary rater reliability statistics for each day of the scoring project. Frequency Distribution Task-level summary of score point distribution percentages Report on both a daily and a cumulative basis. Daily/Cumulative Validity Summary of agreement for validity reads of a given task on Summary both a daily and a cumulative basis. Completion Report Breakdown of the number of responses scored and the number of responses in each stage of scoring (first score, second score, resolution). Performance Scoring Quality Summary of task-level validity and inter-rater reliability on Management Report a daily and cumulative basis. This report also shows the number of resolutions required and completed, and task- level frequency distribution.

4.3.2 Automated Scoring Combining human and AI scoring provides an effective and efficient process for scoring student responses while maintaining scoring quality.

For ACT Aspire Constructed Response (CR) items in Writing, Science, and Reading, we provide a combined human and artificial intelligence approach to scoring. We train an automated scoring engine on responses collected during the field test to determine which items can be satisfactorily scored using automated scoring. Rigorous statistical measures are used to ensure that the scores provided by each item’s automated scoring calibration are comparable to, and interchangeable with, human scores. This entails comparing human and automated scores by exact agreement, Quadratic Weighted Kappa correlation, and Conditional Agreement by Scorepoint. Automated scoring is only used for an item if the item’s calibration passes the established performance benchmarks for these key calibration approval criteria.

To provide an effective and efficient solution, our goal is for the CR responses to be scored using automated scoring as the first score with a 10% second score by human scorers.

4.4 Item and Task Scoring

Overview of the Automated Scoring Process Automated scoring for ACT Aspire uses a machine-learning approach in which the engine is trained to score based on the collective wisdom of trained human scorers. Using a sample of human-scored student responses, the engine learns the different features that human scorers evaluate when scoring a response and how they weigh and combine those features to produce a score. The process begins with several hundred to one thousand or more human-scored student responses. The collection of student responses is divided into two sets, one set used to train and tune the scoring engine and the other used to validate performance.

The performance of automated scoring depends on both the sample of student responses available and the performance of human scorers. The goal is to have as much, and as accurate information as possible about how a response should be evaluated. Pearson’s human scorers will double-score the field test responses, applying resolution for non-adjacent disagreements, to fully support automated scoring. Having double-scored human responses is critically important. First, double- scoring provides more information about the true quality of a constructed response that can be used when training the automated scoring engine. Knowing that a response is on the cusp between two score points, a high 2 or a low 3 for example, provides a more complete picture of the score range. Second, double-scoring provides a measure of human performance against which automated scoring performance can be evaluated.

Involving Humans in the Automated Scoring Process Humans play a key role in maintaining quality throughout the scoring process, even when automated scoring is used. Not only are human scorers involved in the selection of responses to train the scoring engine, but human scorers also second score 10% of the responses in order to provide an additional quality check of the engine. Human scoring is further monitored through increased validity distribution and through supervisor backreading—key indicators of scoring accuracy.

If human scores are discrepant from the automated scoring engine, an expert human scorer reviews the response to resolve the discrepancy.

Finally, if the automated scoring engine determines it cannot score a response, that response is routed to a human scorer. This is particularly useful with responses that appear to be off-topic, not English, highly unusual, or creative.

4.5 Item and Task Scoring

Training the Automated Scoring Engine To learn to score essays, the automated scoring engine incorporates a range of machine learning and natural language processing technologies to evaluate the structure, style, and content of writing in the responses. The automated scoring engine uses a variety of semantic analysis techniques to generate semantic similarity of words and passages by analyzing large bodies of relevant text. The automated scoring engine can then “understand” the meaning of text much the same as a human reader.

Figure 4.1 illustrates some of the features used by the automated scoring engine to identify key components of student writing performance.

Figure 4.1. Components of student writing performance. To evaluate aspects of the essay related to development, audience, content, and organization, the automated scoring engine derives semantic models of English based on a statistical analysis of large volumes of text. For essay scoring applications, it is trained on a collection of text that is equivalent to the reading a student is likely to have done over the course of their academic career (about 12 million words).

Because semantic analysis operates over the semantic representation of texts, rather than at the word level, it is able to evaluate similarity even when texts have few words in common. For example, semantic analysis finds the following two sentences to have a high semantic similarity despite having no words in common: Surgery is often performed by a team of doctors. On many occasions, several physicians are involved in an operation.

4.6 Item and Task Scoring

Semantic analysis allows the automated scoring engine to compare a new response against a large set of prescored responses, using the scores of the most similar responses to help determine the score of the new response. This comparison is similar to the approach that human scorers take when evaluating responses. Human scorers are trained based on anchor papers of annotated student responses with agreed upon scores. Human scorers compare new responses against the anchor set to determine the appropriate score.

The automated scoring engine scores responses similarly, except that it is able to make comparisons against a much larger set of examples. Rather than comparing a new response against a set of 16-24 examples in an anchor set, for example, it compares against the set of hundreds or thousands of responses on which it was trained. It is thus able to take advantage of a much wider range of examples that have been scored and validated based on all of the training and knowledge that goes into human scoring and the methods that are used to monitor and maintain human scoring quality.

Role of Automated Scoring Automated scoring is used in a carefully calculated combination with human scoring of the Constructed Response items, with the intent to reduce scoring turnaround times and lead to more rapid reporting, while still maintaining scoring reliability and consistency.

Enhancing Human Scoring with an Automated System AI technology enhances human scoring rather than replacing it. In the traditional scoring model, two human scorers assess each essay. In an AI model, the scoring engine will apply one score and a human the 10% second score. Consistent Results with Artificial Intelligence Scoring The automated scoring engine has been used in a variety of assessment settings over two decades to score responses written by regular education students, English learners, and students with disabilities. Indeed, automated scoring can isolate and evaluate different aspects of an essay without bias. As a result, students who express good ideas, but perhaps do not use standard English, can still be rewarded for their ideas.

4.7 Item and Task Scoring

4.8 CHAPTER 5 Accessibility

ACT Aspire uses a variety of levels of accessibility support including default embedded tools, open access tools, and full accommodations to allow students with disabilities to participate in testing. For more information about ACT Aspire accommodations, see the ACT Aspire Accessibility User’s Guide.

5.1 Development of the ACT Aspire Accessibility Support System The Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014), address fairness in testing as the central concern posed by the threat to validity known as measurement bias. The Standards specify two major concepts that have emerged in the literature for minimizing such bias, namely accessibility and Universal Design. Accessibility is defined as, “the notion that all test takers should have an unobstructed opportunity to demonstrate their standing on the construct(s) being measured.” (p.49). The second major concept, Universal Design, is defined as, “an approach to test design that seeks to maximize accessibility for all intended examinees” (p.50).

The development of all ACT Aspire accessibility supports followed a theory of action known as Access by Design (Fedorchak, 2013) which incorporates into its conceptual structure elements of Universal Design for Learning (UDL) described by the Center for Applied Special Technologies (CAST, 2011), and Evidence Centered Design (Mislevy, Almond, & Loves, 2004; Mislevy & Haertel, 2006).

As illustrated in Appendix B, the ACT Aspire assessments are designed to be developmentally and conceptually linked across grade levels. To reflect that linkage, the assessment names are the same across all ACT Aspire levels and the scores are on the same vertical score scale. Higher grade levels of ACT Aspire are directly

5.1 Accessibility

connected with the ACT and, therefore, offer an empirical bridge to understanding college and career readiness.

Universal Design is necessary but it is not sufficient by itself to meet the needs of every learner with diverse needs (UDL Guidelines, Version 2.0, Principle III, 2011). Multiple means of engagement are required to meet the needs of all users. To build a system of accessibility supports that meets the needs and provides a fair performance pathway for all learners, a system of accessibility levels of support was identified and created using a structured data collection procedure designed at ACT and called Accessibility Feature Mapping. The development team conducted a detailed feature analysis of every passage and item to determine which accessibility supports could be permitted or provided for at least 7 diverse learner populations that would fully honor the constructs being measured. The data collected from this feature mapping process served to inform the accessibility policy and support system for ACT Aspire. (The ACT Aspire Accessibility Levels of Support resulting from this procedure are described more fully in section 5.2.)

The Accessibility Feature Mapping Process involved creating a data chart for every item in every form for all audio scripted ACT Aspire items. Every feature within each item (passage, stem, equations, graphics, interaction type; etc.) was cross- inventoried by: 1. Constructs being measured (original content metadata targets) and 2. Cognitive Performance demands of the item: (Presentation demands, Interaction & Navigation demands, Response demands and General Test Condition demands). Then for each of 7 targeted learner populations (see schematic below) valid access pathways to this item (either already available or shown to be needed by the analysis) were identified and compared with targets to ensure that all task performance pathways led to valid measurement outcomes and did not violate construct intended for item. The schematic components of this development procedure are shown below:

Components of the Accessibility Feature Mapping Process used to determine valid communication performance pathways (accessibility support options) to be allowed during ACT Aspire Testing.

5.2 Accessibility

What claim(s) does performance on this item support? Task/Item Content Metadata-Claims (as available) Claim level 1: Content Area Claim level 2: Broad subarea w/in content Claim level 3: Domain Claim level 4: Primary Cluster Claim level 5: Secondary Cluster, as applicable

What performance does Who has a valid pathway to demonstrating the required this task require? performance? Communication Demands of the Known Communication Access Pathways Task Based on Claims Usable/Needed by learners with: (boundaries of Acceptable Performance Evidence): 1. Default/Typical communication access needs 2. Blind/Low Vision communication access needs 1. Task Presentation demands 3. Deaf/Hard of Hearing communication access needs 2. Task Interaction & Navigation 4. Limited Motor Control communication access needs demands 5. English Language learner communication access needs 3. Task Response demands, and 6. Reading or Language impaired communication access needs 4. General Test Condition demands 7. Attention, Focus or Endurance communication access needs

Figure 5.1. Accessibility feature mapping process.

5.2 Test Administration and Accessibility Levels of Support All accessibility supports permitted during ACT Aspire testing and described in ACT Aspire Accessibility User Guide are designed to remove unnecessary barriers to student performance on the assessments. Based on the feature mapping development process described above that incorporated the targeted construct definitions, all permitted supports fully honor the content, knowledge, and skills the tests measure.

5.2.1 Understanding Levels of Accessibility Support Accessibility is universal concept that is not restricted to any one group of students. It describes needs we all have regardless of whether or not we have an official diagnostic label. The term “accessibility” also fully addresses the needs of those students with disabilities as well as those who are English Learners. The older and more familiar term “accommodations” describes only one intensive level of support that few students actually need.

Over the last decade in educational research and practice we have come to understand that all students have tools they need and use every day to engage in the classroom and communicate effectively what they have learned and can do. (See the References section later in this manual.) There are different levels of support that students may need in order to demonstrate what they know and can do on academic 5.3 Accessibility

tests. ACT Aspire assessments make several possible levels of support available. All these levels of support taken together are called accessibility supports. These accessibility supports: • allow all students to gain access to effective means of communication that in turn allow them to demonstrate what they know without providing an advantage over any other student • enable effective and appropriate engagement, interaction, and communication of student knowledge and skills • honor and measure academic content as the test developers originally intended • remove unnecessary barriers to students’ demonstrating the content, knowledge, and skills being measured on ACT Aspire assessments.

Figure 5.2. ACT Aspire Levels of Accessibility In short, accessibility supports do nothing for the student academically that he or she should be doing independently; they just make interaction and communication possible and fair for each student.

The ACT Aspire accessibility system defines four levels of support that range from minor support (default embedded system tools) to extreme support (modifications). Figure 5.2 shows the architectural structure of ACT Aspire accessibility supports.

ACT Aspire permits the use of those accessibility supports that will honor and validly preserve the skills and knowledge that our tests claim to measure, while removing needless, construct- irrelevant barriers to student performance. The four levels of support in the ACT Aspire accessibility system represent a continuum of supports,

5.4 Accessibility

from least intensive to most intensive, and assumes all users have communication needs that fall somewhere on this continuum. The unique combination of supports needed by a single test taker is called the Personal Needs Profile (PNP). A PNP tells the system which supports to provide for a specific test taker. Many students will not need a documented PNP. When a student’s communication needs are not documented in a PNP, the system treats the student as a default user whose accessibility needs are sufficiently met through the default test administration represented by the base of the pyramid—that is, without accessibility features other than the basic set already embedded for all test takers. (See support level 1, “Default Embedded System Tools” in Figure 5.1; these supports are also described in the next section.) The continuum of supports permitted in ACT Aspire results in a personalized performance opportunity for all. This systematic continuum of supports is schematically illustrated in Figure 5.2.

5.2.2 Support Level 1: Default Embedded System Tools The first level of supports is called the default embedded system tools (see Figure 5.3). They are automatically available to a default user whose accessibility needs are sufficiently met through the basic test administration experience.

Default embedded system tools meet the common, routine accessibility needs of the most typical test takers. All students are provided these tools, as appropriate, even students who have no documented PNP. Default embedded system tools include but are not limited to the following examples in online and paper tests: • paper test booklet (paper) • answer document (paper) • number 2 pencils (paper) • erasers (paper) • computer keyboard (online) • computer screen display (online) • mouse (online) • cut, copy, and paste functions in a text entry box (online) • browser zoom magnification (online) • answer eliminator (online and paper) • scratch paper (online and paper) • personal calculators for Mathematics tests (online and paper) • mark items for review (online and paper)

These tools are either embedded in the computer test delivery platform or provided at the local level automatically. They are the accessibility tools that nearly everyone uses routinely and assumes will be made available although we seldom think of them in this way. These tools serve a basic accessibility function for all.

5.5 Accessibility

As seen in Figure 5.3. Default embedded system tools are common supports made available to all users upon launch/start of test. These tools are either embedded in the basic computer test delivery platform, or they may be locally provided as needed. No advance request is needed for these supports. Students whose needs are met by default embedded tools do not need a PNP.

Figure 5.3. Default embedded system tools. 5.2.3 Support Level 2: Open Access Tools Open Access tools (see Figure 5.4) are available to all users but must be identified in advance in the PNP, planned for, and then selected from the pull-down menu inside the test to be activated (online), or else provided locally.

Many students’ unique sensory and communication accessibility needs are predictable and can be met through a set of accessibility features designed into the underlying structure and delivery format of test items. Rather than overwhelm the user with all the possible tools, Open Access tools provide just the tools needed by individual users.

Open Access tools are slightly more intensive than default embedded system tools but can be delivered in a fully standardized manner that is valid, appropriate, and personalized to the specific access needs identified within an individual student’s PNP. Some of these require the use of tool-specific administration procedures. In ACT Aspire, Open Access tools include but are not limited to the following examples: • large print (paper) • color overlay (paper) • respond in test booklet or on separate paper (paper) • line reader (online and paper) • magnifier tool (online and paper) • answer masking (online and paper) • dictate responses (online and paper) • keyboard or augmentative or assistive communication (AAC) + local print (online and paper) • breaks: supervised within each day (online and paper)

5.6 Accessibility

• special seating/grouping (online and paper) • location for movement (online and paper) • individual administration (online and paper) • home administration (online and paper) • other setting (online and paper) • audio environment (online and paper) • visual environment (online and paper) • physical/motor equipment (online and paper

Open Access tools should be chosen carefully and specifically to prevent overwhelming or distracting the student during testing. Remember: routine annual documentation of successful (and unsuccessful) use of accessibility tools through the student’s educational experience helps to inform and improve future choices.

As seen in Figure 5.4, Open Access tools may be used by anyone, but to be activated they must be identified in advance in the PNP, planned, and selected from the pull- down menu inside the test to activate them (online), or else provided locally. Room supervisors must follow required procedures. Users should be practiced, familiar, and comfortable using these types of tools as well as comfortable using them in combination with any other tools.

Figure 5.4. Open access tools. 5.2.4 Support Level 3: Accommodations Accommodations are high-level accessibility tools needed by relatively few students (see Figure 5.5). The ACT Aspire system requires accommodation-level supports to be requested by educational personnel on behalf of a student through the online PNP process. This will allow any needed resources to be assigned and documented for the student.1

Typically, students who receive this high level of support have a formally documented need for resources or equipment that requires expertise, special training, and/or extensive monitoring to select, administer, and even to use the support effectively and

1 Qualifying procedures or formal documentation required to request and receive accommodation-level support during ACT Aspire testing should be set by schools 5.7 Accessibility

securely. These can include but are not limited to the following examples: • text-to-speech English audio • text-to-speech English audio + orienting description for blind/low vision • text-to-speech Spanish audio • word-to-word dictionary • human reader, English audio • translated test directions • Braille + tactile graphics (online and paper) • sign language interpretation • abacus, locally provided (online and paper) • extra time (online and paper) • breaks: securely extend session over multiple days (paper)

Decisions about accommodation-level supports are typically made by an educational team on behalf of and including the student. Accommodation decisions are normally based on a formal, documented evaluation of specialized need. Accommodation supports require substantial additional local resources or highly specialized, expert knowledge to deliver successfully and securely.

As seen in Figure 5.5, accommodations are available to users who have been qualified by their school or district to use them. ACT Aspire recommends that students who use accommodation- level supports have a formally documented need as well as relevant knowledge and familiarity with these tools. Accommodations must be requested through the online PNP process. Any formal qualifying procedure that is required by the responsible educational authority must be completed prior to completing the PNP request process.

Figure 5.5. Accommodations

5.8 Accessibility

5.2.5 Support Level 4: Modifications Modifications are supports that are sometimes used during instruction, but they alter what the test is attempting to measure and thereby prevent meaningful access to performance of the construct being tested (see Figure 5.6). Because modifications violate the construct being tested, they invalidate performance results and communicate low expectations of student achievement. Modifications are not permitted during ACT Aspire testing. (Modifications are further discussed in the Accessibility User Guide in the section titled, “When Instruction and Assessment Supports Differ.”)

As seen in Figure 5.6 modifications are supports that alter what the test is attempting to measure and therefore are not permitted in ACT Aspire tests.

Figure 5.6. Modifications

5.3 Accommodations, Open Access and Embedded Tools In our commitment to provide a level field and fair performance opportunities for all students, ACT Aspire provides an integrated system of accessibility supports that include accommodations as well as other forms (less intensive levels) of accessibility support. There are times when supports provided for those who test using the online format are combined with other types of locally provided or paper-format supports. The reverse is also true, as students using the paper format sometimes also take advantage of certain online options. Regardless of test format, all students who use accessibility features must also have this use documented in the online Personal Needs Profile (PNP) portal by appropriate school personnel. For this reason we have provided the general description of ACT Aspire Accessibility Supports here in one section. Full procedural requirements and instructions for using permitted supports during test administration are provided in the ACT Aspire Accessibility User Guide. In addition the following procedural appendices and test administration resources are provided with the ACT Aspire Accessibility User Guide: • Appendix A: Personal Needs Profile (PNP) Worksheet • Appendix B: General Response Dictation and Scribing Procedures • Appendix C: Guidelines for Sign Language Interpretation • Appendix D: Approved Bilingual Word-to-Word Dictionaries • Appendix E: Procedures for Local Delivery of Read-Aloud Support 5.9 Accessibility

The following eight tables provide the full list of supports permitted during the ACT Aspire Summative Tests. Tables 5.1-5.4 on the following pages identify the accessibility supports available in the paper summative test format. Tables 5.5-5.8 identify the accessibility supports available in the online summative test format.

5.4 2016 Paper Summative Testing Table 5.1. Paper Summative Testing Presentation Supports

Content Area

Presentation Supports Support Level Reading English Writing Math Science

Human Reader (English Audio) • Intended for user with ability to see graphics. Directions Directions • Requires: Locally provided; follow procedure Accommodation* Ye s Ye s Ye s in appendix E Only Only • Recommended: Extra time 300%—must separately select.

Human Reader (English Audio + Orienting Description) • Intended for user with blindness or low vision. Directions Directions • Requires: Locally provided; follow procedure Only Only Ye s Ye s Ye s in appendix E. Must separately select and use Braille + Tactile Graphics companion. Accommodation* (then must (then must (with Braille (with Braille (with Braille use Braille use Braille + Tactile + Tactile + Tactile • Allow time for shipping of braille materials. + Tactile + Tactile Graphics) Graphics) Graphics) Student will also need response support to Graphics) Graphics) record responses in paper form. • Recommended: Extra time 300%—must separately select.

Translated Directions • Allowed for all grades. Accommodation* Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided.

Word-to-Word Dictionary, ACT-Approved • Requires: Locally provided; follow procedure Accommodation* — — Ye s Ye s Ye s in appendix D.

American Sign Language (ASL): Directions Only (English Text) Accommodation* Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided; follow procedure in appendix C.

American Sign Language (ASL): Test Items (English Text) • Requires: Locally provided 1:1 administration; Accommodation* — — Ye s Ye s Ye s follow procedure in appendix C. • Recommended: Extra time.

Signed Exact English (SEE): Directions Only (English Text) Accommodation* Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided; follow procedure in appendix C.

5.10* Qualification for use of permitted accessibility supports must follow policies of your local educational authority. Accessibility

Table 5.1. (continued)

Content Area

Presentation Supports Support Level Reading English Writing Math Science

Signed Exact English (SEE): Test Items (English Text) • Requires: Locally provided 1:1 administration; Accommodation* — — Ye s Ye s Ye s follow procedure in appendix C. • Recommended: Extra time.

Cued Speech • Requires: Locally provided; follow procedure Accommodation* — — Ye s Ye s Ye s in appendix E.

Braille, Contracted, American Edition (EBAE) Includes Tactile Graphics • Requires: Response support to record Accommodation* Ye s Ye s Ye s Ye s Ye s responses; time for shipment of materials. • Recommended: Extra time.

Braille, Uncontracted, American Edition (EBAE) Includes Tactile Graphics • Requires: Response support to record Accommodation* Ye s Ye s Ye s Ye s Ye s responses; time for shipment of materials. • Recommended: Extra time.

Braille, Contracted, Unified English (UEB) Includes Tactile Graphics • Requires: Response support to record Accommodation* Ye s Ye s Ye s Ye s Ye s responses; time for shipment of materials. • Recommended: Extra time.

Braille, Uncontracted, Unified English (UEB) Includes Tactile Graphics • Requires: Response support to record Accommodation* Ye s Ye s Ye s Ye s Ye s responses; time for shipment of materials. • Recommended: Extra time.

Large Print Open Access Ye s Ye s Ye s Ye s Ye s • Requires: Time for shipment of materials.

Magnifier Tool Open Access Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided.

Line Reader Open Access Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided.

Color Overlay Open Access Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided.

* Qualification for use of permitted accessibility supports must follow policies of your local educational authority.

5.11 Accessibility

Table 5.2. Paper Summative Testing Interaction and Navigation Supports

Content Area Interaction and Navigation Supports Support Level Reading English Writing Math Science Abacus Accommodation* — — — Ye s — • Requires: Locally provided. Answer Masking Open Access Ye s Ye s — Ye s Ye s • Requires: Locally provided. Answer Eliminator Embedded Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided; used in test booklet only. Highlighter Embedded Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided; used in test booklet only. Scratch Paper Embedded Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided. Calculator (Grades 6–EHS) • Requires: Locally provided. Embedded — — — Yes — • Follow ACT Aspire Calculator Policy; may use accessible calculators. * Qualification for use of permitted accessibility supports must follow policies of your local educational authority.

Table 5.3. Paper Summative Testing Response Supports

Content Area Response Supports Support Level Reading English Writing Math Science Electronic Spell Checker

• Requires: Locally provided separate device which Accommodation* — — Ye s Ye s Ye s must meet specifications provided in Procedures for Administration in Guide. Respond in Test Booklet or on Separate Paper • Requires: Response transcription; original work must Open Access Ye s Ye s Ye s Ye s Ye s be returned. • Recommended: Extra time. Dictate Responses • Requires: Follow procedure in appendix B. Open Access Ye s Ye s Ye s Ye s Ye s • Recommended: Extra time. Keyboard or AAC + Local Print • Requires: Response transcription; original work must Open Access Ye s Ye s Ye s Ye s Ye s be returned. • Recommended: Extra time. Mark Item for Review • Requires: Student mark, once made, must be erased Embedded Ye s Ye s Ye s Ye s Ye s thoroughly. * Qualification for use of permitted accessibility supports must follow policies of your local educational authority. 5.12 Accessibility

Table 5.4. Paper Summative Testing General Test Condition Supports

Content Area

General Test Condition Supports Support Level Reading English Writing Math Science

Extra Time† Accommodation* Ye s Ye s Ye s Ye s Ye s

Breaks: Securely Extend Session over Accommodation* Ye s Ye s Ye s Ye s Ye s Multiple Days

Breaks: Supervised within Each Day Open Access Ye s Ye s Ye s Ye s Ye s

Special Seating/Grouping Open Access Ye s Ye s Ye s Ye s Ye s

Location for Movement Open Access Ye s Ye s Ye s Ye s Ye s

Individual Administration Open Access Ye s Ye s Ye s Ye s Ye s

Home Administration Open Access Ye s Ye s Ye s Ye s Ye s

Other Setting Open Access Ye s Ye s Ye s Ye s Ye s

Audio Environment Open Access Ye s Ye s Ye s Ye s Ye s

Visual Environment Open Access Ye s Ye s Ye s Ye s Ye s

Physical/Motor Equipment Open Access Ye s Ye s Ye s Ye s Ye s

* Qualification for use of permitted accessibility supports must follow policies of your local educational authority.

† Extra time represents the maximum allowed within a same-day test. Session may end earlier if time not needed. Automatic assignment of extra time occurs ONLY with TTS audio formats. If using only paper form braille without TTS, extra time must be manually selected in PNP.

5.5 2016 Online Summative Testing Table 5.5. Online Summative Testing Presentation Supports

Content Area

Presentation Supports Support Level Reading English Writing Math Science

Text-to-Speech (English Audio) • Intended for user with ability to see graphics. Directions Directions Accommodation* Ye s Ye s Ye s Only Only • Requires: PNP system automatically assigns extra time 300%.†

Text-to-Speech (English Audio + Orienting Description) • Intended for user with blindness or low Directions Directions vision. Only Only Ye s Ye s Ye s • Requires: Braille + Tactile Graphics (then must (then must (with Braille (with Braille (with Braille Companion; response support to record Accommodation* use Braille use Braille + Tactile + Tactile + Tactile responses; time for shipment of braille + Tactile + Tactile Graphics) Graphics) Graphics) materials; PNP system automatically assigns Graphics) Graphics) extra time 300%.† • PNP system automatically prompts choice of Braille, Contracted or Braille, Uncontracted.

5.13 Accessibility

Table 5.5. (continued)

Content Area

Presentation Supports Support Level Reading English Writing Math Science

Translated Test Directions • Allowed for all grades.

• Requires: Must be provided before test Accommodation* Ye s Ye s Ye s Ye s Ye s launch. • Spanish provided in online system; other languages must be locally provided.

Text-to-Speech (Spanish Audio) Item Translation • Grades 3–6 only. Accommodation* — — Ye s Ye s Ye s • Requires: Online prerecorded format; PNP system automatically assigns extra time 300%.†

Word-to-Word Dictionary, ACT-Approved • Requires: Locally provided; follow procedure Accommodation* — — Ye s Ye s Ye s in appendix D.

Cued Speech • Requires: Locally provided, follow procedure Accommodation* — — Ye s Ye s Ye s in appendix E.

Braille Contracted, Unified English (UEB) Includes Tactile Graphics • Requires: Response support to record Accommodation* Ye s Ye s Ye s Ye s Ye s responses; time for shipment of materials. • Recommended: Extra time.

Magnifier Tool Open Access Ye s Ye s Ye s Ye s Ye s • Online platform tool; may be locally provided. 5.14 Accessibility

Table 5.5. (continued)

Content Area

Presentation Supports Support Level Reading English Writing Math Science

Line Reader Open Access Ye s Ye s Ye s Ye s Ye s • Online platform tool; may be locally provided.

Color Contrast • Online platform tool or locally provided color Open Access Ye s Ye s Ye s Ye s Ye s overlay.

Browser Zoom Magnification Embedded Ye s Ye s Ye s Ye s Ye s • Online only.

* Qualification for use of permitted accessibility supports must follow policies of your local educational authority.

Table 5.6. Online Summative Testing Interaction and Navigation Supports

Content Area

Interaction and Navigation Supports Support Level Reading English Writing Math Science

Abacus Accommodation* — — — Ye s — • Requires: Locally provided.

Answer Masking Open Access Ye s Ye s — Ye s Ye s • Online platform tool.

Answer Eliminator Embedded Ye s Ye s Ye s Ye s Ye s • Online platform tool.

Highlighter Tool Embedded Ye s Ye s Ye s Ye s Ye s • Online platform tool.

Browser Cut, Copy, and Paste Embedded Ye s Ye s Ye s Ye s Ye s • Online only.

Scratch Paper Embedded Ye s Ye s Ye s Ye s Ye s • Requires: Locally provided.

Calculator (Grades 6–EHS) • Requires: Locally provided. Embedded — — — Yes — • Follow ACT Aspire Calculator Policy; may use accessible calculators.

* Qualification for use of permitted accessibility supports must follow policies of your local educational authority.

5.15 Accessibility

Table 5.7. Online Summative Testing Response Supports

Content Area

Response Supports Support Level Reading English Writing Math Science

Electronic Spell Checker

• Requires: Locally provided separate device Accommodation* — — Ye s Ye s Ye s which must meet specifications provided in Procedures for Administration in Guide.

Respond on Separate Paper • Requires: Locally provided; response Open Access Ye s Ye s Ye s Ye s Ye s transcription; original work must be returned. • Recommended: Extra time.

Dictate Responses • Requires: Follow procedure in appendix B. Open Access Ye s Ye s Ye s Ye s Ye s • Recommended: Extra time.

Keyboard or AAC + Local Print • Requires: Response transcription; original Open Access Ye s Ye s Ye s Ye s Ye s work must be returned. • Recommended: Extra time. Mark Item for Review Embedded Ye s Ye s Ye s Ye s Ye s • Online platform. * Qualification for use of permitted accessibility supports must follow policies of your local educational authority.

Table 5.8. Online Summative Testing General Test Condition Supports

Content Area

General Test Condition Supports Support Level Reading English Writing Math Science

Extra Time† Accommodation* Ye s Ye s Ye s Ye s Ye s

Breaks: Supervised within Each Day Open Access Ye s Ye s Ye s Ye s Ye s

Special Seating/Grouping Open Access Ye s Ye s Ye s Ye s Ye s

Location for Movement Open Access Ye s Ye s Ye s Ye s Ye s

Individual Administration Open Access Ye s Ye s Ye s Ye s Ye s

Home Administration Open Access Ye s Ye s Ye s Ye s Ye s

Other Setting Open Access Ye s Ye s Ye s Ye s Ye s

Audio Environment Open Access Ye s Ye s Ye s Ye s Ye s

Visual Environment Open Access Ye s Ye s Ye s Ye s Ye s Physical/Motor Equipment Open Access Ye s Ye s Ye s Ye s Ye s * Qualification for use of permitted accessibility supports must follow policies of your local educational authority.

5.16 Accessibility

Table 5.9. Frequency of Use of Accommodated Forms, ACT Aspire, Spring 2015 Subject/ Braille Braille English Audio English Audio Grade Contracted Uncontracted for Blind for Sighted Large Print Spanish Grand Total English 80 15 455 550 3 15 5 79 99 4 11 3 58 72 5 13 2 68 83 6 7 1 50 58 7 9 60 69 8 7 2 60 69 EHS 18 2 80 100 Mathematics 104 9 10 23,291 590 33 24,037 3 17 3 1 2,582 103 7 2,713 4 17 1 2,692 83 9 2,802 5 17 2,814 94 13 2,938 6 16 1 2,592 66 4 2,679 7 10 1 2,417 78 2,506 8 7 1 3 2,309 79 2,399 EHS 20 3 5 7,885 87 8,000 Reading 117 14 606 737 3 18 4 110 132 4 18 2 86 106 5 19 1 97 117 6 11 3 67 81 7 14 81 95 8 14 3 81 98 EHS 23 1 84 108

5.17 Accessibility

Table 5.9. (continued) English English Subject/ Braille Braille Audio for Audio for Grade Contracted Uncontracted Blind Sighted Large Print Spanish Grand Total

Science 50 7 9 18,046 337 16 18,465 3 2 2 1 1,331 52 1,388 4 3 1 1,494 32 2 1,532 5 11 1 2,656 56 12 2,736 6 6 1 1,266 28 2 1,303 7 5 1 2,129 51 2,186 8 4 2 2 1,412 38 1,458 EHS 19 5 7,758 80 7,862 Writing 70 23 7 13,600 452 6 14,158 3 13 5 1 912 85 1,016 4 9 4 1,043 57 3 1,116 5 8 3 941 70 1 1,023 6 8 3 1,055 52 2 1,120 7 6 3 1 1,039 57 1,106 8 6 3 2 1,017 58 1,086 EHS 20 2 3 7,593 73 7,691 Grand Total 421 68 26 54,937 2,440 55 57,947 % of Total 0.73% 0.12% 0.04% 94.81% 4.21% 0.09% 100.00%

5.18 CHAPTER 6 Test Administration

The ACT Aspire Test Coordinator and Room Supervisor Manuals contain the instructions for administering the ACT Aspire grades 3–Early High School English, Mathematics, Reading, Science, and Writing subject tests. The test coordinator is the main ACT Aspire contact at a school and the person who makes arrangements for the test administration. The room supervisor is responsible for secure administration of the tests to the students in a testing room. There are separate Room Supervisor Manuals for paper testing formats and for computer based testing formats.

In addition to the Test Coordinator and Room Supervisor Manuals, all training and test administration resources are available online on the websites associated with ACT Aspire: actaspire.tms.pearson.com houses training videos, actaspire.avocet.pearson.com includes links to other training materials, and actaspire.pearson.com has access to many more resources.

6.1 Policies and Procedures The Test Coordinator and Room Supervisor Manuals provide direction in administering ACT Aspire tests, including specific instructions for conducting the timed tests. It is important to follow these instructions to successfully measure students’ academic skills. All testing personnel, including room supervisors and other testing staff, are required to read the materials provided by ACT Aspire.

6.2 Standardized Procedures Throughout the Test Coordinator and Room Supervisor Manuals, there are detailed directions for securing test materials and administering tests in a standardized manner.

It is preferred that relatives or guardians of students taking ACT Aspire should not serve in the role of test coordinator. Whenever possible, this policy should be followed. 6.1 Test Administration

However, in some circumstances, this may not be possible. In those circumstances, the district/school should monitor the testing process so that test coordinators who have a relative testing do not handle the test materials of the relative without another responsible individual present.

Relatives of students taking ACT Aspire should not serve in the role of room supervisor in the same testing room as the student relative. It is permissible for a relative to serve as a room supervisor in the same school/district as a related student, provided that student is not testing in the room being supervised by the related testing staff person.

The following actions violate ACT Aspire policies and procedures: • accessing or obtaining a test booklet or test questions prior to the test for any reason (An exception is provided for American Sign Language and Signing Exact English interpreters assisting students. See “Preparation for Sign Interpretation” in appendix C of the ACT Aspire Accessibility User’s Guide.) • photocopying, making an electronic copy, or keeping a personal copy of the test or of any test items (An exception is provided for students who need to use digital scanning magnification for test items. See the ACT Aspire Accessibility User’s Guide.) • taking notes about test questions or any paraphrase of test questions to aid in preparing students for testing • aiding or assisting a student with a response or answer to a secure test item, including providing formulas • rephrasing test questions for students • creating an answer key or “crib sheet” of answers to test questions • editing or changing student answers after completion of the test, with or without the student’s permission • allowing students to test in an unsupervised setting • leaving test materials in an unsecured place or unattended • failing to properly report and document incidents of prohibited behavior involving students, staff, or others • allowing students to test longer than the permitted time • failing to return and account for all testing materials after the testing session has ended

6.3 Selecting and Training Testing Staff The test coordinator is responsible for selecting and training all room supervisors and other testing staff. The test coordinator provides the continuity and administrative uniformity necessary to ensure that students at a school are tested under the same conditions as students at every other school, and to ensure the security of the test. 6.2 Test Administration

6.3.1 Room Supervisors Typically, teachers will administer the tests to students during regular class periods. Depending on the number of students in a class, you may wish to assign adult school staff (not students) to assist the teacher with distributing and collecting test materials and with monitoring testing.

Be sure that everyone involved in test administration has access to the Room Supervisor Manual and is familiar with its contents. All manuals are periodically updated, so it is important to check the Avocet website for updated versions before each new test administration. A room supervisor is needed in each testing room to read directions and monitor students. If test rooms have more than 25 students, additional personnel may be assigned to assist the room supervisor.

Before the test day, all testing personnel should read all of the testing instructions carefully, particularly the verbal instructions, which will be read aloud to students on the test day. It is important that testing personnel are familiar with these instructions.

6.3.1.1 Room Supervisor Qualifications The ACT Aspire test coordinator should confirm that the room supervisor(s) meet all of the following criteria. Each room supervisor should be • proficient in English • experienced in testing and classroom management • a staff member of the institution or district where the test administration will take place To protect both students and the room supervisor from questions of possible conflict of interest, the following conditions should also be met. The room supervisor should • not be a relative or guardian of a student in the assigned room • not be a private consultant or individual tutor whose fees are paid by a student or student’s family

6.3.1.2 Room Supervisor Responsibilities Specific responsibilities are: • Read and thoroughly understand the policies, procedures, and instructions in the Room Supervisor Manual and other materials provided. • Supervise a test room. • Distribute test materials if administering paper tests.* • Start a test session if administering online tests. • Help students sign in to the online testing system if administering online tests.* • Read test instructions. • Properly time tests. • Walk around the testing room during testing to be sure students are working on the correct test and to observe student behavior.* 6.3 Test Administration

• Monitor the online testing system as needed. • Pay careful attention to monitoring students’ behavior during the entire testing session.* • Collect and account for all answer documents, test booklets, and scratch paper (or, for online testing, authorization tickets) before dismissing students.* • Ensure students have stopped testing and have correctly signed out of the online testing system.* • Complete documentation of any testing irregularities. *Other school staff may assist with these activities.

6.3.2 Responsibilities of Other Testing Staff Other school staff can assist the room supervisor with an administration to a group. Another staff member is recommended if a room has 26 or more students. Other staff members can be added for each incremental increase of 25 students in the case of testing in a large group setting.

The staff member can assist in the administration of the tests according to the policies and procedures in the appropriate Room Supervisor Manual. If used, the staff members should meet the same qualifications as a room supervisor (see above).

The testing staff member’s responsibilities can be: • Read and understand the appropriate Room Supervisor Manual • Help distribute test materials if administering paper tests. • Help students sign in to the online testing system if administering online tests. • Verify the timing of the test with the room supervisor. • Walk around the room during testing to be sure all students are working on the correct test and to observe student behavior. • Report any irregularities to the room supervisor immediately. • Accompany students to the restroom if more than one leaves during the timed portion of the test. • Pay strict attention to monitoring students during the entire testing session. • Help collect and account for all answer documents, test booklets, and scratch paper (or, for online testing, authorization tickets). • Ensure students have stopped testing and have correctly signed out of the online testing system.

6.4 Test Administration

6.3.3 Staff Training Sessions To ensure that the administration of the tests is standard, it is important for all testing staff to understand the testing procedures outlined in the Room Supervisor Manual and other training resources.

ACT Aspire recommends that a training session be conducted prior to testing for all testing staff to discuss the testing guidelines and organizational details, including: 1. Security of Test Materials (paper testing only) a. Describe how the materials will be distributed to the test rooms and how room supervisors are to count them. b. Emphasize that room supervisors are to count test booklets when they receive them from the test coordinator and again before students are dismissed. c. Emphasize that staff members should never leave a test room unattended. 2. Security and Materials for Online Testing a. Ensure that all room supervisors have access to the ACT Aspire Portal. b. Explain that authorization tickets should be printed in advance and stored in a secure location. c. Describe how to use the online testing system—see the ACT Aspire Portal User Guide posted on the Avocet website for step-by-step instructions. d. Emphasize that room supervisors must collect used and unused scratch paper and authorization tickets after testing. e. Emphasize that staff members should never leave a test room unattended. f. Emphasize that test sessions must be started in the ACT Aspire Portal before students can sign in to the test. 3. Activities Before the Test a. Determine which set of verbal instructions room supervisors are to follow. Room supervisors should clearly mark those instructions in their manuals. b. Develop and share with staff a contingency plan for handling unexpected situations that may arise that could affect testing. See Avocet for the ACT Aspire Contingency Plan document. 4. Test Day a. Discuss when and where staff members are to report on the test day. b. Stress that no one may be admitted to the testing room once the timed tests have begun. Determine how to handle late arrivals. c. Stress that verbal instructions for the tests must be read verbatim. d. Stress that answer documents, test booklets, and login credentials should not be distributed prior to admitting students.

6.5 Test Administration

e. Stress that accurate timing of each test is critical. For paper testing, room supervisors must record the start, five-minute warning, and stop times in the manuals. Discuss the consequences of a mistimed section. f. Emphasize that staff members must not read (other than the Room Supervisor Manual), correct papers, or do anything unrelated to administering the test. Their attention should be focused on the students. g. Emphasize that conversations among staff must be quiet and kept to a minimum. Even whispered conversations can be distracting to students while testing. h. For room supervisors who are administering the grades 6–Early High School (grades 9–10) mathematics test, emphasize that calculators should be checked before testing to ensure they meet ACT Aspire standards. Review permitted and prohibited calculator listed on the Avocet website (Avocet/Calculators/Calculator FAQ). i. Note that during the test, staff members should walk quietly around the room, be available to respond to students’ questions, assist in the case of illness, and check that students are working on the correct test. In rooms where students are taking online tests, staff should ensure students have signed in to the correct test and assist them with technical or system navigation issues. j. Discuss procedures for a student leaving during the test to go to the restroom. For online testing, review “Test Session Interruptions” on the Avocet website for instructions on how to pause a test for a student. k. Discuss what actions to take if staff members observe prohibited behavior. Review plans for dismissing students (e.g., where they are to be sent, how to maintain vigilance in the test room, documenting actions taken). l. Discuss what actions to take in the case of a group irregularity (e.g., a power outage) or an emergency. m. Discuss potential individual irregularities and actions to take. n. Review the Testing Irregularity Report. 5. After the test a. Emphasize that room supervisors administering paper tests must verify the count of used and unused test booklets, then return test materials, scratch paper, and Irregularity Reports to the test coordinator. b. Emphasize that room supervisors administering paper tests should verify that all answer documents have the correct test form gridded. c. Emphasize that room supervisors administering online tests must collect all used and unused scratch paper and authorization tickets and return them to the test coordinator. d. Emphasize that all test sessions must be closed in the ACT Aspire Portal after all students assigned to the test session have completed testing. Once test sessions are closed, they cannot be reopened.

6.6 CHAPTER 7 Test Security

In order to ensure the validity of the ACT Aspire test scores, test takers, individuals that have a role in administering the tests, and those who are otherwise involved in facilitating the testing process, must strictly observe ACT Aspire’s standardized testing policies, including the Test Security Principles and test security requirements. Those requirements are set forth in the instructions for the ACT Aspire test and the ACT Aspire Test Supervisor’s Manual and may be supplemented by ACT from time to time with additional communications to test takers and testing staff.

ACT Aspire’s test security requirements are designed to ensure that examinees have an equal opportunity to demonstrate their academic achievement and skills, that examinees who do their own work are not unfairly disadvantaged by examinees who do not, and that scores reported for each examinee are valid. Strict observation of the test security requirements is required to safeguard the validity of the results.

Testing staff must protect the confidentiality of the ACT Aspire test items and responses. Testing staff should be aware of and competent for their roles, including understanding ACT Aspire's test administration policies and procedures and acknowledging and avoiding of conflicts of interest in their roles as test administrators for ACT Aspire.

Testing staff must be alert to activities that can compromise the fairness of the test and the validity of the scores. Such activities include, but are not limited to, cheating and questionable test taking behavior (such as copying answers or using prohibited electronic devices during testing); accessing questions prior to the test; taking photos or making copies of test questions or test materials; posting test questions on the internet; or test proctor or test administrator misconduct (such as providing answers or questions to test takers or permitting test takers to engage in prohibited conduct during testing).

7.1 Data Security ACT systems hardening baselines are built from the Center for Internet Security (CIS)

7. 1 Test Security

(see http://www.cisecurity.org) standards and modified to meet the risk profile of applications and systems that they support, per the CIS guidance.

7.1.1 Personally Identifiable Information ACT Aspire, LLC’s, general policy on the release of personally identifiable information (PII) that is collected as part of the ACT Aspire program is that we will not provide such information to a third party without the consent of the individual (or, in the case of an individual under age 18, the consent of a parent or legal guardian). However, ACT Aspire may share PII as follows, as allowed by applicable laws: • ACT Aspire will share PII with its member companies, NCS Pearson, Inc., and ACT, Inc. Both member companies provide services to ACT Aspire that are essential for ACT Aspire to provide testing services, and both members will use PII only in the same ways ACT Aspire can, in accordance with this policy. In the event of dissolution of ACT Aspire, LLC, its members will assume ownership of its assets, including routine business records that may include PII. • ACT Aspire may provide PII to CDE once it has been determined that the state will pay for the individual to take the assessment. This PII may include linking information relating to ACT Aspire assessments with information relating to ACT programs such as the ACT assessment. • If the individual is under 18 years of age, ACT Aspire may provide PII to the individual’s parent or guardian, who may direct us to release the information to third parties. If the individual is 18 years of age or older, ACT Aspire may provide PII to the individual, who may direct us to release the information to third parties. • ACT Aspire may enter into contracts with third parties to perform functions on its behalf, which may include assisting it in processing PII. These parties are contractually required to maintain the confidentiality of the information and are restricted from using the information for any purposes other than performance of those functions. • ACT Aspire may disclose PII to facilitate the testing process, for example, to test proctors. • ACT Aspire may provide PII as required or allowed by operation of law, or to protect the health and safety of its customers or others.

ACT Aspire may authorize its members and other subcontractors to use de-identified and aggregate data for other purposes, such as research. PII will not be included in any de-identified or aggregate data sets.

7.1.2 Security Features of Data Systems ACT employs the Principle of Least Privilege when providing access to users, and access entitlement reviews are conducted periodically. All student Information is secured and password-controlled. Access to student information during various testing processes is limited to personnel who need that access in order to fulfill the

7. 2 Test Security

requirements of the contract. Access to confidential student information is reviewed on a periodic basis. ACT is in the process of modernizing our infrastructure to measure up to established policy and that project will be ongoing. ACT takes a risk- based approach in prioritizing the order in which systems are to be modernized. ACT does not permit any customer access to data systems in its environment as this would cause exposure for other ACT customers.

The overall approach to security and cyber security standards by Pearson, ACT Aspire’s subcontractor, includes information about how the systems prevent infiltration and protect student information. Pearson’s overall approach involves the following facets: Security and Audit Management. Pearson undergoes regular audits from internal corporate audit function as well as third-party audits. Identified gaps are reported to the Pearson Chief Information Security Officer and responsible Busi- ness Information Security Officers and the specific business function management, where a remediation plan is identified, responsibilities are assigned, mile- stones are created, and follow-up documentation is stored. Summary reports from the independent audits can be made available to the OIT Office of Cyber Security upon request. Update Processes. The “change management” process manages changes to optimize risk mitigation, minimizes the severity of any known impact, and aims to be successful on the first attempt. This includes additions, modifications, and remov- als of authorized components of services and their associated documentation. The process includes acknowledging and recording changes; assessing the impact, cost, benefit, and risk of proposed changes; obtaining approval(s); managing and coordinating implementation of changes; and reviewing and closing requests for change. Critical steps in the change process require management approval before proceeding to the next step of the process. System Behavior Monitoring. Both during test-taking and at other times, Pear- son monitors for anomalous activity throughout the entire system, not just at the application layer. Should student testing activity trigger a behavior alert, the test is halted and the proctor is alerted. System monitors may be triggered, which will generate an alert to the appropriate team for further investigation. Features That Prevent Infiltration.Several methods mitigate the risk of uninvited access to systems. All external traffic is encrypted. All item content payloads are encrypted end-to-end using AES-128. All student responses are encrypted using AES-256, using a different key than the content payloads. Students receive a one-time password to take a test. Account and password controls meet accepted standards for length, complexity, lockout, re-use, and expiration.

Systems and components use the following appropriate protocols and design features to neutralize known threats: Component-to-Component. All sensitive communication between application components that crosses the Internet is encrypted using Secure Sockets Layer 7. 3 Test Security

(SSL)/Transport Layer Security (TLS). The strongest version of TLS is provided to the user’s browser depending on that browser’s capabilities. This does not apply to assessment item content delivery, because these are encrypted as described below. A different method will enable proctor caching, which is required to reduce their overall bandwidth requirements. User Authentication and Authorization. Users are only created when there is a business need and are granted a user role in the system based on job requirements. A user requests access with specific privileges, a role, that are needed to perform their required activities. User accounts are only authorized to access the data specific to their role and organization. The request for access is reviewed by the appropriate program manager and either approved or denied. Users are removed as part of an overall Termination Process in accordance with Human Resources policy. Network access is removed immediately, and accountable managers are notified of specific applications from which the users must have access removed. All applications, devices, databases, and network access require authentication of the user. Assessment Item-Level Security. When a test is marked “secure” during publishing, each individual item is encrypted using Advanced Encryption Standard (AES) and a 128-bit key. Test items remain encrypted until they are downloaded to the TestNav client, decrypted, and presented to the student during testing. Student Response and Score Data Security. Student responses are encrypted using an AES 256-bit key. Score data are communicated between components using SSL/TLS. Data Storage. Database backups are AES encrypted using Amazon’s S3 Server Side Encryption functionality

Internal corporate security policies are aligned with ISO 27001 and are updated regularly to reflect changes to the industry landscape. Application security practices are based on a variety of accepted good practices, including ISO 27001, Open Web Application Security Project, Web Application Security Consortium, and internally developed standards. Pearson security analysts update the standards and practices on a regular basis.

Specific security protocols for device types, operating systems, browsers, plugins and settings, enterprise management, and virtual desktop practices vary over time as new software versions are released by vendors, such as a new version of Windows or Java, and as new devices come on the market or are rendered obsolete because of age. These protocols are communicated regularly and clearly, early enough for schools and technology managers to confirm they are prepared well in advance of test day. In general, protocols include locking down web browsers or devices so that nothing but the test may be accessed during testing and verifying that supported versions of software include the latest security releases from the original vendor.

7. 4 Test Security

7.1.3 Data Breaches and Remedies ACT’s Information Security Policy governs the handling of student data that is classified as confidential restricted. The policy states that confidential restricted information must meet the following guidelines: • Electronic information assets must only be stored on ACT-approved systems/media with appropriate access controls. • Only limited authorized users may have access to this information. • Physical records must be locked in drawers or cabinets while not being used. • ACT also has a Clear Desk and Clean Screen Standard, Access Management Standard, Malware Protection Standard, Network Security Management Standard, Secure System Configuration Standard, Security Event Logging and Monitoring Standard, and System Vulnerability and Patch Management Standard to form a system of control to protect student data.

ACT has an Information Security Incident Response Plan (ISIRP) that brings needed resources together in an organized manner to deal with an incident, classified as an adverse event, related to the safety and security of ACT networks, computer systems, and data resources. The adverse event could come in a variety of forms: technical attacks (e.g., denial of service attack, malicious code attack, exploitation of a vulnerability), unauthorized behavior (e.g., unauthorized access to ACT systems, inappropriate usage of ACT data, loss of physical assets containing Confidential or Confidential Restricted data), or a combination of activities. The purpose of the plan is to outline specific steps to take in the event of any information security incident.

This Information Security Incident Response Plan charters an ACT Security Incident Response Team (ISIRT) with providing an around-the-clock (i.e., 24x7) coordinated security incident response throughout ACT. The Information Security Officer is the overall ACT representative, and has the responsibility and authority to manage the Information Security Incident Response Team and implement necessary ISIRP actions and decisions during an incident.

The ACT Aspire program, as part of the National Services group of NCS Pearson, Inc., (Pearson) responds to circumstances that relate to a security or privacy issues involving loss or access to student data processed by or on behalf of ACT Aspire.

Pearson has a formally documented Incident Response Policy and Process so that necessary stakeholders are notified and engaged in the process to investigate, remediate, and follow-up on any suspected or actual breach. If a breach is suspected, Pearson staff will communicate with our Chief Information Security Officer and Security team to initiate an investigation. If a breach is confirmed during the initial investigation, we execute the Incident Response Plan. Key stakeholders will be notified in the security, technical, business, and legal groups. Steps are promptly taken to restrict access to the compromised host or application while maintaining the integrity of the environment, to preserve any potential evidence for the investigation. 7. 5 Test Security

Pearson analysts will perform an investigation using a combination of tools and manual analysis in an attempt to identify the root cause and the extent of the compromise.

At the same time, the business and legal teams will determine what external notification requirements Pearson is subject to, based on legislation and contractual obligations. Required notifications and remedies will be performed.

If/when the root cause is identified, a remediation plan will be created to address the vulnerability, not only on the host(s) or application(s) in question, but across systems that may face the same vulnerability even if they have not yet been compromised. Systems or applications are patched promptly and must pass a security assessment prior to being released for general use. The investigator will submit a report with the details of the incident, and a post-mortem meeting will be held to review that report and discuss any lessons learned from the incident.

7.1.4 Data Privacy and Use of Student Data 7.1.4.1. Data Privacy Assurances ACT and our subcontractors agree not to use student PII in any way or for any purpose other than those expressly granted by the contract.

ACT does not employ the technology to encrypt at rest our storage frames, however, we have the following compensating solutions in this space:

• Storage frames supporting the ACT are dedicated to ACT for our in house system • Storage frames support the ACT are housed at secure data centers which have SOC1 controls and provide reports to ACT • E-Business does encrypt PII data within the database structures • ACT does not use any external tape solutions; all backups are replicated within the solution to a secondary solution at our non-prod data center for disaster recovery purposes • Office 365 (email, SharePoint) is encrypted as a cloud SaaS solution

7.1.4.2 Security Processes and Procedures ACT conducts new hire training in Information Security practices and the handling of sensitive data. Annual refreshers are then conducted on an ongoing basis. ACT employs the Principle of Least Privilege so that only those individuals that have a legitimate need to access the data will be able to access the data and employs role- based security so that users only have accesses to the pieces of data necessary to conduct their work on behalf of our customers. Accounts are terminated immediately when a user departs the organization and periodic entitlement reviews are conducted to ensure that those that have access still require that level of access to perform their duties.

7. 6 Test Security

To manage access to identifiable student data for ACT Aspire, Pearson authorizes access to networks, systems, and applications based on business need to know, or the least privilege necessary to perform stated job duties. Any elevated rights not granted by virtue of job responsibilities at time of hire must be approved by management, and in certain instances, the Information Security Group (such as administrative rights).

Regarding de-identifying data during scoring, the digital scoring platform routes student responses to scorers randomly. Scorers cannot see any student information such as name or other demographics, unless a student self-identifies in his or her response.

The security processes include the following aspects: Security Training of Staff. A Security Awareness Program is an ongoing effort providing guidance to every employee so they understand security policies, individual responsibilities for compliance, and how behaviors affect the ability to protect systems and data. The core of these efforts is built on the well-known principles of Confidentiality, Integrity, and Availability (CIA). Security awareness begins immediately with orientation for new hires. Training covers acceptable use of systems and fundamental best practices. Web-based training modules are available, and all employees are required to complete the course when hired and thereafter are routinely asked to refresh their knowledge. Specific online courses covering Payment Card Industry (PCI) compliance and PII have also been developed. An annual re- fresher course including components of all of these control areas is also provided. Access to Identifiable Student Data.Pearson authorizes access to networks, systems, and applications based on business need to know, or the least privilege necessary to perform stated job duties. Any elevated rights not granted by virtue of job responsibilities at time of hire must be approved by management, and in certain instances, the Information Security Group (such as administrative rights). Internal Network Access Controls. Policy requires system access to be granted with a unique user ID. Every employee is provided with a unique domain account and a secure one-time password. To safeguard confidentiality, passwords must be changed upon first logon. Pearson stipulates a minimum password length as well as complexity requirements. Accounts are locked out after five failed login attempts. Passwords must be changed every 60 days, and cannot be reused within one year. A screen lock is also enabled by Domain Security Policy to activate after 15 minutes of inactivity. All accounts are locked at time of termination. To verify compliance, domain controls identify and lock all accounts that have been idle for more than 30 days. If they are still unused after 90 days, the account is deleted. Application Access Controls. For externally facing systems and applications, access to specific data is determined based on user roles and permission requirements established by our customers. At a minimum, applications hosted on Pear- son systems are required to provide the same level of security as network access would require. 7.7 Test Security

System Access Controls. Separation of duties between system administrators, application developers, application DBAs, and system DBAs provides clear definition of responsibility and prevents an individual from attaining the access rights that would allow for easy compromise. De-identifying Data During Scoring. The digital scoring platform routes student responses to scorers randomly. Scorers cannot see any student information such as name or other demographics, unless a student self-identifies in his or her response.

7.1.4.3 Transparency Requirements For paper testing, ACT will store all scored answer documents and used test books at a secure facility until securely destroyed based upon ACT’s retention schedule. This will include all materials used to capture student responses for scoring. ACT’s standard retention policy is to store all scanned answer documents and secure test materials for two years. ACT is also committed to retaining any electronic files including, but not limited to, scanned images, scored files, file exchanges, and contractual documentation at the ACT standard retention rate.

7.1.4.4 Student Privacy Pledge

ACT does not put limits on data retention as it is not aligned with our mission to be able to issue college reportable scores and conduct research to improve the ACT and provide data trends in education. Pearson, the subcontractor for ACT Aspire, LLC, has not signed the Student Privacy Pledge. Pearson remains committed to protecting student privacy and to working with state and local school officials to be good stewards of student personally identifiable information in our care.

7.2 Test Security and Administration Irregularities The irregularity reporting tool in the ACT Aspire Portal is intended for use primarily by school personnel to record any test administration irregularities that may affect student scores or the analysis of ACT Aspire results. Recording an irregularity for a student is not the same as voiding his or her test and dismissing him or her for prohibited behavior. Instructions for using this tool can be found in the related section of the ACT Aspire Portal User Guide. Testing personnel should use the tool to report any of the irregularities occurring within the room.

Room supervisors should enter test irregularities directly into the reporting tool in the ACT Aspire Portal, if possible. If room supervisors do not have access to the Portal, they should use the Testing Irregularity Report located at the end of this manual. All Testing Irregularity Reports should be forwarded to the local test coordinator after testing and be entered into the Portal.

7. 8 Test Security

For either format of the test, room supervisors should document any of the following occurrences during administration: • A student engages in any instance of prohibited behavior as outlined above. • A student becomes ill or leaves the room during testing. • A student fails to follow instructions (responds to questions randomly, obviously does not read questions prior to responding, or refuses to respond to questions). • A general disturbance or distraction occurs which could affect one or more students’ results. • A student questions the accuracy or validity of an item. • A student has a technical issue that interrupts testing (online testing). • A student has a defective test booklet or answer document (for paper testing).

For any instances where students can resume testing after illness, a technical issue, or a general disturbance, room supervisors should follow the instructions about how to resume a test session from the “Resume a Test” section of the Avocet website.

For the latest update of irregularity categories and codes used in the ACT Aspire Portal, see the “Irregularities” section of the Avocet website.

The irregularities in the Environment/Materials category include external factors that may affect student testing. These include things like outside noises or hot/cold room temperatures; damaged, missing or stolen test materials; and occurrences like power outages, severe weather, or emergency evacuations.

The Examinee category of irregularities includes student behaviors that may affect their performance or the performance of other students. These include the exhibition of prohibited behaviors described previously, student complaints about testing conditions, or challenges of test items.

The Staff category includes actions testing staff may engage in that affect testing. These include failure to follow testing procedures like mistiming a test or not reading the verbal instructions from the Room Supervisor Manual, or other inappropriate behavior like engaging in personal communication via other staff, telephone, or text during testing.

The Technical category pertains to the performance of online testing and includes system failure, slowness, or freezing; difficulties launching the test or with students using the testing platform; and other system issues like problems with using a keyboard, mouse, monitor, or other related hardware.

7. 9

CHAPTER 8 ACT Aspire Reporting

ACT Aspire assessments are vertically scaled and designed to measure student growth in a longitudinal assessment system. ACT Aspire’s scale scores are linked to college and career data through scores on the ACT and the ACT National Career Readiness Certificate™ (ACT NCRC®) program.

The objectives of ACT Aspire summative reporting are: • To clearly communicate what College and Career Readiness means and whether a student is on track to achieve it • To clearly communicate strengths and opportunity areas early on in a child’s educational career to enable greater personal academic and career success. • To include new measures that provide a better picture of student academic and personal achievement • To clearly communicate each student’s personal path to improvement and provide actionable next steps • To connect teaching and learning by enabling educators with reporting on student specific skills • To provide educators at all levels with the insights needed to improve student performance • To leverage technology to make the reporting user experience faster and more effective through the inclusion of deeper, richer insights and more valuable data online than in a traditional paper report • To leverage technology to make reporting results available across more platforms including computers and tablets

The ACT Aspire Portal serves as the delivery engine for test results. Individual student reports, a student performance file, and several aggregate reports are posted in the portal after assessment scoring is completed. Interpretive materials are available to help parents, teachers, and students better understand what is needed for continual academic growth.

8.1 ACT Aspire Reporting

Support materials include the following: Interpretive Guide for ACT Aspire Summative Reports, Understanding Your ACT Aspire Summative Results (for students and parents), and Sample Items for Reporting. These and other reporting resources can be found on the ACT Aspire Avocet web page.

Subsequent chapters in this Technical Manual describe in greater detail, the scores and metrics included in ACT Aspire reporting.

8.2 CHAPTER 9 ACT Aspire Scores

This chapter introduces the ACT Aspire subject scale scores, Composite score, ELA score and STEM score. Also described are scores based on subsets of items from each subject test, including the reporting category scores and the Progress with Text Complexity score.

9.1 ACT Aspire Subject Scores ACT Aspire scale scores are reported from grade 3 through EHS in English, mathematics, reading, science and writing. Scale scores for each subject are provided on a three-digit scale. Scale score ranges for each grade and subject are provided in Table 10.13 (Chapter 10). Summary descriptive statistics for the ACT Aspire subject scale scores from the 2013 scaling study are listed in Tables 10.8–10.12.

9.2 ACT Aspire Composite Score The ACT Aspire Composite score represents the overall performance on the English, mathematics, reading and science tests.1 It is calculated as the average of the scale scores on the four subjects rounded to an integer (.5 rounds up). The Composite score is reported only for students taking grade 8 or EHS tests in all four subjects. Table 9.1 lists descriptive statistics for ACT Aspire Composite scores from the 2013 scaling study.

Table 9.1. Composite Score Descriptive Statistics Based on the 2013 Scaling Study

Grade N Mean SD Min P10 P25 P50 P75 P90 P95 Max 8 2945 423.02 6.84 406 414 418 423 428 432 435 442 9 1641 423.99 7.93 408 414 418 423 430 435 438 447 10 1649 426.28 8.53 409 415 419 426 433 438 440 447

1 Note that the writing test is not included in the Composite score. 9.1 ACT Aspire Scores

Averaging four ACT Aspire subject test scale scores to obtain a Composite score implies that each test is contributing equally to the Composite score. The weights used to calculate the Composite (in this case, .25) are often referred to as nominal weights. Other definitions of the contribution of a test score to a composite score may be more useful. For example, Wang and Stanley (1970) described effective weights as an index of the contribution of a test score to a composite score. Specifically, the contribution of a test score is defined as the sum of the covariances between the test score and all components contributing to the Composite score. These contributions can be summed over tests and then each can be divided by its sum to arrive at proportional effective weights. Proportional effective weights are referred to as effective weights here.

With nominal weights of .25 for each test, the effective weights can be used to verify that the nominal weight interpretation of Composite scores (i.e., composite as an equal weighted combination of contributing scores) is reasonable. Wang and Stanley (1970) state that variables will rarely have equal effective weights, unless explicitly designed to do so. Therefore, the effective weights would need to deviate substantially and consistently from nominal weights to justify applying different weights or a different interpretation of weights.

The effective weights using spring 2013 scale scores are shown in Table 9.2 for the ACT Aspire Composite scores and range from .14 to .32. The mathematics tests at grades 3–6 appeared to have smaller effective weights compared to English, reading, and science due to relatively smaller variances and covariances. However, the equal nominal weights appeared justifiable for ACT Aspire Composite scores.

Table 9.2. Composite Score Effective Weights-Equal Nominal Weights

Grade English Mathematics Reading Science

3 .29 .16 .25 .29 4 .28 .14 .27 .31 5 .29 .15 .26 .29 6 .29 .18 .25 .28 7 .29 .21 .23 .27 8 .29 .23 .23 .26 9 .32 .23 .22 .24 10 .31 .24 .22 .24

9.2 ACT Aspire Scores

9.3 ACT Aspire ELA Score The ACT Aspire ELA score represents the overall performance on the English, reading and writing tests. It is calculated as the average of the scale scores on the three subjects rounded to an integer (.5 rounds up). ELA scores are provided to students at all grade levels between 3 and EHS but are provided only when a student obtains scale scores for all three subject tests at the same grade level. Nominal weights for ELA score are .33 (equal weights for each of the three contributing subject scores). The effective weights of spring 2013 scale scores are listed in Table 9.3 and ranged from .26 to .43. English appeared to contribute more to the effective weights for grades 7–10, with weights greater than .40, compared to writing, where weights dipped below .30. However, the effective weights did not deviate far from the equal nominal weights for ELA scores.

Table 9.3. ELA Score Effective Weights-Equal Nominal Weights

Grade English Reading Writing

3 .35 .30 .35 4 .33 .31 .36 5 .36 .32 .33 6 .36 .30 .33 7 .41 .31 .29 8 .42 .32 .26 9 .43 .30 .27 10 .43 .30 .27

9.4 ACT Aspire STEM Score The ACT Aspire STEM score represents the overall performance on the mathematics and science tests. It is calculated as the average of the scale scores on the two subjects rounded to an integer (.5 rounds up). The STEM score is provided for students at all grade levels between 3 and EHS but only when a student obtains scale scores for mathematics and science tests at the same grade level. Nominal weights for STEM are .5 (equal weights for each of the two contributing subject scores). The effective weights of spring 2013 scale scores are listed in Table 9.4 and ranged from .33 to .67. Similar to the Composite scores, the mathematics scores had smaller variances and covariances compared to science, particularly for grades 3, 4, and 5, which contributed to smaller effective weights. Despite the observed differences at lower grades, where science contributed nearly double to the STEM score variance compared to mathematics, the effective weights for grades 6–10 were progressively more similar to the nominal weights.

9.3 ACT Aspire Scores

Table 9.4. STEM Score Effective Weights-Equal Nominal Weights

Grade Mathematics Science

3 .36 .64 4 .33 .67 5 .37 .63 6 .42 .58 7 .45 .55 8 .48 .52 9 .50 .50 10 .50 .50

9.5 Reporting Category Scores Student performance is also described in terms of the ACT Aspire reporting category scores. Score reports describe the percent and number of points students earn out of the total number of points possible in each reporting category. A list of the reporting categories can be found in Table 9.5. Reporting category scores can be used as supplemental information to further identify strength and weakness in areas within a subject.

Table 9.5. Reporting Categories within each Subject Area

Subject Reporting Category

English Production of Writing Knowledge of Language Conventions of Standard English Mathematics Grade Level Progress Number and Operations—Fractions Number and Operations in Base 10 The Number System Number and Quantity Operations and Algebraic Thinking Expressions and Equations Ratios and Proportional Relationships Algebra Functions Geometry Measurement and Data Statistics and Probability

9.4 ACT Aspire Scores

Table 9.5. (continued)

Subject Reporting Category

Foundation Justification and Explanation Modeling Reading Key Ideas and Details Craft and Structure Integration of Knowledge and Ideas Science Interpretation of Data Scientific Investigation Evaluation of Models, Inferences, and Experimental Results Writing Ideas and Analysis Development and Support Organization Language Use and Conventions

9.6 Progress with Text Complexity The ACT Aspire reading test includes a Progress with Text Complexity indicator (“Yes” or “No”) to indicate whether students have made sufficient progress in reading increasingly complex texts (see Chapter 3, Section 3.2.2.3). This indicator is based on a set of items from the ACT Aspire reading test that are judged as indicative of performance when reading and understanding increasingly complex texts. It is calculated by comparing the percent correct on the items to an empirically defined cut score. Regression was used to predict the percent correct on the set of items indicative of progress with text complexity using the reading scale score at the reading benchmark (see Chapter 13 for a description of the reading benchmark).

9.7 Progress toward Career Readiness Indicator Students at grade 8 through EHS with Composite scores (i.e., receiving scores on English, mathematics, reading, and science tests) receive a Progress toward Career Readiness indicator to help them predict future achievement on the ACT National Career Readiness Certificate (NCRC) toward the end of high school using their ACT Aspire performance. See Chapter 15 for details of how the indicator was developed. The indicator reported to the student is a verbal statement indicating which certificate level (i.e., Bronze, Silver, or Gold) the student is making progress toward. One purpose of the indicator is to encourage students to think about the knowledge and skills future job training will require. Whether or not a student plans to enter college immediately after high school, there is likely a need for some type of postsecondary training.

9.5

CHAPTER 10 ACT Aspire Score Scale

This chapter presents the vertical scaling process for English, mathematics, reading, and science. The scaling test design was employed to establish the vertical scales for these ACT Aspire tests. Each student took the grade appropriate tests and the scaling tests of the same subjects. To be operationally feasible, four separate individual scaling tests, instead of one whole scaling test spanning all grade levels per subject were constructed. These four separate individual scaling tests were then re-connected to recover the whole scaling test for each subject for further analysis. Interim vertical scales with desired psychometric properties were first constructed based on the re-connected whole scaling test. Each on-grade test was then linked to the interim vertical scale for each subject and each grade. To maintain the vertical scale, future forms will be horizontally equated to the base form. Scaling for ACT Aspire writing, presented at the end of the chapter, was rubric-driven and writing scale scores do not have a vertical scale interpretation like those for scale scores obtained from other subjects.

10.1 ACT Aspire Scaling Philosophy A conceptual definition of growth is crucial in developing the vertical scale. Kolen and Brennan (2014) provide two general definitions—the domain definition and the grade- to-grade definition. Under the domain definition, growth is defined over the entire range of test content covered by the battery, or the domain of content. Under the grade-to- grade definition, growth is defined over the content that is on a test level appropriate for typical students at a particular grade. In designing ACT Aspire assessments, ACT did not treat each grade level skills isolated only to that grade level. ACT Aspire assessments and items were designed to elicit evidence across the construct and across the learning trajectory. As such, ACT ascribes to a domain definition of growth, where student achievement over the entire range of content is considered. This conceptualization of growth leads naturally to the scaling test data collection design. 10.1 ACT Aspire Score Scale

The scaling test design involves creation of a single test with items that cover the range of content and difficulty across the domain. This test is then administered to students across all covered grade levels and vertical scaling is used to place performance of students across grade levels onto the same scale. The next section describes in detail the scaling study used to establish the ACT Aspire score scales.

10.2 Scaling Study 10.2.1 Scaling Study Design The purpose of the scaling study is to establish a vertical scale for the ACT Aspire English, mathematics, reading, and science assessments. The ACT Aspire assessments include seven grade level tests per subject: grades 3–8 and early high school (EHS), which is given to students in grades 9 and 10. In addition, by including grade 11 in the scaling study, the vertical scale could be extended to include the ACT (see Appendix B), even though the ACT is not included in the ACT Aspire assessments.

The scaling test design was adopted to create the vertical scale for each subject. Under this design, each student completes two tests: an on-grade test appropriate for his/her grade level and a scaling test with items from multiple grades. The vertical scale is defined using the scaling test, and the scores for each on-grade test are linked to the scale through the scaling test. 10.2.2 Construction of Scaling Tests Design of the scaling tests is a key component in creating a sound vertical scale under the scaling test design. Items from the scaling tests should cover all content that defines the vertical scale and be sensitive enough to measure growth. Since the vertical scale covers grade 3 to grade 11, it is ideal to construct one scaling test per subject with items from all grade levels and administer this single scaling test to students from grades 3 to 11. However, to include a sufficient number of items from each content domain in the scaling test, this single test consisting of eight levels of tests (i.e., grades 3–8, 9/10, and 11) would be much too long to administer operationally and would require students in certain grades to complete items which are significantly above or below their grade level and likely achievement level. The length of the scaling test and the prospect of, for example, administering high school students grade 3 items and third graders items intended for high school students, would be unlikely to result in good measurement. To create scaling tests of reasonable length, a single scaling test covering items from grades 3 through 11 (hereafter referred to as the whole scaling test and abbreviated WST) was broken into four separate tests in this study, each of which includes items from two or three consecutive grades with one overlapping grade (referred to as bridge grades) between tests. Specifically, scaling test 1 (ST1) includes items from grades 3 to 5, scaling test 2 (ST2) includes items from grades 5 to 7, scaling test 3 (ST3) includes items from grades 7 to EHS, and scaling test 4 (ST4) includes items from EHS and the ACT (see Figure 10.1). 10.2 ACT Aspire Score Scale

Figure 10.1. Construction of the ACT Aspire scaling tests

The scaling tests were designed to represent a miniature version of each on-grade test. The items chosen for the miniature version of each on-grade test were selected to be representative with respect to both content coverage and item statistics to the extent possible (approximately one-third of the items on an on-grade form for each grade were used in the scaling tests). There were no common items between the scaling test and the on-grade test administered to a student.

Within each scaling test, items were grouped and sorted by grade levels so that lower grade items were positioned earlier in the test than upper grade items. Items in the bridge grade 5 were identical in ST1 and ST2. Similarly, items in the bridge grades 7 and EHS were identical in ST2 and ST3, and in ST3 and ST4, respectively. The only difference between these items in different scaling tests was that they appear at the end of the lower level scaling test but at the beginning of the upper level scaling test. The time limit for each scaling test was 40 minutes.

10.2.3 Data Collection Design The purpose of the scaling study was to set up a vertical scale spanning from grades 3 to 11 and cross-sectional data collected for the study would be used for growth analysis. To minimize the impact on student growth caused by potential differences with regard to student characteristics across grade levels, it was desired to have cross-grade samples that were similarly representative of student populations. To achieve this goal, invitations were sent out at the district level. If districts agreed to participate in the study, they were asked to test students from each grade level. While participating districts were requested to test at least some students from every grade level, not every school in a district needed to participate. ACT encouraged each district to test at least half, but preferably all of its students.

Each student in grades 3–10 was asked to take five on-grade tests at the appropriate grade level (English, mathematics, reading, science, and writing) and two scaling 10.3 ACT Aspire Score Scale

tests of different subjects, all of which were delivered online. They were not required to take the tests in one sitting or on the same day; a four-week testing window from April 22 through May 17, 2013, provided flexibility in scheduling and administering the tests. grade 11 students took the ACT battery tests with multiple choice items on paper plus the ACT writing test on paper if the district selected to do so. All ACT tests were administered under the standardized condition and students received college reportable ACT scores. In addition, each grade 11 student took three online delivered 30-minute constructed-response sections as the augmented piece to the ACT (mathematics, reading, and science), two scaling tests of different subjects, and the ACT writing test (if the districts did not select to administer the paper ACT writing test together with the ACT battery tests). In total, around 37,000 students from 232 schools and 23 districts in 14 states completed at least one test in the study. Tables C1–C5 (Appendix C) presents demographic information for students participating in the scaling study.

Once data were collected, analyses to create the scale were conducted separately for each subject. Students from grades 9 and 10 were combined for analysis of the early high school test. Table 10.1 presents the data structure used to analyze each subject. Each row lists the student grade level and the columns list the on-grade or scaling test assigned to students. Bridge grades are listed twice, one row as “A” for those students assigned to a lower level scaling test and one row as “B” for those assigned the upper level scaling test. The second column lists the on-grade test taken by students. The third column lists the scaling test taken by students. For example, grade 7 students (bridge grade) assigned to the “A” group would take the grade 7 on-grade test and the grades 5–7 scaling test (ST2). Because grades 9 and 10 were bridge grades, grade 9 students completed the EHS on-grade test plus either the lower level scaling test (ST3) or the upper level scaling test (ST4). Grade 10 followed a similar pattern: grade 10 students completed the EHS on-grade test plus ST3 or ST4.

In the scaling study, each student in each grade completed one scaling test and one on-grade test for the same subject. Approximately twice as many students were sampled for bridge grades (5, 7, and 9/10) compared to those from non-bridge grades to allow half of the bridge grade students to take the lower level scaling test and half to take the upper level one. All students in the same grade took the same on-grade test. The two groups taking lower/upper level scaling tests in each bridge grade were designed to be randomly equivalent. Once lists of recruited students were available, students at a bridge grade were randomly assigned to take the upper or lower level scaling test by spiraling tests across students. For example, for four students at a bridge grade within the same classroom, if the first student was assigned the lower level scaling test, the second would be assigned the upper, the third the lower, and the fourth the upper level scaling test. This spiraling pattern continued for all students in this classroom and continued into other classrooms and schools at the same grade level.

10.4 ACT Aspire Score Scale

Table 10.1. Data Collection Design of the Scaling Study

Student Grade On-Grade Test Scaling Test

3 3 ST1

4 4 ST1

5A 5 ST1

5B 5 ST2

6 6 ST2

7A 7 ST2

7B 7 ST3

8 8 ST3

9A EHS ST3 9B EHS ST4

10A EHS ST3 10B EHS ST4 11 ACT ST4 12 ACT ST4

Note. ST1 includes items from grades 3–5; ST2 incudes items from grades 5–7; ST3 includes items from grades 7–EHS; ST4 includes items from EHS and the ACT.

10.2.4 Creation of Vertical Scales Under the scaling test design, the vertical alignment of student performance across all grades was obtained from the scaling tests. Once the alignment was established, a scale with desirable properties (i.e., target mean, standard deviation, standard error of measurement, number of scale score points, etc.) was created. Then, the base form of each on-grade test was linked to the vertical scale. As new on-grade forms are developed, they are horizontally equated to the base form to maintain the vertical scale (see Chapter 20).

The process of establishing the ACT Aspire vertical scale can be summarized in three steps: 1. Link across the four scaling tests to establish the vertical relationship from grades 3 to 11 in a subject. 2. Create the scale with desired properties based on the linked scaling test. 3. Link each on-grade test to the vertical scale.

Detailed descriptions of these three steps are presented below.

10.5 ACT Aspire Score Scale

Step 1: Link across the Four Scaling Tests The goal of this step was to link the four separate scaling tests so that scores of students taking different scaling tests were put on the same scale. Note that if the whole scaling test were given to all students, this step would not be necessary. ST3 (composed of items from grades 7 to EHS) was selected as the base test to conduct the linking and was chosen because (1) it contained items at the top of the ACT Aspire scale, (2) two of the three other scaling tests (ST2 and ST4) could be linked to it directly, and (3) it is adjacent to ST4, which was used to link the ACT with constructed-response tests (see Appendix B) to the ACT Aspire scale.

Adjacent scaling tests were linked through random equivalent groups taking lower/ upper level scaling tests in each bridge grade (e.g., link ST1: grades 3-5 to ST2: grades 5-7 using grade 5 students). This linking design assumes the two groups at the bridge grades, one taking the upper level scaling test (denoted as B) and one taking the lower level scaling test (denoted as A), are randomly equivalent.

At the data collection stage, form spiraling occurred at the student level within each grade in a school. Therefore, the number of students assigned to the lower level scaling test was very close to that of students taking the upper level test in each bridge grade of each school. However, for a variety of reasons, students recruited and assigned to a form may not have actually tested. This could lead to unbalanced numbers of students taking different scaling tests that would affect the defensibility of randomly equivalent groups. A data cleaning rule was applied to attempt to reduce such effects and help ensure group equivalence. Specifically, if all students in the bridge grade in a school only took one level of a scaling test or if the ratio of students taking one level of the scaling test over the other level was more than 2, all students in that grade from that school were removed from the analysis.

To further evaluate whether the two groups were equivalent after data cleaning, on-grade test raw scores were compared between the two groups because they took the same on-grade test form. Table 10.2 presents sample size, mean, and standard deviation of on-grade test scores for the two groups in the bridge grades, as well as p-values of independent-sample t-tests and effect sizes between the two groups. None of the t-tests were statistically significant at the .05 level — an indication that scores for the two groups from each bridge grade did not differ significantly. Magnitudes of effect sizes were also very small (mostly within ±0.05).

Instead of using random equivalent groups to link across scaling tests, an alternative possible design was to link through common items between adjacent scaling tests. However, since common items appeared at the end of the lower level scaling test and at the beginning of the upper level scaling test, context effects were a concern. Items might appear easier when they were positioned at the beginning of the test than at the end of the test. To investigate whether context effects were present, the total raw score points on the common items were compared between the two groups

10.6 ACT Aspire Score Scale

of students taking lower/upper level scaling tests. Since it was previously shown that the two groups were randomly equivalent, the average raw scores on the common item set across groups should be similar if there were no context effects. On the other hand, if context effects do exist, non-ignorable mean differences on the common items across groups would be expected. For example, it has been shown that the Groups 5A (grade 5, taking lower level scaling test) and 5B (grade 5, taking upper level scaling test) are equivalent, so if the average raw score on grade 5 items in the scaling test for 5A is different from that for 5B, context effects likely exist. Table 10.3 shows descriptive statistics of lower and upper level scaling test raw scores for groups taking common items, as well as p-values from t-tests and effect sizes between scores from the two groups. Group B scored higher on the common items for all subjects and all bridge grades. Items appearing at the beginning of the upper level scaling test appeared to be easier than when the same items appeared at the end of the lower level scaling test. The t-tests were statistically significant (at the 0.05 level) for grades 5 and 7 in all subjects and grade 9/10 in mathematics and science. Context effects appeared to affect performance on the common items, which is a violation of the assumption that items should perform similarly under the common- item design. Therefore, this design was not adopted.

Table 10.2. Raw Scores of On-Grade Tests for Two Groups Taking Lower/Upper Level Scaling Tests

Group A Group B p-value from Effect Subject Grade N Mean SD N Mean SD t-test Size

English 5 845 14.588 4.345 871 14.701 4.456 0.595 –0.03 7 889 19.580 5.898 928 19.496 5.820 0.760 0.01 9/10 667 26.187 10.461 687 25.128 10.431 0.062 0.10 Mathematics 5 868 10.058 3.382 870 10.176 3.459 0.472 –0.03 7 863 15.651 6.453 882 15.593 6.239 0.849 0.01 9/10 688 17.247 8.440 706 17.479 8.927 0.618 –0.03 Reading 5 822 15.798 5.492 833 15.739 5.717 0.831 0.01 7 806 13.166 5.241 800 13.340 5.364 0.511 –0.03 9/10 707 14.651 7.058 684 14.415 7.088 0.534 0.03 Science 5 727 18.171 7.132 721 18.337 7.104 0.657 –0.02 7 819 18.812 7.511 834 18.801 7.955 0.977 0.00 9/10 634 14.218 8.082 648 14.520 7.772 0.495 –0.04

10.7 ACT Aspire Score Scale

Table 10.3. Raw Scores on Common Items for Two Groups Taking Lower/Upper level Scaling Tests

Group A Group B p-value Effect Subject Grade N Mean SD N Mean SD from t-test Size

English 5 845 10.174 3.818 871 11.200 3.292 < 0.0005 -0.29 7 889 9.529 3.907 928 10.335 3.534 < 0.0005 -0.22 9/10 667 12.337 5.614 687 12.582 5.280 0.408 -0.04 Mathematics 5 868 4.358 1.930 870 4.670 1.952 0.001 -0.16 7 863 3.928 2.299 882 4.195 2.296 0.015 -0.12 9/10 688 3.911 2.954 706 4.249 2.840 0.030 -0.12 Reading 5 822 3.456 1.720 833 3.619 1.635 0.048 -0.10 7 806 4.227 2.262 800 4.591 2.309 0.001 -0.16 9/10 707 4.494 3.225 684 4.722 3.182 0.185 -0.07 Science 5 727 3.884 2.129 721 4.277 2.080 < 0.0005 -0.19 7 819 3.891 1.984 834 4.125 1.910 0.015 -0.12 9/10 634 6.412 4.302 648 7.110 4.180 0.003 -0.16

Item response theory (IRT) was used as the statistical method to conduct the linking. This method involves putting IRT item parameters from the four scaling tests on the same scale. The two-parameter logistic (2PL) model was used for the dichotomously-scored items. The generalized partial credit model (GPCM) was used for polytomously-scored (i.e., constructed-response) items. The 2PL model contains two parameters for an item plus a parameter for student proficiency (theta). The GPCM is analogous to the 2PL model but incorporates more than two score categories. All calibrations were conducted using single group analysis with PARSCALE software (Muraki & Bock, 2003). Each of the four scaling tests was first independently calibrated. Under the randomly equivalent groups design, the mean and standard deviation (SD) of theta scores between the two groups in the bridge grade were used to compute the scale transformation constants (see Kolen & Brennan 2014, pp. 180–182). Students with extreme theta scores (less than or equal to −6 or greater than or equal to +6) were excluded from the calculation of mean and SD, because these theta scores were fixed by arbitrary boundaries set up in the scoring software. The obtained scale transformation slope and intercept were then applied to item parameter estimates from each independent calibration to generate the scaled item parameters for all scaling tests. Students’ theta scores were then estimated from the scaled item parameters which enabled them to be put on the same scale.

To examine and ensure the process of linking the four scaling tests did not distort the grade-to-grade growth relationships in the scaling tests, effect sizes were computed

10.8 ACT Aspire Score Scale

based on the predicted whole scaling test scores and compared against the effect sizes computed from individual scaling test raw scores. The predicted whole scaling test scores were true score estimates obtained by applying the scaled item parameter estimates of all items on the whole scaling test given the scaled theta estimates of each student obtained from the part of the whole scaling test each student was actually administered.

Effect size was computed as the difference between the mean scores of adjacent grades divided by the square root of the average variances for the two groups (Kolen & Brennan, 2014, p. 461). Using the reading test as an example, the grade-to-grade growth in effect size derived from the whole scaling test scores using 2PL model and effect size derived from the individual scaling test raw scores are plotted in Figure 10.2. For example, the grade 3 to grade 4 effect size using individual scaling test raw scores was computed using ST1 raw scores on grades 3 and 4 students. For the bridge grades, the effect size was computed between one of the random groups and its adjacent grade taking the same scaling test. For example, effect size between grades 4 and 5 was computed on ST1 raw scores between grade 4 and 5A students; effect size between grades 5 and 6 was computed on ST2 raw scores between grade 5B and 6 students. Since 5A and 5B groups were shown to be equivalent, either group could be treated as representative of the entire group. The effect size between grades 9 and 10 was the average of effect sizes for 9A versus 10A and 9B versus 10B.

The overall growth pattern is very similar between the two sets of effect sizes, which indicate that the linking process did not distort the original grade-to-grade relationship. The magnitude of growth tends to decrease as grade level increases, with growth from grades 3 to 5 among the largest. There is zero to slightly negative growth observed between grade 5 and 6.

Figure 10.2. Effect sizes on individual scaling test raw score scale and whole scaling test raw score scale derived from the 2PL model

Reading 0.60

0.50

0.40

0.30

0.20

Effect Size 0.10

0.00 3-4 4-5 5-6 6-7 7-8 8-9 9-10 10-11 -0.10 Grade Span

Individual Scaling Test Raw Score Whole Scaling Test Score 2PL 10.9 ACT Aspire Score Scale

Step 2: Create the Vertical Scale Based on the Linked Scaling Tests Generate the Projected Whole Scaling Test Raw Scores After step 1 was completed, IRT item parameters for all four scaling tests were placed on the same scale. Students’ theta scores were also on the same scale when estimated from the scaled item parameters. Based on the scaled item parameter estimates and theta estimates, students’ true scores on the whole scaling test, which were on the raw score metric, were estimated from the IRT test characteristic curve (TCC) of the whole scaling test.

Create an Interim Scale with Constant Conditional Standard Error of Measurement (CSEM) The conditional standard error of measurement (CSEM) of the raw scores on a test is typically an inverted U shape with the CSEM values being much smaller at the two ends. To stabilize the CSEM along the score scale, raw scores on the whole scaling test were nonlinearly transformed to an interim score scale using the arcsine transformation (e.g., see Kolen & Brennan, 2014, p. 405). Although this stabilization was applied to the scaling test to obtain a constant CSEM property for the ACT Aspire scale, after the on-grade tests were linked to the whole scaling test, there was no guarantee that the constant CSEM property would be maintained on the on-grade tests. Empirical results indicate that applying the arcsine transformation did help to stabilize the scale score CSEM of the on-grade test after it was linked to the whole scaling test.

To compute the CSEM, an extended Lord-Wingersky recursive algorithm (Hanson, 1994; Wang, Kolen, & Harris 2000, p. 219; Kolen & Brennan, 2014, p. 199) was adopted to obtain the expected distribution on the interim scale score for each student based on his/her theta score. The standard deviation of the expected score distribution was the CSEM of the interim score scale for that student. Weights were used to equalize the contribution of students from different grade levels to the CSEM calculation because sample sizes varied across grades, particularly for the bridge grades where the data collection design resulted in larger samples of students compared to other grades (e.g., see sample sizes listed in Tables 10.4–10.7).

Linear Transformation of the Interim Scale to the Final Scale The interim scale scores, with the property of constant CSEM, were then linearly transformed to the ACT Aspire scale. Since linear transformation does not change the relative magnitude of the CSEMs, the constant CSEM property was carried over to the final scale. The linear transformation was selected so that the standard error of measurement on the whole scaling test was about 1.8 scale score points and the mean was set to an arbitrary value to anchor the scale.

10.10 ACT Aspire Score Scale

Step 3: Link the On-Grade Test to the Vertical Scale The last step in the vertical linking process was to link each of the on-grade English, mathematics, reading, and science tests to the vertical scale established using the whole scaling test in each subject. A single group linking design was adopted, since the same group of students took both the on-grade test and the scaling test in the same subject.

The IRT observed score linking method (Kolen & Brennan, 2014) was used to create the link. First, each on-grade test form was calibrated independently. Second, item parameter estimates from the on-grade test were transformed to the whole scaling test scale by matching the mean and standard deviation of theta scores estimated from each on-grade test to theta scores estimated from the scaled item parameters of the scaling tests. Third, the estimated distribution of number-correct scores (using scaled item parameters) on the on-grade form was linked to that of the whole scaling test. The IRT observed score linking yields the raw-to-scale conversion for each on-grade test. The scale scores were rounded, truncated, and shifted to desired score ranges to provide the final reportable scale scores.

10.3 Evaluation of the Vertical Scales Once the scale was created and students’ scale scores were generated, the following analyses were conducted to evaluate the scales: 1. Check whether the scale maintains the on-the-same-scale property. When multiple grades of students took the same scaling test, their scores on that scaling test were automatically on the same scale. The entire scaling process for ACT Aspire involved multiple steps, including connecting individual scaling tests and linking the on-grade test to the whole scaling tests. Therefore, it was important to verify that ACT Aspire scale scores obtained from on-grade tests could maintain the vertical relationships obtained from the scaling test. This property is referred to as the on- the-same-scale property. To evaluate this property, scaling test raw score on each of the four scaling tests were used as a reference to define the observed relationships across grade level. If the on-the-same-scale property was maintained, students who had the same raw scores on the scaling test should have similar scale scores derived from the on-grade test, even though their scale scores are from on-grade tests at different grade levels. 2. Evaluate the constant CSEM property. As mentioned above, the CSEM stabilization process was applied to the scale defined by the scaling tests. To evaluate whether this property was maintained for the on-grade tests, CSEM curves for each on- grade test were plotted and examined. 3. Evaluate the growth pattern. The growth pattern derived from ACT Aspire scale scores were examined and compared against the growth pattern derived from the raw scaling test scores.

10.11 ACT Aspire Score Scale

10.4 Results 10.4.1 Scaling Test Raw Scores Tables 10.4–10.7 present the means and standard deviations of raw scores on individual scaling tests and the means and standard deviations of raw scores on the portion of grade-specific items in each test. For example, the grade 3 students’ average raw score is 22.525 for the English scaling test (ST1) and when items are grouped by grade levels, the average score is 7.313 on all grade 3 items, 7.481 on all grade 4 items, and 7.731 on all grade 5 items. The average scores on items from different grade levels (i.e., across the columns) are not directly comparable since the number of items is different across grade levels. Similarly, average scaling test raw scores are only comparable among groups who took the same scaling test since the numbers of score points and most of the items are different across the four scaling tests.

Within each scaling test, summarizing the average total raw score on the scaling test by grade-level provides diagnostic information regarding where growth occurs. For example, a slightly negative growth occurs between grade 5 students (5B) and grade 6 students on ST2 in reading. Reviewing the means on items grouped by grade levels reveals that reverse growth occurred on the grade 6 items for reading. Another example of negative growth is on ST4 in reading between group 9B and 10B. Group 10B performed worse on both the EHS and the ACT items.

Context effects are also observable in Tables 10.4–10.7 between the bridge grade groups of students (A and B groups), because the common item mean scores are always higher in the B group (where common items are given at the beginning of the test) than the A group (where common items are given at the end of the test). This is true for all subjects and all bridge grades. As explained earlier, this was one reason why the common item linking design was not used to develop the vertical scale.

The standard deviations of raw scores tended to increase as grade level increased within the same scaling test. In other words, group variability increased as students advance in grade level.

10.4.2 Evaluating On-the-Same-Scale Property Figure 10.3 presents plots of the average scale scores per grade against the scaling test raw score, by scaling test. As can be seen, students with the same scaling test raw score have similar scale scores, on average, regardless of which on-grade tests they have taken. These results are consistent with an interpretation of scores of on-grade tests as being on the same vertical scale.

10.12 ACT Aspire Score Scale

Table 10.4. Mean and Standard Deviation of Raw Scores on English Scaling Test (ST) by Grade Specific Items

Mean (SD) of Grade Specific Items (Score Range) ST Mean (Score (SD) G3 G4 G5 G6 G7 G8 G_EHS G_ACT Range) Grade N of ST (0-16) (0-17) (0-17) (0-16) (0-17) (0-16) (0-25) (0-45)

22.525 7.313 7.481 7.731 3 1810 (7.975) (2.542) (3.308) (3.499)

ST1 26.078 8.208 8.730 9.140 4 1617 (0 – 50) (8.320) (2.585) (3.347) (3.689) 29.060 9.058 9.828 10.174 5A 845 (8.631) (2.713) (3.380) (3.818) 28.021 11.200 8.503 8.318 5B 871 (8.704) (3.292) (2.998) (3.582)

ST2 28.548 11.386 8.490 8.672 6 1824 (0 – 50) (9.528) (3.604) (3.189) (3.834) 30.670 12.048 9.093 9.529 7A 889 (9.678) (3.496) (3.280) (3.907) 29.763 10.335 8.672 10.755 7B 928 (10.335) (3.534) (3.524) (4.514) 31.769 10.818 9.171 11.781 8 1436 ST3 (11.059) (3.683) (3.677) (4.807) (0 – 58) 32.402 11.117 9.172 12.114 9A 343 (12.237) (3.975) (4.042) (5.228) 32.710 10.843 9.293 12.574 10A 324 (13.787) (4.267) (4.410) (5.995) 29.481 12.332 17.149 9B 368 (12.552) (5.122) (8.281)

ST4 31.367 12.871 18.495 10B 319 (0 – 70) (14.036) (5.45) (9.288) 32.925 13.943 18.982 11 760 (13.797) (5.206) (9.356)

10.13 ACT Aspire Score Scale

Table 10.5. Mean and Standard Deviation of Raw Scores on Mathematics Scaling Test (ST) by Grade Specific Items

Mean (SD) of Grade Specific Items (Score Range) ST Mean (Score (SD) G3 G4 G5 G6 G7 G8 G_EHS G_ACT Range) Grade N of ST (0-12) (0-12) (0-13) (0-11) (0-13) (0-7) (0-17) (0-21)

9.090 3.732 2.697 2.661 3 1793 (3.573) (1.944) (1.472) (1.503)

ST1 11.580 4.573 3.345 3.662 4 1624 (0 – 37) (4.360) (2.029) (1.841) (1.743) 13.778 5.315 4.105 4.358 5A 868 (5.169) (2.133) (2.212) (1.930) 9.924 4.670 2.600 2.654 5B 870 (3.434) (1.952) (1.237) (1.501)

ST2 10.817 4.812 2.939 3.065 6 1751 (0 – 37) (4.216) (2.047) (1.428) (1.752) 12.818 5.395 3.495 3.928 7A 863 (5.17) (2.129) (1.693) (2.299) 9.347 4.195 2.407 2.745 7B 882 (4.689) (2.296) (1.508) (1.942) 10.580 4.504 2.682 3.394 8 1451 ST3 (5.289) (2.369) (1.505) (2.375) (0 – 37) 11.213 4.559 2.877 3.777 9A 381 (6.218) (2.704) (1.679) (2.719) 12.160 5.156 2.925 4.078 10A 307 (7.023) (2.836) (1.745) (3.219) 10.580 4.133 6.447 9B 376 (5.691) (2.632) (3.427)

ST4 11.361 4.382 6.979 10B 330 (0 – 38) (6.926) (3.060) (4.269) 12.765 5.139 7.626 11 669 (6.632) (3.041) (4.087)

10.14 ACT Aspire Score Scale

Table 10.6. Mean and Standard Deviation of Raw Scores on Reading Scaling Test (ST) by Grade Specific Items

Mean (SD) of Grade Specific Items (Score Range) ST Mean (Score (SD) G3 G4 G5 G6 G7 G8 G_EHS G_ACT Range) Grade N of ST (0-8) (0-8) (0-7) (0-9) (0-10) (0-5) (0-13) (0-18)

8.077 3.003 2.757 2.317 3 1769 (4.054) (1.686) (1.879) (1.451) ST1 9.811 3.516 3.587 2.709 4 1569 (0 – 23) (4.467) (1.755) (2.04) (1.573) 12.102 4.153 4.493 3.456 5A 822 (4.67) (1.71) (2.154) (1.72) 12.552 3.619 5.261 3.672 5B 833 (4.903) (1.635) (2.322) (2.036) ST2 12.495 3.624 5.077 3.794 6 1759 (0 – 26) (5.357) (1.695) (2.422) (2.179) 13.878 3.97 5.681 4.227 7A 806 (5.384) (1.685) (2.388) (2.262) 10.824 4.591 2.644 3.589 7B 800 (5.344) (2.309) (1.502) (2.491) 11.775 4.861 2.864 4.05 8 1321 ST3 (5.687) (2.359) (1.597) (2.697) (0 – 28) 12.212 4.982 2.876 4.355 9A 386 (6.105) (2.375) (1.621) (2.988) 12.879 5.202 3.016 4.66 10A 321 (7.243) (2.807) (1.769) (3.487) 11.756 4.772 6.984 9B 369 (6.02) (3.094) (3.529) ST4 11.514 4.663 6.851 10B 315 (0 – 31) (6.701) (3.287) (3.932) 12.795 5.413 7.382 11 707 (6.181) (3.059) (3.775)

10.15 ACT Aspire Score Scale

Table 10.7. Mean and Standard Deviation of Raw Scores on Science Scaling Test (ST) by Grade Specific Items

Mean (SD) of Grade Specific Items (Score Range) ST Mean (Score (SD) G3 G4 G5 G6 G7 G8 G_EHS G_ACT Range) Grade N of ST (0-6) (0-16) (0-9) (0-14) (0-8) (0-11) (0-17) (0-22)

12.108 3.538 6.095 2.475 3 1806 (5.049) (1.515) (2.909) (1.699)

ST1 14.453 3.983 7.362 3.108 4 1551 (0 – 31) (5.466) (1.434) (3.168) (1.873) 16.993 4.415 8.693 3.884 5A 727 (5.945) (1.286) (3.484) (2.129) 15.434 4.277 7.712 3.445 5B 721 (6.267) (2.08) (3.247) (1.852)

ST2 15.687 4.327 7.814 3.546 6 1514 (0 – 31) (6.789) (2.236) (3.448) (1.984) 17.098 4.75 8.457 3.891 7A 819 (6.836) (2.245) (3.44) (1.984) 14.138 4.125 4.788 5.225 7B 834 (6.306) (1.91) (2.266) (3.302) 15.243 4.391 5.1 5.752 8 1426 ST3 (6.993) (1.946) (2.4) (3.749) (0 – 36) 16.202 4.464 5.39 6.348 9A 336 (8.015) (2.141) (2.728) (4.195) 16.319 4.47 5.366 6.483 10A 298 (8.281) (2.181) (2.707) (4.425) 14.137 6.747 7.39 9B 328 (6.547) (3.832) (3.517)

ST4 15.272 7.481 7.791 10B 320 (0 – 39) (7.795) (4.484) (4.066) 15.54 7.672 7.869 11 685 (7.576) (4.232) (4.138)

10.16 ACT Aspire Score Scale

Figure 10.3. Scatter plots of average scale scores against scaling test raw scores by grade

A. English

10.17 ACT Aspire Score Scale

Figure 10.3. (continued)

B. Mathematics

10.18 ACT Aspire Score Scale

Figure 10.3. (continued)

C. Reading

10.19 ACT Aspire Score Scale

Figure 10.3. (continued)

D. Science

10.20 ACT Aspire Score Scale

10.4.3 Evaluating Constant Conditional Standard Error of Measurement Property The CSEMs of raw scores and scale scores for English, mathematics, reading, and science are plotted in Figure 10.4. Each curve represents a grade level, and each figure represents CSEM for raw or scale scores on one ACT Aspire subject.

The ACT Aspire score scales begin at 400 and have different maximum value (up to 460) depending on the grade and subject area. For most grades and subjects, the scale score CSEM curves are relatively flat along the scale range, especially when compared to the raw score CSEM, which show a typical inverted U-shape.

In Figure 10.4, for most ACT Aspire scale score points, CSEMs are within a range of two scale score points. The CSEMs drop dramatically when scale scores are low and they never appear higher than roughly 4 scale score points. While these CSEMs are not perfectly flat, which would imply a perfectly constant CSEM across the score scale, from a practical perspective they are reasonably constant. For example, the second graph in Figure 10.4 presents CSEMs for English, and the CSEMs for most of the scale score points are between 2 and 4 scale score points across all grades. Within each grade for all subjects, scale score CSEMs are generally within one scale score point for most of the score scale points. For example, the grade 4 English CSEM for scale scores fluctuates between 2 and 3 across most of the score scale points excluding the bottom few scale score points.

10.21 ACT Aspire Score Scale

Figure 10.4. Conditional standard error of measurement on raw and scale score scales

A. English

10.22 ACT Aspire Score Scale

Figure 10.4. (continued)

B. Mathematics

10.23 ACT Aspire Score Scale

Figure 10.4. (continued)

C. Reading

10.24 ACT Aspire Score Scale

Figure 10.4. (continued)

D. Science

10.25 ACT Aspire Score Scale

10.4.4 Growth Patterns Effect sizes derived from comparing different grade levels across three types of scores were computed and compared. Figure 10.5 displays three sets of effect sizes, one derived from the final ACT Aspire scale scores on the on-grade test, one from the projected whole scaling test scores, and one from individual scaling test raw scores. Effect sizes derived from individual scaling test raw scores were computed between two groups taking the same scaling test, as described in the section of “Creating the Vertical Scale – Step 1: Link across the Four Scaling Tests.”

In general, the three sets of effect sizes for lower grades (3–8) are very similar. The patterns diverge in some cases for upper grades. One reason might be that grade 9 and 10 students were combined during the scaling analysis. Also, there were two sources of growth from grade 9 to grade 10: between 9A and 10A students who took ST3 and between 9B and 10B who took ST4. The effect size computed using the scaling test raw scores was the average of both sources. If the growth patterns were different from 9A to 10A than from 9B to 10B, this might lead to different effect sizes between 9 and 10 when combining the A and B groups.

Figure 10.5. Effect sizes on individual scaling test raw score scale, whole scaling test raw score scale, and ACT Aspire score scale

English

0.90 0.80 0.70 0.60 0.50 0.40 0.30

Effect Size 0.20 0.10 0.00 3-4 4-5 5-6 6-7 7-8 8-9 9-10 10-11 Grade Span Individual Scaling Test Raw Score Whole Scaling Test Score ACT Aspire Scale Score

10.26 ACT Aspire Score Scale

Figure 10.5. (continued) Mathematics

0.90 0.80 0.70 0.60 0.50 0.40 0.30

Effect Size 0.20 0.10 0.00 3-4 4-5 5-6 6-7 7-8 8-9 9-10 10-11 Grade Span

Individual Scaling Test Raw Score Whole Scaling Test Score ACT Aspire Scale Score Reading

0.90 0.80 0.70 0.60 0.50 0.40 0.30

Effect Size 0.20 0.10 0.00 -0.10 3-4 4-5 5-6 6-7 7-8 8-9 9-10 10-11 Grade Span Individual Scaling Test Raw Score Whole Scaling Test Score ACT Aspire Scale Score

Science

0.90 0.80 0.70 0.60 0.50 0.40 0.30 Effect Size 0.20 0.10 0.00 3-4 4-5 5-6 6-7 7-8 8-9 9-10 10-11 Grade Span Individual Scaling Test Raw Score Whole Scaling Test Score ACT Aspire Scale Score

10.27 ACT Aspire Score Scale

Basic descriptive statistics of the final scale scores for all students in the scaling study are presented in Tables 10.8–10.12.. Note that the n-counts in these tables are larger than the groups used for the scaling analysis, where only students who took both scaling tests and on-grade tests were included. Since this is not longitudinal data and samples may not be representative in each grade, the decreasing means observed for certain subjects as grade increases may be explainable as a characteristic of the sample rather than a characteristic of the scale.1 Future results from operational administrations with larger samples of highly motivated students taking ACT Aspire over multiple years will provide more robust estimates of longitudinal growth on the ACT Aspire scale.

Table 10.8. English Scale Score Descriptive Statistics Based on Scaling Study by Grade

Grade N Mean SD Min P10 P25 P50 P75 P90 P95 Max

3 4,277 416.50 6.14 403 409 412 416 421 424 428 435 4 3,825 419.65 6.23 402 411 415 420 423 428 430 438 5 4,301 421.98 7.04 403 413 417 421 427 431 435 442 6 4,683 422.56 8.09 400 412 416 423 428 433 437 448 7 4,762 424.84 8.56 400 413 419 425 430 436 439 450 8 3,609 426.38 8.94 401 415 421 426 433 438 442 452 9 2,415 425.48 10.76 400 413 417 424 433 440 445 456 10 2,258 429.07 11.59 400 414 419 429 439 445 447 456 11 2,240 429.15 11.64 400 414 420 429 438 445 448 460

Table 10.9. Mathematics Scale Score Descriptive Statistics Based on Scaling Study by Grade

Grade N Mean SD Min P10 P25 P50 P75 P90 P95 Max

3 4,300 411.75 3.69 400 407 409 412 414 417 418 426 4 3,830 414.75 3.81 403 410 413 415 417 420 421 429 5 4,260 416.78 4.43 404 412 414 417 419 422 424 435 6 4,577 418.15 5.72 402 412 414 418 421 426 429 441 7 4,497 419.83 6.42 402 413 415 419 424 429 431 442 8 3,746 422.48 7.30 403 413 417 422 427 432 436 448 9 2,479 422.82 8.05 406 414 417 422 428 434 438 449 10 2,528 425.15 9.10 406 414 418 424 432 438 441 450 11 1,796 427.34 9.38 407 417 420 426 433 441 445 457

1 Writing scale scores would not be expected to increase across grades due to how it was scaled. See section 10.5 “Scaling ACT Aspire Writing.”

10.28 ACT Aspire Score Scale

Table 10.10. Reading Scale Score Descriptive Statistics Based on Scaling Study by Grade

Grade N Mean SD Min P10 P25 P50 P75 P90 P95 Max 3 4,307 411.56 5.27 401 406 407 411 415 419 422 429 4 3,661 414.02 5.66 401 407 410 414 418 423 424 431 5 4,129 416.20 6.23 401 408 411 416 420 425 427 434 6 4,520 416.90 6.84 402 409 412 416 422 427 428 436 7 4,475 418.75 6.73 402 410 414 419 424 427 429 438 8 3,585 419.76 7.14 401 410 414 420 425 429 431 440 9 2,257 419.71 7.86 403 410 414 419 426 430 433 442 10 2,260 421.30 8.22 403 411 415 421 428 433 434 442 11 1,789 422.43 7.37 402 413 417 422 428 433 435 442

Table 10.11. Science Scale Score Descriptive Statistics Based on Scaling Study by Grade

Grade N Mean SD Min P10 P25 P50 P75 P90 P95 Max 3 4,214 413.89 5.99 401 407 409 414 418 422 424 433 4 3,571 416.29 6.53 400 407 412 416 421 425 427 435 5 3,903 419.05 6.78 401 410 414 420 424 427 430 438 6 4,642 418.76 7.54 400 409 412 419 424 429 431 440 7 4,756 420.49 7.65 401 410 414 421 426 431 432 441 8 3,544 422.14 7.91 401 412 416 422 427 433 435 446 9 2,167 422.65 8.22 402 412 417 421 429 434 437 447 10 2,314 424.43 8.89 402 414 417 424 431 436 439 449 11 1,717 424.59 9.04 400 412 417 425 431 437 439 449

Table 10.12. Writing Scale Score Descriptive Statistics Based on Scaling Study by Grade

Grade N Mean SD Min P10 P25 P50 P75 P90 P95 Max 3 4,109 422.57 6.49 408 414 418 422 428 430 432 440 4 3,439 423.16 7.06 408 416 418 424 428 432 434 440 5 3,935 422.21 7.00 408 412 418 424 426 432 432 440 6 4,512 425.80 8.06 408 416 420 426 432 436 440 448 7 4,666 425.47 6.87 408 416 420 426 430 434 438 448 8 3,522 422.66 6.40 408 416 418 424 426 432 432 448 9 2,117 422.22 7.66 408 410 418 424 428 432 434 448 10 2,258 424.40 8.28 408 412 418 424 430 434 440 448

10.29 ACT Aspire Score Scale

The Lowest Obtainable Scale Scores (LOSS) and the Highest Obtainable Scale Scores (HOSS) of the ACT Aspire scales for all subjects and grades are presented in Table 10.13. Note that English, mathematics, reading, and science tests all have the same minimum scale score of 400 but the maximum scale scores vary across grade level within a subject. The HOSS are different among the four subjects and also across grade levels. HOSS increase as the grade level increase in each subject. All writing tests across grades have the same minimum scale score of 408 but the maximum scale scores are 440 or 448, depending on the grade level. Scaling of the ACT Aspire writing test was conducted differently from the other subjects and is described in the next section.

Table 10.13. Lowest Obtainable Scale Scores (LOSS) and Highest Obtainable Scale Scores (HOSS)

Subject Grade LOSS HOSS Subject Grade LOSS HOSS English 3 400 435 Science 3 400 433 4 400 438 4 400 436 5 400 442 5 400 438 6 400 448 6 400 440 7 400 450 7 400 443 8 400 452 8 400 446 9 400 456 9 400 449 10 400 456 10 400 449 Mathematics 3 400 434 Writing 3 408 440 4 400 440 4 408 440 5 400 446 5 408 440 6 400 451 6 408 448 7 400 453 7 408 448 8 400 456 8 408 448 9 400 460 9 408 448 10 400 460 10 408 448 Reading 3 400 429 4 400 431 5 400 434 6 400 436 7 400 438 8 400 440 9 400 442 10 400 442

10.30 ACT Aspire Score Scale

10.5 Scaling ACT Aspire Writing The ACT Aspire writing scale scores are rubric-driven and are based on four domains (Ideas and Analysis, Development and Support, Organization, and Language Use and Conventions), with each domain scored on a 1 to 5 or a 1 to 6 scale (1 to 5 in grades 3–5, 1 to 6 in grades 6–EHS). The scoring rubrics become more complex as grade increases, however, and a domain score of 4 indicates expected performance at each grade in each rubric domain. The total writing raw score is the sum of the four domain raw scores and ranges from 4 to 20 for grades 3–5 and from 4 to 24 for grades 6– EHS. A common linear function shared by all grades was used to convert the total raw scores to the three-digit ACT Aspire scale scores for the writing base forms. The lowest obtainable scale score for writing is 408 for all grades, and the highest obtainable scale score is 440 for grades 3–5 and 448 for grades 6–EHS. Table 10.13 lists the lowest and highest obtainable scale scores for writing by grade level.

The scoring process for writing implies that writing test scale scores do not have a vertical scale interpretation like those for scale scores obtained in the other subjects. Specifically, writing scale scores across grade levels reflect performance on the rubric-driven expectations; the underlying performance needed to receive the same score is automatically adjusted to grade-level expectations. Consistency in scores across grade levels suggests the same performance relative to grade-level expectations. For example, if a student receives a score of 432 on the writing test in grade 3 and a score of 432 in grade 8, it means that the student has achieved a level of performance that is consistently located relative to grade-level expectations. In other words, the same score in grades 3 and 8 means that a student has grown to meet grade-level expectations. Table 10.12 lists the descriptive statistics for ACT Aspire writing scale scores from the sample of students included in the scaling study.

10.31

CHAPTER 11 ACT Aspire Norms

This chapter describes percentages of students meeting ACT Readiness Benchmarks, sample demographic information, and norm tables based on the norm sample that is used for spring 2014 – spring 2015 ACT Aspire reporting.

ACT Aspire norms are defined as the cumulative percentage of students scoring at or below a given score in the norm sample. The norm sample is a reference sample of students taking the ACT Aspire tests. Norms tables list the cumulative percentage of students who are at or below each scale score point in the norm sample. For example, a cumulative percent of 50 for a scale score of 420 on the mathematics test means that 50% of students in the norm group achieved a scale score of 420 or below on that test.

In 2014, normative information was provided based on student samples from 2013 special study data and 2014 operational data. Starting in fall 2015, ACT Aspire began reporting three-year rolling norms, similar to the ACT. Rolling norms include student samples from the three most recent years of ACT Aspire test administrations.

Prior to fall 2015, ACT Aspire norms were national, with a broad representation across the country, but they were not nationally representative norms, since they had not been statistically weighted to more closely mimic the national distribution of demographics. The norm information starting from fall 2015 was described in the later addendum section. ACT Aspire includes national norms for grades 3 through EHS in English, mathematics, reading, science and writing. Table 11.1 shows the percentage of students who met the benchmark in the 2014 norm sample in each of the five subjects, which are also percentages of students who attain the ‘Ready’ or ‘Exceeding’ performance levels (see Chapter 13). Norm group demographic characteristics, including gender, state, and race/ethnicity, for each grade and subject area are included in Tables 11.2–11.6. The 2014 ACT Aspire norms are listed in Tables 11.7–11.11.

11.1 ACT Aspire Norms

Table 11.1. Percentage of Students Meeting Subject Benchmark in the 2014 Norm Sample

Grade English Mathematics Reading Science Writing 3 71 50 34 29 16

4 69 45 37 35 19

5 68 40 33 37 26

6 68 43 41 38 42

7 71 34 35 33 26

8 73 31 45 34 26

9 60 35 38 30 35

10 63 32 34 31 45

Table 11.2. 2014 ACT Aspire English Norm Group Demographics

Grade (%) 3 4 5 6 7 8 9 10 (n=36,139) (n=36,353) (n=36,890) (n=35,258) (n=35,618) (n=37,796) (n=23,096) (n=19,417) Gender F 45.24 45.29 46.12 44.99 44.49 45.41 44.34 44.26 M 47.21 47.22 46.89 46.11 45.61 45.72 43.31 43.59 No response 7.56 7.48 6.99 8.90 9.90 8.87 12.36 12.15 State AL 66.38 67.74 65.16 61.33 56.29 51.44 7.13 6.44 AR 0.05 0.05 0.04 0.07 0.04 0.06 0.07 0.09 AZ — — — — — 0.15 0.31 0.26 CA 0.28 0.24 2.94 2.99 0.33 0.24 1.26 1.59 CO 2.42 2.28 2.03 1.70 1.62 1.91 2.22 1.98 CT 0.16 0.16 0.20 0.18 0.12 0.19 1.27 1.34 FL 0.06 0.11 — — — 0.10 1.02 0.62 GA 0.08 0.09 0.12 0.10 — — — — IA 0.54 0.58 0.54 0.35 0.52 0.33 0.23 1.12 IL 1.54 1.47 1.60 1.79 1.87 1.66 3.97 4.29 IN 1.01 1.10 1.02 2.22 1.84 0.64 1.13 1.73 KS 1.66 1.64 1.51 2.84 2.54 1.64 2.55 2.44 KY 4.91 4.98 4.03 3.40 3.82 2.66 5.94 2.42 LA 2.17 2.07 1.99 2.05 1.66 1.55 3.16 2.98 MA — — — — — — — 0.09 MI 3.65 2.96 3.59 4.75 8.41 17.28 22.38 26.19 MN 0.32 0.31 0.34 0.48 0.38 0.37 — 0.52 MO 1.84 1.44 1.93 1.65 1.51 1.37 2.93 4.11

Note.11.2 — indicates no students tested. ACT Aspire Norms

Table 11.2. (continued) Grade (%)

3 4 5 6 7 8 9 10 (n=36,139) (n=36,353) (n=36,890) (n=35,258) (n=35,618) (n=37,796) (n=23,096) (n=19,417)

MS 2.27 2.64 2.59 2.77 2.65 2.07 2.55 1.48 ND — — — — — 0.23 0.16 0.31 NE — — — — — — 1.59 1.89 NJ — — — — — — 0.44 — NM — — — — — — 0.57 — NV — — — — — 0.09 0.06 0.07 OH 1.48 1.24 1.14 1.61 3.76 1.74 9.30 5.28 OK 0.70 0.91 0.69 1.14 1.08 1.03 0.14 0.09 PA — — — — — — 1.13 1.18 SC 3.61 3.57 3.63 3.73 3.66 3.63 1.69 1.99 SD — — 0.20 — — 0.21 — 0.25 TN 1.31 1.43 1.42 1.45 1.84 1.95 4.30 4.99 TX 1.96 1.16 1.60 1.32 1.90 2.07 2.32 5.04 UT 0.79 0.82 0.80 0.96 1.34 0.96 0.71 — VA — — — 0.08 — 0.25 — — WI 0.82 1.01 0.89 1.01 2.74 4.11 19.16 18.98 No response — — — 0.01 0.06 0.08 0.31 0.27 Race/Ethnicity Black/African 22.62 22.41 22.43 21.47 20.97 18.37 9.07 8.49 American American Indian/Alaska 1.07 1.03 0.92 1.19 0.82 0.88 0.52 0.64 Native White 44.81 45.24 43.64 41.79 42.25 44.53 35.21 38.01 Hispanic/ 5.47 5.27 5.29 6.67 6.52 5.62 5.19 4.65 Latino Asian 1.68 1.62 1.59 1.28 1.42 1.44 1.40 2.02 Native Hawaiian/ 0.58 0.52 0.91 0.78 0.55 0.40 0.48 0.37 Other Pacific Islander Two or more 0.41 0.32 0.32 0.41 0.61 0.45 0.51 0.78 races No response 23.35 23.59 24.89 26.42 26.85 28.31 47.63 45.04 Note. — indicates no students tested.

11.3 ACT Aspire Norms

Table 11.3. 2014 ACT Aspire Mathematics Norm Group Demographics Grade (%)

3 4 5 6 7 8 9 10 (n=76,861) (n=76,012) (n=77,083) (n=75,651) (n=77,891) (n=80,763) (n=23,011) (n=19,441)

Gender

F 47.15 47.00 47.30 46.91 46.46 47.50 44.60 44.04

M 48.92 49.08 48.96 48.50 48.62 48.19 43.26 43.56

No response 3.93 3.92 3.74 4.59 4.92 4.32 12.14 12.40

State

AL 83.98 84.62 82.65 81.73 80.05 76.23 5.53 4.99

AR 0.02 0.02 0.02 0.03 0.02 0.03 0.07 0.09

AZ — — — — — 0.07 0.30 0.26

CA 0.13 0.11 1.33 1.37 0.11 0.09 1.24 1.62

CO 1.15 1.10 0.98 0.81 0.74 0.89 2.23 2.06

CT 0.06 0.09 0.09 0.02 0.06 0.08 1.27 1.33

FL 0.03 0.05 — — — 0.04 0.98 0.59

GA 0.04 0.04 0.06 0.05 — — — —

IA 0.26 0.28 0.26 0.16 0.24 0.15 0.24 1.15

ID — — — — — 0.09 — —

IL 0.74 0.70 0.76 0.85 0.86 0.79 4.05 3.93

IN 0.52 0.52 0.49 1.06 0.82 0.37 1.02 1.91

KS 0.81 0.77 0.73 1.20 1.00 0.75 2.88 2.74

KY 2.25 2.38 1.93 1.52 1.72 1.28 5.59 2.41

LA 1.03 0.99 0.95 0.95 0.76 0.73 3.09 2.95

MA — — — — — — — 0.09

MI 1.77 1.44 1.88 2.79 4.10 8.67 22.38 26.31

MN 0.15 0.15 0.16 0.29 0.01 0.17 — 0.53

MO 0.85 0.69 0.91 0.72 0.72 0.64 2.89 3.37

MS 1.11 1.26 1.24 1.32 1.09 0.98 1.79 1.50

ND — — — — — 0.11 0.17 0.31

NE — — — — — — 1.60 1.89

NJ — — — — — — 0.44 —

NM — — — — — — 0.59 —

NV 0.01 0.01 0.01 — — 0.04 0.06 0.07

Note. — indicates no students tested.

11.4 ACT Aspire Norms

Table 11.3. (continued) Grade (%)

3 4 5 6 7 8 9 10 (n=76,861) (n=76,012) (n=77,083) (n=75,651) (n=77,891) (n=80,763) (n=23,011) (n=19,441)

OH 0.69 0.56 0.54 0.75 1.65 0.94 9.51 5.92

OK 0.43 0.43 0.33 0.54 0.54 0.50 0.17 0.09

PA — — — — — — 1.00 1.18

SC 1.69 1.71 1.73 1.74 1.66 1.68 2.05 1.99

SD — — 0.09 — — 0.10 — 0.25

TN 0.62 0.67 0.68 0.68 0.84 0.88 4.24 4.98

TX 0.90 0.57 1.39 0.56 1.07 1.04 2.63 5.12

UT 0.37 0.38 0.36 0.35 0.66 0.46 0.72 —

VA — — — 0.04 — 0.12 — —

WI 0.40 0.45 0.42 0.47 1.25 2.04 20.98 20.11

No response — — — 0.00 0.03 0.04 0.30 0.27

Race/ Ethnicity

Black/African 26.63 27.08 27.02 27.63 27.71 26.77 8.14 8.29 American

American Indian/Alaska 0.84 0.84 0.83 0.93 0.78 0.80 0.65 0.67 Native

White 50.40 50.64 50.00 49.05 49.75 50.61 35.77 36.98

Hispanic/ 6.17 5.96 5.70 5.81 5.64 5.12 5.34 4.92 Latino

Asian 1.48 1.50 1.48 1.25 1.31 1.35 1.40 2.01

Native Hawaiian/ 0.31 0.28 0.47 0.36 0.28 0.22 0.56 0.38 Other Pacific Islander

Two or more 0.21 0.16 0.16 0.18 0.32 0.22 0.53 0.78 races

No response 13.96 13.54 14.34 14.79 14.21 14.91 47.63 45.97

Note. — indicates no students tested.

11.5 ACT Aspire Norms

Table 11.4. 2014 ACT Aspire Reading Norm Group Demographics Grade (%)

3 4 5 6 7 8 9 10 (n=76,817) (n=75,577) (n=76,316) (n=74,862) (n=77,497) (n=80,077) (n=22,711) (n=19,060)

Gender

F 47.17 47.10 47.34 46.93 46.59 47.48 44.57 43.92

M 49.05 49.12 49.06 48.56 48.84 48.20 43.15 43.48

No response 3.78 3.78 3.60 4.51 4.56 4.33 12.28 12.60

State

AL 84.13 85.03 83.56 82.45 80.36 76.93 5.70 5.70

AR 0.02 0.02 0.02 0.03 0.02 0.03 0.07 0.09

AZ — — — — — 0.07 0.32 0.27

CA 0.12 0.10 1.43 1.39 0.19 0.05 1.23 1.58

CO 1.15 1.10 0.98 0.81 0.75 0.91 2.23 2.03

CT 0.04 0.06 0.09 0.09 0.02 0.10 1.16 1.36

FL 0.03 0.05 — — — 0.05 1.06 0.63

GA 0.04 0.04 0.06 0.05 — — — —

IA 0.26 0.28 0.26 0.17 0.24 0.15 0.24 1.17

ID — — — — — 0.09 — —

IL 0.73 0.70 0.77 0.85 0.83 0.79 4.12 4.47

IN 0.49 0.41 0.42 1.02 0.82 0.30 0.78 1.21

KS 0.77 0.74 0.67 1.18 1.00 0.77 2.29 2.46

KY 2.24 2.39 1.95 1.50 1.71 1.03 5.14 1.41

LA 1.03 1.00 0.96 0.94 0.76 0.73 3.05 2.90

MA — — — — — — — 0.09

MI 1.83 1.48 1.85 2.26 3.88 8.46 22.80 26.78

MN 0.15 0.15 0.17 0.26 0.04 0.18 — 0.52

MO 0.85 0.70 0.89 0.75 0.71 0.55 2.90 3.81

MS 1.23 1.27 1.23 1.34 1.09 0.94 2.49 1.45

ND — — — — — 0.11 0.17 0.31

NE — — — — — — 1.62 1.92

NJ — — — — — — 0.43 —

NM — — — — — — 0.61 —

Note. — indicates no students tested.

11.6 ACT Aspire Norms

Table 11.4. (continued) Grade (%)

3 4 5 6 7 8 9 10 (n=76,817) (n=75,577) (n=76,316) (n=74,862) (n=77,497) (n=80,077) (n=22,711) (n=19,060)

NV 0.01 0.01 0.01 — — 0.04 0.06 0.07

OH 0.69 0.59 0.55 0.65 1.75 0.93 9.70 5.67

OK 0.42 0.38 0.27 0.53 0.54 0.48 0.13 0.09

PA — — — — — — 0.96 1.23

SC 1.69 1.71 1.76 1.76 1.67 1.72 2.09 2.00

SD — — 0.09 — — 0.10 — 0.27

TN 0.63 0.68 0.69 0.69 0.85 0.91 4.38 4.99

TX 0.92 0.56 0.75 0.62 0.85 0.94 2.27 5.24

UT 0.17 0.14 0.13 0.22 0.62 0.45 0.70 —

VA — — — 0.03 — 0.12 — —

WI 0.39 0.41 0.43 0.40 1.26 2.04 20.99 19.99

No response — — — 0.00 0.03 0.04 0.32 0.27

Race/ Ethnicity

Black/African 26.87 27.18 27.22 27.82 27.90 26.92 8.44 7.96 American

American Indian/Alaska 0.82 0.83 0.82 0.92 0.77 0.80 0.66 0.71 Native

White 50.30 50.77 50.01 49.17 49.74 50.54 35.08 36.85

Hispanic/ 6.14 5.86 5.56 5.79 5.63 5.00 5.03 4.71 Latino

Asian 1.45 1.48 1.42 1.24 1.34 1.30 1.36 2.00

Native Hawaiian/ 0.26 0.23 0.41 0.31 0.22 0.22 0.53 0.38 Other Pacific Islander

Two or more 0.21 0.15 0.15 0.18 0.33 0.18 0.55 0.80 races

No response 13.94 13.50 14.41 14.57 14.09 15.04 48.35 46.59

Note. — indicates no students tested.

11.7 ACT Aspire Norms

Table 11.5. 2014 ACT Aspire Science Norm Group Demographics Grade (%)

3 4 5 6 7 8 9 10 (n=33,076) (n=34,135) (n=33,317) (n=31,928) (n=32,618) (n=35,162) (n=22,238) (n=18,616)

Gender

F 45.24 44.98 45.81 44.92 44.15 45.35 44.29 44.08

M 46.72 47.18 46.59 45.72 45.37 45.31 43.05 43.60

No response 8.04 7.84 7.59 9.36 10.48 9.34 12.66 12.32

State

AL 63.30 66.95 62.04 57.38 52.50 49.04 5.81 5.23

AR 0.05 0.05 0.05 0.07 0.05 0.06 0.07 0.09

AZ — — — — — 0.16 0.31 0.27

CA 0.24 0.24 3.16 3.19 0.29 0.18 0.36 1.21

CO 2.68 2.31 2.13 1.78 1.64 2.00 2.31 2.19

CT 0.15 0.19 0.20 0.19 0.13 0.17 1.17 1.39

FL 0.07 0.12 — — — 0.11 0.99 0.61

GA 0.08 0.09 0.14 0.12 — — — —

IA 0.60 0.62 0.61 0.39 0.46 0.26 0.24 1.20

IL 1.76 1.58 1.72 1.90 2.06 1.81 3.98 4.26

IN 0.93 0.91 0.80 2.27 1.91 0.71 0.88 1.62

KS 1.86 1.71 1.67 3.16 2.80 1.76 2.32 2.35

KY 5.31 5.59 4.78 3.45 4.06 2.92 5.27 1.44

LA 2.36 2.20 2.19 2.22 1.74 1.67 3.11 3.01

MA — — — — — — — 0.09

MI 3.99 2.74 4.35 6.12 9.31 17.46 22.21 27.14

MN 0.34 0.32 0.37 1.02 0.42 0.39 — 0.56

MO 1.85 1.50 1.91 1.84 1.67 1.46 2.76 3.24

MS 2.84 2.55 2.58 3.11 2.45 2.10 2.62 1.63

ND — — — — — 0.24 0.17 0.32

NE — — — — — — 1.65 1.97

NJ — — — — — — 0.44 —

NM — — — — — — 0.60 —

NV — — — — — 0.10 0.06 0.07

Note. — indicates no students tested.

11.8 ACT Aspire Norms

Table 11.5. (continued) Grade (%)

3 4 5 6 7 8 9 10 (n=33,076) (n=34,135) (n=33,317) (n=31,928) (n=32,618) (n=35,162) (n=22,238) (n=18,616)

OH 1.68 1.27 1.32 1.51 4.43 2.07 9.92 6.53

OK 0.92 0.94 0.77 1.25 1.28 1.10 0.13 0.10

PA — — — — — — 0.98 1.26

SC 3.91 3.79 4.00 4.12 3.98 3.89 1.81 2.05

SD — — 0.22 — — 0.23 — 0.25

TN 1.42 1.51 1.56 1.61 2.18 2.06 4.46 5.12

TX 2.12 1.27 1.78 1.37 2.07 2.44 2.77 4.79

UT 0.68 0.63 0.65 0.86 1.54 1.03 0.71 —

VA — — — 0.09 — 0.27 — —

WI 0.89 0.90 1.00 0.98 2.96 4.22 21.55 19.72

No response — — — 0.01 0.07 0.09 0.32 0.28

Race/ Ethnicity

Black/African 21.71 22.31 23.33 21.86 22.99 18.78 8.16 7.71 American

American Indian/Alaska 0.98 0.86 0.61 1.04 0.47 0.78 0.65 0.70 Native

White 43.12 44.11 40.92 40.85 38.62 43.42 35.12 37.13

Hispanic/ 6.35 5.72 5.35 6.79 6.95 5.72 4.99 4.40 Latino

Asian 1.57 1.39 1.34 1.23 1.23 1.32 1.34 2.01

Native Hawaiian/ 0.64 0.54 1.00 0.78 0.58 0.45 0.44 0.40 Other Pacific Islander

Two or more 0.47 0.35 0.34 0.43 0.77 0.49 0.54 0.79 races

No response 25.18 24.70 27.12 27.01 28.38 29.04 48.75 46.85

Note: — indicates no students tested.

11.9 ACT Aspire Norms

Table 11.6. 2014 ACT Aspire Writing Norm Group Demographics Grade (%)

3 4 5 6 7 8 9 10 (n=25,301) (n=26,867) (n=27,071) (n=29,045) (n=30,935) (n=32,984) (n=15,361) (n=13,144)

Gender

F 47.21 45.59 46.30 46.83 46.20 46.68 43.84 42.73

M 46.01 47.74 47.12 47.15 47.07 46.36 41.22 41.68

No response 6.78 6.67 6.58 6.01 6.73 6.96 14.94 15.59

State

AL 61.82 66.06 60.88 61.58 56.81 55.33 — —

AR 0.07 0.07 0.06 0.08 0.05 0.06 0.10 0.13

AZ — — — — — 0.17 0.47 0.39

CA 0.39 0.30 4.14 3.55 0.37 0.18 0.44 1.34

CO 3.36 3.10 2.79 2.06 1.87 2.18 3.19 2.91

CT 0.20 0.08 0.10 0.05 0.10 0.06 1.74 1.45

FL 0.07 0.15 — — — 0.11 1.38 0.73

GA 0.10 0.11 0.17 0.12 — — — —

IA 0.13 0.12 0.13 0.12 0.34 0.09 0.17 1.12

IL 2.09 1.88 2.07 2.00 1.95 1.81 5.06 5.25

IN 0.83 0.49 0.75 2.20 1.92 0.79 1.35 2.58

KS 2.21 2.13 1.94 3.39 2.83 1.72 2.87 3.38

KY 6.40 6.32 5.10 3.43 3.81 2.62 5.81 0.86

LA 2.81 2.64 2.56 2.25 1.74 1.29 3.71 3.25

MA — — — — — — — 0.13

MI 1.78 1.24 2.75 3.47 7.98 14.88 27.43 30.48

MN 0.44 0.41 0.46 0.63 0.44 0.42 — 0.68

MO 2.49 1.73 2.56 1.93 1.80 1.59 3.54 4.50

MS 2.99 2.77 2.12 2.52 2.09 1.76 1.87 1.38

ND — — — — — 0.26 — 0.12

NE — — — — — — 0.10 0.17

NJ — — — — — — 0.64 —

NM — — — — — — 0.77 —

NV — — — — — — 0.09 0.10

OH 1.91 1.54 1.39 1.55 3.98 1.73 9.35 6.77

Note. — indicates no students tested.

11.10 ACT Aspire Norms

Table 11.6. (continued) Grade (%)

3 4 5 6 7 8 9 10 (n=25,301) (n=26,867) (n=27,071) (n=29,045) (n=30,935) (n=32,984) (n=15,361) (n=13,144)

OK 0.98 1.08 0.94 1.34 1.27 1.14 0.19 0.14

PA — — — — — — 1.41 1.73

SC 4.88 4.78 4.92 4.50 4.15 4.15 2.67 2.87

SD — — 0.26 — — 0.25 — —

TN 0.53 0.52 0.64 0.67 0.61 0.67 0.87 1.71

TX 2.57 1.53 2.22 1.39 2.16 2.25 3.32 7.44

UT 0.37 0.38 0.37 0.49 1.10 0.48 — —

VA — — — 0.10 — 0.29 — —

WI 0.57 0.57 0.69 0.56 2.61 3.62 21.00 18.00

No response — — — 0.01 — 0.09 0.45 0.40

Race/ Ethnicity

Black/African 24.68 24.97 23.11 25.31 23.19 22.37 6.61 6.48 American

American Indian/Alaska 1.00 0.89 0.99 1.05 1.16 0.95 0.60 0.70 Native

White 43.80 45.24 43.80 42.16 43.24 46.71 31.91 34.24

Hispanic/ 6.62 6.35 5.94 7.56 7.45 6.16 4.95 5.58 Latino

Asian 1.36 1.22 1.16 1.05 1.17 1.13 0.92 1.64

Native Hawaiian/ 0.25 0.19 0.68 0.46 0.23 0.11 0.05 0.11 Other Pacific Islander

Two or more 0.42 0.33 0.33 0.39 0.77 0.48 0.59 1.01 races

No response 21.88 20.82 23.99 22.02 22.80 22.07 54.37 50.23

Note. — indicates no students tested.

11.11 ACT Aspire Norms

Table 11.7. 2014 ACT Aspire English Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 2 1 1 1 1 1 1 1 406 3 1 1 1 2 1 1 1 407 5 2 1 2 2 1 2 1 408 8 2 2 2 4 2 2 2 409 12 4 2 4 4 3 3 2 410 17 6 4 5 6 3 4 3 411 24 9 5 6 7 4 5 4 412 29 13 7 8 8 6 7 5 413 36 16 10 10 10 6 8 6 414 43 21 15 13 12 8 10 7 415 50 27 18 15 14 10 13 9 416 56 31 23 18 17 11 15 10 417 63 38 27 22 20 14 16 12 418 66 44 32 26 21 17 19 14 419 72 52 37 32 26 21 22 16 420 77 55 44 35 29 22 25 19 421 81 61 48 42 33 27 28 21 422 83 69 52 46 38 31 30 22 423 88 72 60 50 43 36 33 25 424 91 77 65 56 44 39 37 28 425 93 84 68 60 50 42 40 31 426 95 85 72 66 55 47 43 33 427 95 90 79 68 60 53 47 37 428 97 92 80 73 62 55 50 40 429 97 95 86 77 67 61 54 43 430 99 96 86 81 73 67 57 47 431 99 98 91 84 75 67 61 50 432 99 98 91 87 78 73 65 53 433 99 99 95 89 83 76 68 57 434 99 99 95 92 85 78 71 60 435 100 99 97 93 87 83 74 63 436 99 98 95 91 86 77 67 437 99 98 96 92 88 80 70 438 100 99 97 94 90 82 72 11.12 ACT Aspire Norms

Table 11.7. (continued)

Grade Scale Score 3 4 5 6 7 8 9 10 439 99 98 96 92 84 75 440 99 98 97 94 87 79 441 99 99 98 95 89 81 442 100 99 99 97 91 84 443 99 99 97 93 87 444 99 99 99 94 90 445 99 99 99 96 92 446 99 99 99 97 94 447 99 99 99 98 95 448 100 99 99 99 96 449 99 99 99 98 450 100 99 99 98 451 99 99 99 452 100 99 99 453 99 99 454 99 99 455 99 99 456 100 100

11.13 ACT Aspire Norms

Table 11.8. 2014 ACT Aspire Mathematics Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 3 1 1 1 1 1 1 1 405 5 1 1 1 1 1 1 1 406 8 1 1 1 2 1 1 1 407 11 3 1 2 3 1 1 1 408 16 5 3 2 5 2 1 1 409 22 6 5 3 5 3 2 2 410 32 10 8 5 8 5 2 2 411 39 17 10 8 12 7 4 4 412 50 23 16 11 13 10 6 6 413 58 34 21 16 19 13 8 6 414 69 45 31 22 25 17 11 9 415 76 55 40 29 32 24 14 12 416 84 65 50 35 39 29 18 15 417 89 74 60 42 43 34 23 19 418 94 80 64 50 50 41 28 23 419 96 85 70 57 56 46 32 26 420 98 90 76 63 61 50 37 30 421 99 93 82 70 66 55 39 32 422 99 95 87 75 71 61 44 36 423 99 96 88 79 75 65 49 40 424 99 98 91 83 79 69 53 44 425 99 98 94 86 82 72 58 48 426 99 99 95 89 86 75 61 51 427 99 99 97 91 89 79 65 54 428 99 99 98 94 91 82 69 58 429 100 99 98 95 93 84 72 61 430 100 99 99 96 94 87 75 65 431 100 99 99 97 96 88 79 68 432 100 99 99 98 97 90 82 72 433 100 99 99 99 97 92 84 75 434 100 100 99 99 98 94 88 79 435 100 99 99 98 95 90 82 436 100 99 99 99 96 92 85 437 100 99 99 99 97 93 87 438 100 99 99 99 98 95 89 11.14 ACT Aspire Norms

Table 11.8. (continued)

Grade Scale Score 3 4 5 6 7 8 9 10 439 100 99 99 99 98 96 91 440 100 99 99 99 99 97 93 441 99 99 99 99 98 95 442 99 99 99 99 98 96 443 99 99 99 99 99 97 444 99 99 99 99 99 98 445 100 99 99 99 99 99 446 100 99 99 99 99 99 447 99 99 99 99 99 448 99 100 99 99 99 449 99 100 99 99 99 450 100 100 99 99 99 451 100 100 99 99 99 452 100 100 99 99 453 100 100 99 99 454 100 99 99 455 100 99 99 456 100 99 99 457 99 100 458 99 100 459 100 100 460 100 100

11.15 ACT Aspire Norms

Table 11.9. 2014 ACT Aspire Reading Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 5 1 1 1 1 1 1 1 405 8 3 2 1 1 1 1 1 406 14 6 3 2 1 1 1 1 407 23 10 5 4 3 2 2 2 408 28 14 7 6 5 2 4 3 409 36 20 10 9 5 4 4 4 410 43 25 16 13 9 5 8 6 411 48 31 21 16 13 8 11 10 412 54 36 26 18 16 10 15 13 413 61 45 31 22 21 13 16 14 414 66 51 35 26 25 17 21 17 415 72 57 40 31 31 20 26 21 416 77 63 49 35 33 23 30 24 417 82 68 55 40 38 27 34 28 418 87 75 61 46 44 32 39 31 419 91 80 67 50 48 36 43 35 420 94 84 73 59 53 41 47 39 421 95 86 78 61 59 47 52 43 422 96 90 80 67 65 49 57 47 423 98 93 84 73 71 55 61 51 424 99 96 86 78 76 58 62 52 425 99 97 91 83 81 64 66 57 426 99 98 94 87 86 70 71 62 427 99 99 94 89 90 76 75 66 428 99 99 97 93 93 82 80 72 429 100 99 98 96 96 87 84 77 430 99 99 96 96 89 89 82 431 100 99 98 98 91 89 83 432 99 99 99 95 92 87 433 99 99 99 97 95 92 434 100 99 99 98 97 95 435 99 99 99 99 98 436 100 99 99 99 98 437 99 99 99 99 438 100 99 99 99 11.16 ACT Aspire Norms

Table 11.9. (continued)

Grade Scale Score 3 4 5 6 7 8 9 10 439 99 99 99 440 100 99 99 441 99 99 442 100 100

11.17 ACT Aspire Norms

Table 11.10. 2014 ACT Aspire Science Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 4 2 1 1 1 1 1 1 405 6 3 2 2 1 1 1 1 406 10 6 3 2 2 1 1 1 407 15 8 4 4 3 1 1 1 408 22 12 6 5 5 2 2 2 409 28 15 8 8 7 3 2 2 410 34 18 11 10 10 5 4 4 411 39 22 14 13 14 7 4 4 412 46 26 18 17 18 9 7 6 413 51 31 22 22 21 12 7 7 414 58 37 26 25 26 15 11 9 415 62 42 30 29 28 19 16 14 416 66 47 34 34 32 23 17 15 417 71 54 39 38 38 27 22 19 418 76 60 45 42 42 28 24 21 419 81 65 51 47 46 33 29 25 420 84 70 57 51 50 37 35 29 421 88 75 63 57 56 42 40 34 422 91 79 68 62 60 46 45 38 423 94 84 74 67 64 51 46 39 424 95 87 78 72 67 57 50 42 425 97 90 83 77 72 62 54 46 426 98 93 89 81 76 66 59 50 427 99 95 92 85 80 70 63 53 428 99 97 94 89 83 75 67 57 429 99 98 96 91 86 79 70 61 430 99 99 97 93 88 82 76 66 431 99 99 98 95 91 85 79 69 432 99 99 99 96 94 88 82 73 433 100 99 99 98 96 90 84 76 434 99 99 99 98 92 87 79 435 99 99 99 99 95 91 83 436 100 99 99 99 96 92 86 437 99 99 99 97 94 88 438 100 99 99 98 96 91 11.18 ACT Aspire Norms

Table 11.10. (continued)

Grade Scale Score 3 4 5 6 7 8 9 10 439 99 99 99 97 94 440 100 99 99 98 95 441 99 99 99 97 442 99 99 99 98 443 100 99 99 99 444 99 99 99 445 99 99 99 446 100 99 99 447 99 99 448 99 99 449 100 100

11.19 ACT Aspire Norms

Table 11.11. 2014 ACT Aspire Writing Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 408 1 1 2 1 1 2 1 1 409 1 1 2 2 1 2 1 1 410 4 4 5 2 5 4 5 4 411 4 4 5 2 5 5 5 4 412 7 5 7 3 9 7 7 6 413 7 5 7 3 9 7 7 6 414 11 7 8 7 11 9 9 7 415 11 7 8 7 12 9 9 7 416 20 14 18 14 19 19 13 10 417 20 14 18 15 19 19 13 10 418 33 27 27 18 29 31 26 20 419 33 29 27 19 30 31 26 20 420 51 40 35 22 40 39 31 24 421 51 40 35 22 41 39 31 24 422 63 51 40 35 49 44 35 27 423 63 51 40 42 49 44 35 27 424 74 65 58 51 65 61 47 37 425 75 65 58 51 65 61 47 37 426 83 80 73 58 73 74 65 55 427 84 81 74 58 74 74 65 55 428 89 87 78 61 81 81 71 61 429 89 87 79 61 82 81 71 61 430 94 92 82 64 86 84 76 66 431 94 92 82 80 87 84 76 66 432 97 97 93 87 95 95 88 81 433 97 97 93 89 95 95 88 81 434 98 98 96 91 96 97 95 90 435 98 98 96 91 96 97 95 90 436 99 99 97 92 98 98 96 92 437 99 99 97 93 98 98 96 92 438 99 99 98 94 98 98 97 94 439 99 99 98 94 98 98 97 94 440 100 100 100 99 99 99 99 98 441 99 99 99 99 98 442 99 99 99 99 99 443 99 99 99 99 99 444 99 99 99 99 99 445 99 99 99 99 99

11.20 ACT Aspire Norms

Table 11.11. (continued)

Grade Scale Score 3 4 5 6 7 8 9 10 446 99 99 99 99 99 447 99 99 99 99 99 448 100 100 100 100 100

11.21

CHAPTER 12 EPAS® to ACT Aspire Concordance

This chapter presents a concordance study between the ACT Educational Planning and Assessment System (EPAS®) 1–36 scale, which consists of the ACT (grades 11–12) and two legacy tests, ACT Explore® (grades 8–9) and ACT Plan® (grade 10), and the three-digit ACT Aspire scale. The concordance study established a direct link between scores on EPAS and ACT Aspire. This link was used to facilitate a smooth transition to ACT Aspire for users of ACT Explore and ACT Plan, to establish the ACT Readiness Benchmarks for grades 3–11 (described in Chapter 13), and to establish the Progress toward Career Readiness indicator (Chapter 15).

EPAS is an integrated series of paper-administered, curriculum-based tests of educational development with selected-response items typically taken by students from grade 8 through high school. ACT Aspire, on the other hand, is offered both on paper and online; includes selected-response, constructed-response, and technology- enhanced item types; and is a vertically-articulated, benchmarked, and standards- based system of assessments that can be taken by students from grades 3 through early high school. The grade 8 through early high school tests in EPAS and ACT Aspire are intended to measure similar constructs but differ in test specifications, which is a circumstance where concordances are an applicable type of scale alignment (Holland & Dorans, 2006).

12.1 EPAS® to ACT Aspire Concordance

12.1 Method The concordance analysis linked relating scores from ACT Explore, ACT Plan, and the ACT to ACT Aspire using percentile ranks, where concorded scores are defined as those having the same percentage of students at or below a given score with respect to the group of students used in the study. The collection of concorded scores is referred to as a concordance table, which is useful for determining the cut scores on one test that result in approximately the same proportion of students identified by the other test although not necessarily the same students.

The students in the concordance analysis were from the spring 2013 special studies in which participating students took ACT Aspire English, mathematics, reading, or science tests plus historical ACT Explore, ACT Plan, or the ACT assessments from the 2012–2013 academic year. Note that grade 11 students took the ACT plus a separately timed section of constructed-response items designed as an add-on to the ACT (see Appendix B). The constructed-response portion was combined with an operationally administered form of the ACT (selected response only) to obtain scores on the ACT Aspire scale. This version of the ACT is referred to as the ACT with constructed-response tests. The final analysis sample included students from grade 7 and above who took one of the EPAS tests (ACT Explore, ACT Plan, or the ACT) and the ACT Aspire test in the same subject and grade level (which included the ACT with constructed response test for grade 11 students).

Since both EPAS and ACT Aspire are vertically scaled assessments covering different grade spans, the concordance was established to link between two scales (i.e., the 1–36 EPAS scale to the ACT Aspire 400+ scale), rather than between two tests (e.g., between ACT Explore and ACT Aspire grade 8 test).

The concordance relationship between EPAS and ACT Aspire scales was estimated using the equipercentile method described by Kolen and Brennan (2014). This method uses the percentile ranks (i.e., the proportion of students at or below each score) to define the relationship between the two scales. For a particular EPAS score, the corresponding ACT Aspire score is the one that has the same percentile rank.

Evaluation of Results The derived concordance table was applied to the existing datasets with students’ ACT Explore and ACT Plan scores. The purpose of the evaluation was to examine distributions of the concorded ACT Aspire scale scores that should be similar to those of the original EPAS scale scores. Two evaluation samples were used. One sample was composed of cross-sectional data in the academic year of 2013–2014 with the cohort of grade 8 students who took ACT Explore and the cohort of grade 10 students who took ACT Plan. The other sample was a longitudinal data set including the cohort of grade 8 students in the academic year of 2010–2011 who took ACT Explore in that year and took ACT Plan two years later when they were in grade 10. The following properties were examined: 12.2 EPAS® to ACT Aspire Concordance

1. Using the cross-sectional data, compare distributions of ACT Explore/ACT Plan scores against the distributions of concorded ACT Aspire scale scores. The purpose of this analysis was to verify that ACT Explore/ACT Plan and concorded ACT Aspire score distributions did not appear to differ. 2. Using the longitudinal data, create box plots of ACT Plan scores conditional on each ACT Explore score point, and create the same set of plots based on the concorded ACT Aspire scale scores. The purpose of this analysis was to examine whether the relationship between grade 8 and grade 10 scores of the same cohort stayed the same when the concorded ACT Aspire scores were used. 3. Using the longitudinal data, compute effect sizes from grade 8 to grade 10, based on the EPAS scale score and the concorded ACT Aspire scale scores. The effect size was computed as the scale score difference between the two grades divided by the square root of the average variances. The purpose of this analysis was to examine whether the magnitude of growth measured by effect size was similar when the concordance was applied.

12.2 Results Table 12.1 presents the n-counts of the analysis sample, the correlation between the EPAS and ACT Aspire scale scores, and descriptive statistics of scores on both scales. As shown, a sample of over 16,000 students who had both EPAS and ACT Aspire scores was used to conduct the concordance analysis for four subject areas: English, mathematics, reading and science. Correlations between the two scale scores are higher in English and mathematics than in reading and science. Table 12.2 displays the n-counts and percentages of students by gender, race/ethnicity, and geographic region for the sample used in the concordance analysis. The scatter plots between the two sets of scores, shown in Figure 12.1, indicate a linear relationship.

Table 12.1. Descriptive Statistics of the Sample for Concordance Analysis

Correlation EPAS ACT Aspire between EPAS and Subject N ACT Aspire Mean SD Min Max Mean SD Min Max

English 18,204 .81 16.04 4.87 1 36 427.68 10.24 400 460 Mathematics 17,787 .80 16.50 4.37 1 36 423.44 8.62 401 457 Reading 16,922 .70 16.02 4.72 1 36 420.74 7.57 401 446 Science 17,102 .74 17.62 3.87 1 36 423.06 8.32 400 449

12.3 EPAS® to ACT Aspire Concordance

Table 12.2. Demographic Information of the Sample for Concordance Analysis

Subject (N) Demographic Grouping English Mathematics Reading Science % (18,204) (17,787) (16,922) (17,102) Gender Not Specified 19 16 16 17 Female 40 41 41 41 Male 41 42 42 42 Race/Ethnicity Not Specified 40 38 39 39 Black/African American 19 20 20 19 American Indian/Alaskan Native 1 1 1 1 White 34 35 34 35 Hispanic/Latino 4 4 4 4 Asian 1 1 1 1 Native Hawaiian/Other Pacific 1 1 1 1 Islander Two or More Races 1 1 1 1 Geographic Region East 56 55 57 57 Midwest 34 35 34 34 Southwest 6 5 5 5 West 4 4 3 4

12.4 EPAS® to ACT Aspire Concordance

Figure 12.1. Scatter plots of ACT Aspire scale scores versus EPAS scale scores

English

Mathematics

12.5 EPAS® to ACT Aspire Concordance

Figure 12.1. (continued)

Reading

Science

12.6 EPAS® to ACT Aspire Concordance

Evaluating Results Descriptive statistics of ACT Explore and the concorded ACT Aspire scores in the cross-sectional evaluation sample are shown in Table 12.3, and the scores for ACT Plan are shown in Table 12.4. Figure 12.2 presents the distributions of ACT Explore and concorded ACT Aspire scores, and Figure 12.3 presents similar information for ACT Plan scores. It can be seen that the concorded score distributions match those of the original scores.

For each pair of panels in Figure 12.4, the top panel presents the box plots of ACT Plan scores conditional on ACT Explore scores using the longitudinal evaluation sample. The bottom panel presents the plots using the concorded ACT Aspire score. As shown, the relationship between students’ grade 10 and grade 8 performance is highly similar when the concordance was applied.

Table 12.3. Descriptive Statistics of ACT Explore and Concorded ACT Aspire Scores in Cross-Sectional Evaluation Sample

ACT Explore Concorded ACT Aspire Subject N Mean SD Min Max Mean SD Min Max

English 1,034,067 14.40 4.37 1 25 424.17 9.64 400 445 Math 1,033,788 15.40 3.73 1 25 421.33 7.71 400 439 Reading 1,032,226 14.56 3.99 1 25 418.23 7.05 400 434 Science 1,030,623 16.54 3.33 1 25 420.78 7.51 400 438

Table 12.4. Descriptive Statistics of ACT Plan and Concorded ACT Aspire Scores in Cross-Sectional Evaluation Sample

ACT Plan Concorded ACT Aspire Subject N Mean SD Min Max Mean SD Min Max

English 1,251,757 16.94 4.72 1 32 429.59 9.72 400 453 Math 1,251,809 17.74 4.85 1 32 425.83 9.25 400 452 Reading 1,250,597 17.15 4.66 1 30 422.66 7.21 400 437 Science 1,250,022 18.58 4.13 1 32 425.01 8.53 400 447

12.7 EPAS® to ACT Aspire Concordance

Figure 12.2. Distributions of ACT Explore scale scores and concorded ACT Aspire scale scores

12.8 EPAS® to ACT Aspire Concordance

Figure 12.2. (continued)

12.9 EPAS® to ACT Aspire Concordance

Figure 12.3. Distributions of ACT Plan scale scores and concorded ACT Aspire scale scores

12.10 EPAS® to ACT Aspire Concordance

Figure 12.3. (continued)

12.11 EPAS® to ACT Aspire Concordance

Figure 12.4. Box plots of ACT Plan Scores (or concorded ACT Aspire scale scores) conditional on ACT Explore Scores (or concorded ACT Aspire Scores)

English—EPAS Scale

English—Concorded ACT Aspire Scale

12.12 EPAS® to ACT Aspire Concordance

Figure 12.4. (continued)

Mathematics—EPAS Scale

Mathematics—Concorded ACT Aspire Scale

12.13 EPAS® to ACT Aspire Concordance

Figure 12.4. (continued)

Reading—EPAS Scale

Reading—Concorded ACT Aspire Scale

12.14 EPAS® to ACT Aspire Concordance

Figure 12.4. (continued)

Science—EPAS Scale

Science—ACT Aspire Concorded Scale

12.15 EPAS® to ACT Aspire Concordance

Descriptive statistics of the ACT Explore and ACT Plan scale scores for the longitudinal sample are presented in Table 12.5. The corresponding concorded ACT Aspire scores are presented in Table 12.6. Effect sizes, computed from both scale scores shown in the last column for each table, are very similar between the EPAS scale and the concorded ACT Aspire scale.

Table 12.5. Descriptive Statistics of ACT Explore Grade 8 and ACT Plan Grade 10 Scale Scores in Longitudinal Data

Grade 8—ACT Explore Grade 10—ACT Plan Effect Subject N Mean SD Min Max Mean SD Min Max Size

English 572,302 14.66 4.17 1 25 17.10 4.42 1 32 .57 Mathematics 572,051 15.63 3.46 1 25 17.84 4.57 1 32 .55 Reading 571,328 14.80 3.91 1 25 17.19 4.48 1 30 .57 Science 570,063 16.80 3.31 1 25 18.52 3.77 1 32 .48

Table 12.6. Descriptive Statistics of Concorded ACT Aspire Scale Scores in Longitudinal Data

Grade 8—Concorded Grade 10—Concorded ACT Aspire ACT Aspire Effect Subject N Mean SD Min Max Mean SD Min Max Size

English 572,302 424.79 9.31 400 445 430.05 9.08 400 453 .57 Mathematics 572,051 421.85 7.24 400 439 426.12 8.76 400 452 .53 Reading 571,328 418.73 6.93 400 434 422.69 6.96 400 437 .57 Science 570,063 421.39 7.54 400 438 425.10 8.04 400 447 .48

The final derived EPAS to ACT Aspire concordance table is presented in Table 12.7.

12.3 Summary The primary purpose of the concordance analysis was to facilitate the transition from ACT Explore and ACT Plan assessments to ACT Aspire assessment. While the EPAS to ACT Aspire concordance tables provide a link between the two scales, the concordance should be used cautiously. For example, the sample of students included in the concordance analysis may not be representative of all EPAS or ACT Aspire test-takers. In addition, population dependence often results when the two tests that are linked do not measure the same construct. It is generally inappropriate to use the concorded scores to estimate individual student performance on the EPAS/ACT Aspire tests because scores resulting from concordance analysis are not viewed as interchangeable with actual scores. It is preferable to use actual scores from the EPAS or ACT Aspire when possible.

12.16 EPAS® to ACT Aspire Concordance

Table 12.7. EPAS to ACT Aspire Concordance

Concorded ACT Aspire Scale Score EPAS Scale Score English Mathematics Reading Science

1 400 400 400 400 2 401 402 401 401 3 402 403 402 402 4 403 404 403 403 5 404 406 404 403 6 405 407 405 404 7 406 408 406 405 8 408 409 407 406 9 410 409 408 407 10 414 411 410 408 11 417 412 411 409 12 419 414 413 410 13 422 415 415 412 14 424 417 418 414 15 426 420 420 416 16 428 422 422 419 17 430 425 424 422 18 433 428 425 425 19 435 431 427 427 20 437 432 428 430 21 439 434 429 432 22 440 435 430 434 23 442 437 431 435 24 443 438 432 436 25 445 439 434 438

12.17 EPAS® to ACT Aspire Concordance

Table 12.7. (continued)

Concorded ACT Aspire Scale Score EPAS Scale Score English Mathematics Reading Science

26 447 440 434 439 27 448 441 435 440 28 449 443 435 441 29 450 444 436 442 30 451 446 437 443 31 451 449 437 445 32 453 452 438 447

12.18 CHAPTER 13 ACT Readiness Benchmarks

This chapter presents the method and procedures used to set up the ACT Readiness Benchmarks, which provide information about students’ college and career readiness by grade 11. The ACT Readiness Benchmarks for grades 8 through 10 were derived from the EPAS to ACT Aspire concordance tables. The corresponding concorded ACT Aspire scale scores for the ACT College Readiness Benchmarks were taken as the ACT Readiness Benchmarks on the ACT Aspire scale. The grade 8 ACT Readiness Benchmarks were then used to obtain the ACT Readiness Benchmarks for grades 3–7 using a z-score backmapping procedure. The backmapped benchmarks will be reviewed and may be updated once longitudinal data are available.

13.1 Introduction The ACT College Readiness Benchmarks are cornerstones of the ACT and the legacy assessments ACT Explore (grades 8 and 9) and ACT Plan (grade 10), which together form EPAS. The Benchmarks were established to reflect college and career readiness. The Benchmark on each of four subject tests of the ACT (English, mathematics, reading, and science) are the score on the 1–36 EPAS scale at which students have a 50% probability of attaining a grade of B or higher or about a 75% chance of obtaining a C or higher in selected credit-bearing first-year college courses (for additional details, see ACT, 2014b).

The ACT Readiness Benchmarks used with ACT Aspire were created to be aligned with the ACT College Readiness Benchmarks used with EPAS. Similar to EPAS, each ACT Aspire grade and subject has its own ACT Readiness Benchmark. Students at or above the benchmark are on target to meet the corresponding ACT College Readiness Benchmarks in grade 11.

13.1 ACT Readiness Benchmarks

13.2 ACT Readiness Benchmarks for English, Mathematics, Reading, and Science For English, mathematics, reading, and science, the ACT Readiness Benchmarks for grades 8 through 10 were derived using the EPAS to ACT Aspire concordance tables (see Chapter 12). Benchmarks for grades 3–7 were created using a backmapping procedure.

13.2.1 Grades 8–10 A concordance between the EPAS 1–36 scale and the ACT Aspire three-digit scale was used to establish ACT Readiness Benchmarks for grades 8–10 using ACT College Readiness Benchmarks that had already been established for EPAS. The concordance study included students taking both the EPAS and ACT Aspire tests. The concordance between EPAS scale scores and ACT Aspire scale scores was obtained using equipercentile linking. The ACT College Readiness Benchmarks for grades 8, 9, and 10 were used to identify the corresponding concorded ACT Aspire scores, which were then defined as the ACT Readiness Benchmarks for grades 8–10. Note that the concorded benchmarks derived for grades 8–10 may be updated as more longitudinal data become available. The ACT College Readiness Benchmarks used in this analysis were obtained by using an alternate set of benchmarks than those typically reported for ACT Explore and ACT Plan. This alternate set of benchmarks was based on students tested in spring, while EPAS benchmarks typically used for grades 8–10 are based on students tested in fall. ACT Aspire is administered in both fall and spring, but it is expected that benchmarks would differ slightly within the same grade due to additional instruction and academic achievement, which occurs throughout the school year. Spring benchmarks were used because they anchored the ACT Readiness Benchmarks to spring levels of educational development, near the end of a particular grade level when ACT Aspire was anticipated to be most commonly administered. Therefore, the ACT Readiness Benchmarks used for ACT Aspire are based on students tested in spring and reflect performance levels of students toward the end of an academic year. This should be kept in mind when interpreting results from fall testing. Table 13.1 presents the ACT College Readiness Benchmarks on the 1–36 scale for grades 8, 9, and 10 used in this study and the corresponding ACT Readiness Benchmarks, which are the concorded ACT Aspire scores obtained from the concordance. As shown in the table, ACT College Readiness Benchmarks and ACT Readiness Benchmarks are based on scale scores, and only one set of such benchmarks are available. Students’ scale scores are made comparable through the equating process regardless if they take the ACT Aspire tests online or on paper. For more details about equating, see Chapter 20.

13.2 ACT Readiness Benchmarks

Table 13.1. ACT College Readiness Benchmarks and ACT Readiness Benchmarks for Grades 8–10

ACT College ACT Readiness Benchmark Subject Grade Readiness Benchmark (Spring) (Concorded ACT Aspire Score) English 8 13 422 9 15 426 10 16 428 Mathematics 8 17 425 9 18 428 10 20 432 Reading 8 17 424 9 18 425 10 20 428 Science 8 19 427 9 20 430 10 21 432

Agreement rates for classifying students at or above both sets of benchmarks were compared using data from the concordance study. There was agreement if students were classified into the same category using both the EPAS benchmarks and the ACT Aspire benchmarks for their particular grade. As shown in Table 13.2, the agreement rates for all subjects were at or above 80%.

Table 13.2. Classification Agreement Rate between ACT Aspire and EPAS Benchmarks

ACT Aspire Classification Agreement Rate ACT Explore ACT Plan

English 81% 80% Mathematics 82% 85% Reading 81% 81% Science 83% 84%

13.2.2 Grades 3–7 ACT Readiness Benchmarks for grades 3–7 in English, mathematics, reading, and science were backmapped from the grade 8 ACT Readiness Benchmark described above. Backmapping used a z-score approach, which involved identifying the ACT Aspire scores at a standardized score (z-score) that corresponded to the z-score for the grade 8 ACT Readiness Benchmark. Specifically, a z-score (z8) corresponding to the grade 8 ACT Readiness Benchmark was computed based on the difference between this benchmark score (x) and mean (μ) of grade 8 scale scores and divided by the standard deviation (σ) of those scores: z8 = (x – μ)/σ. This z-score was then 13.3 ACT Readiness Benchmarks

used to obtain the Readiness Benchmarks for grades 3–7 (xj ) based on the standard

deviation (σj ) and mean (μj ) of the scale scores for those grades, respectively, with

the formula xj = z8σj + μj , where j represents a grade ( j = 3, 4, 5, 6, 7). Spring 2013 ACT Aspire special study data were used to create benchmarks for grades 3–7. Table 13.3 shows the backmapped ACT Readiness Benchmarks for grades 3–7. Note that the backmapped benchmarks derived for grades 3–7 will be reviewed and may be updated as more longitudinal data become available.

Table 13.3. ACT Readiness Benchmarks for Grades 3–7

Grade English Mathematics Reading Science

3 413 413 415 418 4 417 416 417 420 5 419 418 420 422 6 420 420 421 423 7 421 422 423 425

13.3 ACT Readiness Benchmark for Writing The ACT Readiness Benchmark for writing was set as the scale score corresponding to raw scores at a cut determined by ACT content experts, based on their knowledge, expertise, and experience with writing content and their development of the rubrics used to score writing prompts. A domain score of 4 formed the basis for the writing cut, which corresponds to performance consistent with grade-level expectations on the rubric. Recall that writing scores consist of four domains, each scored on a rubric with scores from 1 to 5 (grades 3–5) or 1 to 6 (grades 6–10). Content experts determined that a domain score of 4 was at the expected readiness level for each grade level. Across the four domains, if a student obtained domain scores of two fours and two threes, the student’s score would be considered at the cut for identifying ready performance on the spring 2013 writing base form. This pattern of four domain scores corresponded to a scale score of 428 on the base form. A scale score of 428 is used as the cut score for writing at all grade levels. The writing benchmark may be updated as more longitudinal data become available.

13.4 ACT Readiness Benchmarks

13.4 ACT Readiness Benchmarks for ELA and STEM The ACT Readiness Benchmarks for ELA and STEM are computed as the average of the subject Readiness Benchmarks that contribute to each score. ACT Readiness Benchmarks for ELA are the average of the benchmarks for English, reading, and writing tests. ACT Readiness Benchmarks for STEM were the average of the benchmarks for mathematics and science tests. Table 13.4 lists the ACT Readiness Benchmarks for ELA and STEM across grade levels.1 Benchmarks could have been established through a separate backmapping process, but the average was used because it was anticipated to be simpler for users, less likely to lead to confusion, and there was no such score or benchmark on the ACT test to backmap from when these benchmarks were first established.

Table 13.4. ACT Readiness Benchmarks in ELA and STEM for Grades 3–10

Grade ELA STEM

3 419 416 4 421 418 5 422 420 6 423 422 7 424 424 8 425 426 9 426 429 10 428 432

13.5 ACT Readiness Levels In addition to the ACT Readiness Benchmarks, students are provided with a description of how they perform relative to the ACT Readiness Benchmarks in English, mathematics, reading, science, and writing. This description is called the ACT Readiness Level and includes four categories defined by three cut scores for each subject and each grade: high cut above the benchmark, the benchmark, and low cut below the benchmark.

1 It is possible for a student to be inconsistent with readiness in a particular subject area. This is due to the compensatory nature of the ELA and STEM scores, where subject area scores are combined to obtain these scores. For example, a student could be below the mathematics benchmark and still be above the STEM benchmark if the student had a relatively high science test score. In such a case, the science test score could pull the STEM score up enough to meet the STEM benchmark. 13.5 ACT Readiness Benchmarks

The four ACT Readiness Levels include: 1. Exceeding: at or above the high cut 2. Ready: at or above the benchmark and below the high cut 3. Close: at or above the low cut and below the benchmark 4. In Need of Support: below the low cut

Each of the four ACT Readiness Levels is intended to provide a description of how students perform relative to the ACT Readiness Benchmarks in each subject area.

The high and low cuts were set considering the standard error of measurement (SEM; Geisinger, 1991). Specifically, the high cut was set to be two SEMs above the Benchmark, and the low cut was set to be two SEMs below the Benchmark for all subjects except writing. Two SEMs were chosen because it represented a substantial deviation from the benchmark. From a statistical perspective, under typical assumptions, two standard errors from the benchmark represent a roughly 95% confidence interval for the Ready and Close category; we can be about 95% confident that scores falling outside of this range are indeed above or below the benchmark. Of course, values other than two SEMs could have been chosen to define the ACT Readiness Levels, but two SEMs was deemed a reasonable compromise between adequate representation of the descriptors Exceeding and In Need of Support and statistical characteristics indicative of deviation from Ready and Close.

For writing, once the ACT Readiness Benchmark was established, the low and high cuts were set by content experts based on rubrics. Specifically, the low cut was defined as the scale score on the spring 2013 writing base form that corresponded to two threes and two twos on the four domains. The high cut was defined as the scale score on the spring 2013 writing base form that corresponded to two fives and two fours on the four domains. All benchmarks, low cuts, and high cuts are presented in Table 13.5.

13.6 ACT Readiness Benchmarks

Table 13.5. Benchmarks, Low Cuts, and High Cuts for ACT Readiness Benchmarks by Subject and Grade

Subject Tested Grade Low Cut Benchmark High Cut English 3 408 413 418 4 411 417 423 5 412 419 426 6 413 420 427 7 413 421 429 8 415 422 429 9 419 426 433 10 421 428 435 Mathematics 3 409 413 417 4 411 416 421 5 412 418 424 6 414 420 426 7 416 422 428 8 419 425 431 9 422 428 434 10 426 432 438 Reading 3 411 415 419 4 412 417 422 5 415 420 425 6 416 421 426 7 417 423 429 8 418 424 430 9 419 425 431 10 422 428 434 Science 3 414 418 422 4 415 420 425 5 417 422 427 6 418 423 428 7 420 425 430 8 422 427 432 9 424 430 436 10 426 432 438 Writing 3 420 428 436 4 420 428 436 5 420 428 436 6 420 428 436 7 420 428 436 8 420 428 436 9 420 428 436 10 420 428 436

13.7 ACT Readiness Benchmarks

13.6 Readiness Ranges for Reporting Categories In order to provide students with more detailed information within each subject, items that measure the same skills and abilities are grouped into reporting categories. For each reporting category, the total number of points possible, the total number of points a student achieved and the percentage of points correct are provided.

In addition, the ACT Readiness Range in each reporting category is provided to show how a student who has met the ACT Readiness Benchmark in a particular subject area would typically perform within the reporting category. In this way students can compare the percentage of points achieved in each category to the percentage of points attained by a typical student who is on track to be ready. If their scores fall below the ACT Readiness Range, they may be in need of additional support.

The minimum of the ACT Readiness Range for each reporting category corresponds to the predicted percent correct on that reporting category at the ACT Readiness Benchmark for that subject test. The maximum corresponds to 100% correct. Regression was used to predict the percent correct on the reporting category using the scale score at the Benchmark point. Note that the ACT Readiness Range will differ across forms.

13.8 CHAPTER 14 ACT Aspire Growth

ACT Aspire supports interpretations of student and group-level growth. Score reports include the following components, each of which contributes to interpretations of growth: • A student’s current and prior-year scores in all tested subjects (English, mathematics, reading, science, and writing) • Comparison to ACT Readiness Benchmarks (Chapter 13) that indicate whether students are on target to meet the ACT College Readiness Benchmarks in grade 11 • Classification of a student into ACT Readiness Levels (Chapter 13) that describe where a student scores relative to the ACT Readiness Benchmarks • Predicted score paths that provide ranges for a student’s expected scores in future years (and predicted score ranges on the ACT for grades 9 and 10) • Classification of student growth as “Low,” “Average,” or “High,” based on student growth percentiles (SGPs).

The latter two components (predicted score paths and SGPs) are described in this chapter. Other topics discussed in this chapter include ACT Aspire gain scores, growth-to-standards models, measurement error of growth scores, and aggregate growth scores for research and evaluation.

14.1 Predicted Score Paths Predicted score paths are reported to enhance understanding of where a student (or a group of students) is likely to score in future years, assuming typical growth. This information can be used: • To determine if students are likely to meet ACT Readiness Benchmarks over the next two years 14.1 ACT Aspire Growth

• To identify students who are unlikely to meet a future-year standard (other than the ACT Readiness Benchmark) and thus are candidates for extra academic support • To predict aggregate future achievement for a classroom, school, district, or state • To predict ACT score ranges (for grades 9 and 10)

Predicted ACT Aspire score ranges are provided for English, mathematics, reading, science, and writing. For grades 9 and 10, predicted grade 11 ACT scores are provided for the same subjects, as well as ACT Composite scores. Writing scores are not vertically scaled across grade levels, and predicted paths for writing scores only encompass one year.

Figure 14.1. Prototype of ACT Aspire Student Progress Report

14.1.1 Student Progress Reports An example Student Progress Report that illustrates the predicted path is shown in Figure 14.1. The predicted path is represented by a cone-shaped orange-shaded area that covers two years. In this example, an eighth-grade student scored 417 on the ACT Aspire English assessment. The student’s predicted path covers the score range 416–424 for grade 9 and 415–431 for grade 10. (Note that the numbers forming the predicted score ranges do not appear on the progress report and are subject to change over time). Assuming typical growth, it is estimated that the two-year predicted score ranges cover about 75% of actual future scores. The one-year predicted ranges cover about 50% of actual future scores.

14.1.2 Aggregate Progress Reports Predicted mean scores are used to form aggregate one-year predicted paths for classrooms, schools, and districts. The aggregate predicted paths are drawn as lines connecting the current year mean score to the next year’s predicted mean score.

Predicted mean scores for classrooms, schools, and districts are calculated as the mean of individual student predicted scores. Individual student predicted scores are based on the estimated mean of the test score distribution, conditional on the prior- year test score. A prototype Aggregate Progress Report is shown in Figure 14.2. In this example, the mean ACT Aspire science score was 418 for a group of grade 9 students. The predicted grade 10 mean ACT Aspire science score for the same group is plotted, and the predicted grade 11 mean science score on the ACT is 17.8. 14.2 ACT Aspire Growth

We summarize the methods used to develop the predicted paths later in this chapter. The development of the predicted paths that were used for reporting in spring 2014 through spring 2015 is documented in a separate report (Allen, 2014).

Figure 14.2. Prototype ACT Aspire Aggregate Progress Report

14.1.3 Samples Used to Develop the Predicted Paths Students who tested in multiple years form a longitudinal sample of ACT Aspire- tested students and were used to develop predicted paths that are used for ACT Aspire reports. Each year, the longitudinal sample is updated with data from the most recent testing year. Here, we describe the sample used in 2015 for the development of the predicted paths that are used for reporting in spring 2016. The longitudinal sample was formed by matching ACT Aspire student records from spring 2013, spring 2014, and spring 2015. For grades 9–11 and 10–11, the longitudinal samples were formed by matching ACT Aspire student records from spring 2013 and spring 2014 to ACT test records from spring 2014 and spring 2015.

Table 14.1 provides the sample sizes for each grade level and subject area. For writing (all grade levels) and grade 9 test scores, one-year predicted paths were developed using students tested in consecutive years (e.g., grade level pairs 3–4, 4–5, 5–6, 6–7, 7–8, 8–9, and 9–10). For all other subject areas, two-year predicted paths were developed using students tested two years apart (3–5, 4–6, etc.). Predicted ACT score ranges for grade 10 are based on students tested in consecutive years with ACT Aspire and the ACT, while predicted ACT score ranges for grade 9 are based on students who tested two years apart with ACT Aspire and the ACT.

14.3 ACT Aspire Growth

Table 14.1. Longitudinal Samples Used to Develop Predicted Paths, Predicted ACT Score Ranges, and Student Growth Percentiles for Spring 2016 Reporting

Subject Area Grade Level Pair English Mathematics Reading Science Writing Composite 3–4 16,658 61,514 61,615 15,371 13,935 3–5 4,858 9,473 9,566 7,882 4–5 16,949 61,696 61,640 17,651 15,062 4–6 4,969 9,503 9,510 4,956 5–6 16,498 60,824 60,897 12,974 14,132 5–7 5,196 9,139 9,154 7,984 6–7 15,456 59,407 59,382 15,648 15,240 6–8 3,996 6,996 6,937 3,530 7–8 14,768 58,884 58,810 11,452 15,177 7–9 744 717 711 694 8–9 3,874 4,148 3,924 3,876 3,363 8–10 453 423 405 357 9–10 6,436 6,634 6,600 6,172 4,862 9–11 (the ACT) 2,414 2,085 2,002 2,085 392 1,515 10–11 (the ACT) 11,275 10,916 10,923 10,720 6,444 9,243

*Concordance-derived sample

14.1.4 Statistical Models Used to Develop the Predicted Paths Two-year predicted paths are based on a regression model where grade g test score is used to predict grade g+2 test score. The regression model uses linear and quadratic effects of the grade g score to predict the grade g+2 score, and a 70% prediction interval forms the endpoints of the two-year predicted path score range. (Note that the 70% prediction interval results in a coverage rate of about 75% because the test scores take integer values and scores that are equal to the interval endpoints are counted as being within the interval). The full predicted path, which encompasses one-year and two-year predictions, is drawn by connecting the current year score to the endpoints of the two-year predicted path. This is the approach used for English, mathematics, reading, and science for grades 3–8.

For writing, all grade 9 test scores, and grade 10 predicted ACT score ranges, only one-year predicted score ranges are reported. For these cases, the predicted path endpoints are defined as the scores closest to the 25th and 75th percentiles of the score distribution at grade g+1, conditional on score at grade g. The quantiles of the conditional score distribution are estimated using the same methodology used to estimate student growth percentiles (described later in this chapter). 14.4 ACT Aspire Growth

14.1.5 Limitations of the Predicted Paths The predicted paths were developed using samples of students tested in multiple years between spring 2013, spring 2014, and spring 2015. A large percentage of students come from one state (Alabama, the first state to administer ACT Aspire on a statewide basis). With larger samples of students, the estimation of the predicted paths will have less sampling error. It is also possible that predicted path estimates could shift up or down with the inclusion of more school districts and states in the sample, or with changes in student growth over time. The predicted paths will be re-estimated as more data become available. The predicted paths are currently designed to have coverage rates of about 75% for the two-year paths, and about 50% for the one-year paths.

14.2 Student Growth Percentiles Student growth percentiles (SGPs) represent the relative standing of a student’s current achievement compared to others with similar prior achievement. ACT Aspire SGPs, ranging from 1 to 100, are provided for students who test in consecutive years. They are used to classify students’ growth into the following categories: “Low” (SGP<25), “Average” (SGP between 25 and 75), or “High” (SGP>75). ACT Aspire SGPs measure growth over one-year time intervals.

SGPs are estimated using quantile regression methods (Koenker, 2005) by the SGP R package (Betebenner, VanIwaarden, Domingu, & Shang, 2014). SGPs are reported for students who also tested the previous year with the tests one grade level apart.

When interpreting SGPs, the reference group used to estimate the model should always be considered. The sample sizes used to estimate the SGPs reported in fall 2015 and spring 2016 are given in Table 14.1 (for grade level pairs one year apart). The SGPs used for ACT Aspire are based on all students tested nationally in adjacent years, so the reference group is expected to change over time. Because of these expected changes and possible changes in the amount of student growth observed nationally over time, SGPs reported by ACT Aspire can shift over time, confounding comparisons of growth for different cohorts of students.

14.2.1 Consistency of SGPs across Subject Areas Based on the same samples used to develop the predicted paths and SGPs, we estimated the correlations of SGPs across subject areas. A positive correlation indicates consistency between SGPs for different subject areas. A negative correlation indicates inconsistency between SGPs for different subject areas. In Table 14.2, the correlations are presented for each pair of subject areas. The sample used to estimate each correlation includes each instance of different subject tests being taken in consecutive years. Results are presented for all grade levels combined.

14.5 ACT Aspire Growth

All of the correlations in Table 14.2 are positive and range from 0.149 (writing and mathematics) to 0.273 (reading and science). The correlations provide some evidence of consistency between SGPs for different subject areas. Students who demonstrate high (or low) growth in one subject area are more likely to also demonstrate high (or low) growth in other subject areas. Possible explanations for this consistency in SGPs across subject areas include 1) development of knowledge and skills in one subject area may directly influence development in other subject areas, and 2) growth across subject areas may be driven by a shared factor, such as motivation. The correlations are relatively small, suggesting modest levels of consistency. The correlations are attenuated towards 0 because of measurement error in the SGPs.

Table 14.2. Correlations of SGPs across Subject Areas

Subject area pair N Correlation English, Reading 99,414 0.233 English, Writing 67,233 0.157 English, Mathematics 99,438 0.195 English, Science 78,092 0.236 Reading, Writing 87,606 0.156 Reading, Mathematics 319,935 0.194 Reading, Science 92,207 0.273 Writing, Mathematics 87,601 0.149 Writing, Science 62,957 0.159 Mathematics, Science 92,397 0.227

14.3 ACT Aspire Gain Scores ACT Aspire English, mathematics, reading, and science scores are vertically scaled across grade levels (see Chapter 10 for details on the scaling procedures), making it possible to monitor progress over time on the same scale. Writing scores are not vertically scaled across grade levels, but instead are based on scoring rubrics where a score of “4” indicates meeting expectations for that grade level. Consistency in writing scores over time suggests the same performance, relative to grade level expectations (see Chapter 10). For all subjects, viewing scores graphically over multiple years provides insights about a student’s current achievement level, as well as progress made with respect to ACT Readiness Benchmarks and ACT Readiness Levels.

Gain scores are the arithmetic difference in scores from one year to the next. Gain scores are an attractive growth measure because of their simplicity and intuitive appeal. Unlike SGPs, calculations of gain scores are independent of reference groups. However, normative information from a reference group is still helpful for the 14.6 ACT Aspire Growth

interpretation of gain scores. For example, knowing the mean gain over one year can be used to determine whether a student's gain is higher or lower than average. For all subjects except writing, positive ACT Aspire gain scores are anticipated because students are expected to increase their knowledge and skills in the tested areas after one year of schooling.

Table 14.3. ACT Aspire Gain Score Means and Standard Deviations

Subject Area Grade Level Pair English Mathematics Reading Science Writing Composite 3–4 3.8 (4.9) 3.1 (3.5) 2.6 (4.0) 3.3 (4.5) 2.3 (5.5) 4–5 2.9 (5.2) 2.2 (4.0) 2.5 (4.2) 2.2 (4.5) 1.0 (6.3) 5–6 2.1 (5.8) 2.9 (4.7) 2.3 (4.6) 1.9 (4.8) 1.0 (7.1) 6–7 2.5 (6.1) 0.1 (5.0) 0.3 (4.9) 1.1 (5.1) -2.9 (7.1) 7–8 1.8 (6.3) 2.1 (4.7) 2.5 (4.9) 2.8 (5.4) 1.4 (6.0) 8–9 1.0 (5.9) 1.4 (5.0) 0.0 (5.5) 1.9 (5.6) 1.7 (6.4) 1.1 (3.4) 9–10 1.0 (6.3) 1.3 (5.2) 1.1 (5.7) 1.3 (6.4) 1.9 (6.3) 1.2 (3.7) Source: Based on data from spring 2013, spring 2014, and spring 2015.

For the longitudinal samples used to develop the ACT Aspire Student Growth Percentiles (see Table 14.1), gain score means and standard deviations are provided in Table 14.3. There is considerable variation across grade levels and subject areas in mean gain scores. For most subjects with vertical scales (all but writing), mean gain scores are typically positive, showing that students in the sample typically increased their knowledge and skills in the tested areas after one year of schooling. Mean gain score is very small for mathematics grades 6–7 (0.1), reading grades 6–7 (0.3), and reading grades 8–9 (0.0). For the writing test, one should not necessarily expect positive mean gain scores because scores are not vertically scaled across grade levels. The mean gain scores for ACT Aspire writing ranged from 2.3 for grades 3–4 to −2.9 for grades 6–7.

14.3.1 Interpreting ACT Aspire Gain Scores Because ACT Aspire scores in all subject areas except writing are reported on a common vertical scale, gain scores are intended to be interpreted as measuring change in knowledge and skills from one year’s test to another. However, because no educational scale can be interpreted as having equal intervals in a strict sense, it should not be assumed that the meaning of gain scores is the same across the score scale. For example, it is possible that some regions of the score scale can be more sensitive to student learning. Also, students who score at the low end of a scale are generally expected to have larger one-year gain scores than students who score at the high end. This phenomenon is illustrated in Figure 14.3 using the longitudinal sample of ACT Aspire-tested students who took the grade 7 reading test and the 14.7 ACT Aspire Growth

grade 8 reading test in consecutive years. The figure shows the mean gain score, as well as the 25th and 75th percentile of the gain score distribution, for each grade 7 reading score point with a sample size of at least 100 students. The negative relationship between prior-year score and one-year gain score results because test scores are only estimates of true achievement levels: students who score far above (below) the mean are more likely than others to have scored above (below) their true achievement level and tend to score closer to the mean the next year.

Figure 14.3. ACT Aspire gain score statistics for grade 7–8 reading

Across grade levels and subject areas, the median correlation of prior year score and gain score is -0.29, the 25th percentile is -0.41 and the 75th percentile is -0.25. For reading, the correlation of grade 7 reading score to one-year gain score is -0.29, so Figure 14.3 represents a typical strength of relationship between prior year score and gain score.

14.4 Growth-to-Standards Models with ACT Aspire Unlike normative growth measures such as SGPs, growth-to-standards models determine if students are making sufficient progress toward a performance standard, such as the ACT Readiness Benchmarks or other ACT Readiness Levels. ACT Aspire supports growth-to-standards models because scores are reported over time and plotted against the ACT Readiness Benchmarks and ACT Readiness Levels (see Figure 14.1). The predicted paths also support growth-to-standards models because they indicate how students are likely to perform over the next two years, assuming typical growth. For example, if a fourth grader’s predicted path falls below the ACT Readiness Benchmark for grades 5 and 6, he or she knows that atypically high growth over the next two years will be needed to reach the ACT Readiness Benchmark.

14.8 ACT Aspire Growth

Growth-to-standards models can be implemented by specifying a performance standard such as the ACT Readiness Benchmark and specifying the amount of time students have to reach the performance standard. For the student represented by the example report in Figure 14.1, the student’s score (417) was 5 points below the grade 8 ACT Readiness Benchmark for English (422). If the model assumes students have two years to catch up, the student would need to gain 11 score points to reach the grade 10 ACT Readiness Benchmark for English (428). Under a growth-to-standards model, the student would need to gain at least 6 points from grade 8 to grade 9 to have made sufficient progress toward the grade 10 performance standard. SGPs can also be used within a growth-to-standards model to communicate how much progress is needed using the growth percentile metric instead of the gain score metric.

14.5 Measurement Error of Growth Scores Measures of individual student growth, including SGPs, are subject to measurement error. This means that a student’s actual growth in academic achievement may be different than what is represented by their SGP (Wells, Sireci, & Bahry, 2014) or gain score. All test scores have measurement error, and measurement errors of SGPs and gain scores are more pronounced because measurement error of multiple test scores is compounded. In tests with vertical scales such as ACT Aspire, negative gain scores may be observed, due in part to measurement error. Because of their measurement error, neither SGPs nor gain scores should be used as the sole indicator of a student’s academic progress over one year. Instead, SGPs and gain scores can complement other evidence of learning such as performance on homework and quizzes, unit tests, and teacher observation.

14.6 Aggregate Growth Scores for Research and Evaluation For classrooms, schools, districts, states, and other user-defined groups, aggregate growth statistics are available to describe how much growth occurred in each group. Mean scale scores are plotted on longitudinal progress reports and the percentage of students in each growth category (low, average, or high) is reported. It is also possible to calculate other summary growth measures, such as the mean or median SGP, using available data.

Data from ACT Aspire may be used as one indicator of program effectiveness. One of the secondary interpretations of ACT Aspire scores involves providing empirical data for inferences related to accountability (see Chapter 1). Before using ACT Aspire for evaluating program effectiveness, a content review is critical to ensure that ACT Aspire measures important outcomes of the program. For example, a district might want to use the multiyear change in mean SGP of 10th–grade English students as one measure of the effectiveness of a new 10th–grade English curriculum. Prior to implementation, ACT Aspire content should be reviewed against the intended outcomes of the program to evaluate its appropriateness for measuring curriculum effectiveness. 14.9 ACT Aspire Growth

14.6.1 Normative Data for School Mean SGPs If used for evaluating educator effectiveness, ACT Aspire growth data should be used as one of multiple sources of evidence. As the stakes for particular uses increase, it becomes more important to carefully evaluate ACT Aspire score interpretations for these uses and to gather multiple sources of evidence. Measures of aggregate growth include the mean and median SGP. Research suggests that the mean SGP may have advantages over the median SGP in terms of efficiency, greater alignment with expected values, and greater robustness to scale transformations (Castellano & Ho, 2015).

We now examine distributions of school mean SGPs (MGPs), by subject area. To derive summary statistics of distributions of school mean SGPs, the following steps were taken: 1. For each combination of school, grade level, and subject area with at least ten students tested in consecutive grade levels, the mean SGP is calculated. 2. For each subject area, summary statistics of mean SGP, including N (number of combinations of school and grade level pairs), mean, standard deviation, and selected percentiles are calculated.

The results presented in Table 14.4 (see also Figure 14.4) can be used to aid interpretation of school mean SGP values. For example, a mean mathematics SGP of 40 is between the 10th and 20th percentiles. Conversely, a mean mathematics SGP of 60 is between the 80th and 90th percentiles. The results in Table 14.4 are combined across grade levels, and so do not reveal possible variation in mean SGP across grade levels.

It is also important to note that the distribution of school mean SGP is affected by the size of the group. When mean SGP is based on a smaller sample size, it is less precise, and extreme mean SGP values are more likely to result.

14.10 ACT Aspire Growth

Table 14.4. Distributions of School MGPs, by Subject Area

Subject Area

Statistic English Mathematics Reading Science Writing N students 100,134 322,142 321,935 91,381 86,635 N units* 1,093 3,285 3,279 1,084 1,130 Mean 49.6 49.2 49.5 50.0 50.9 SD 9.1 9.9 8.4 9.9 11.4 Percentiles 5 35 34 36 34 32 10 39 37 39 37 36 20 42 41 42 42 41 25 43 42 44 43 43 30 45 44 45 45 45 40 47 46 48 47 48 50 49 49 50 50 50 60 52 51 52 52 54 70 54 54 54 55 57 75 56 56 55 57 58 80 58 57 56 58 60 90 61 62 60 63 65 95 65 66 64 66 70 *The number of school/ grade level combinations.

Figure 14.4. Distributions of school MGPs, by subject area Distributions of School Mean SGPs, by Subject Area 80

50 P English G S

n 40 Mathematics a e

M Reading 30 Science 20 Writing 10

0 5 10 20 25 30 40 50 60 70 75 80 90 95 Percentile

14.11 ACT Aspire Growth

14.6.2 Mean SGP Differences across Student Subgroups The mean SGP can be used to identify growth differences across districts, schools, classrooms, or other groups. However, this alone does not explain reasons for the growth differences. When comparing mean SGPs across groups, it is important to consider whether differences in the student composition of the groups could explain the group differences. For example, a school serving economically disadvantaged students might be expected to have lower mean SGP than a school serving students from affluent families.

Analyses were conducted to examine SGP differences across student subgroups. The subgroups are defined by race/ethnicity, gender, English language learner status, economic disadvantage status, and special education status. All subgroup indicators are provided by school or district personnel. Special education status is determined by student participation in an individualized education program (IEP). In Table 14.5, we present the SGP sample sizes for selected student subgroups, using all available longitudinal data collected through spring 2015. In Table 14.6, we present the mean SGP for selected student subgroups. Results are aggregated across grade levels.

By race/ethnicity, mean SGPs are consistently higher for Asian students and lowest for African American students. The mean SGPs for white students were slightly above average (50), ranging from 51.6 in English to 53.2 in writing. For Hispanic students, mean SGPs ranged from 45.8 in English to 48.3 in mathematics and writing. The gender differences are relatively small, with the mean SGP for females consistently larger than the mean SGP for males. The largest gender difference is observed for writing (55.2 for females, 47.7 for males) and the smallest difference is observed for mathematics (50.9 for females, 49.2 for males).

Mean SGPs for English language learners were below average, ranging from 42.9 in English to 47.0 in mathematics. Similarly, for economically disadvantaged students, mean SGP ranged from 43.9 in English to 49.6 in writing. Across student subgroups, mean SGPs were lowest for special education students, ranging from 36.0 in writing to 38.4 in mathematics.

14.12 ACT Aspire Growth

Table 14.5. SGP Sample Size by Subject Area and Student Subgroup

Subgroup English Mathematics Reading Science Writing Race/ethnicity Black/African American 23,828 95,901 96,195 23,145 21,664 American Indian/Alaska Native 1,246 2,964 2,934 844 1,024 Asian 1,965 4,700 4,670 1,423 1,159 Hispanic or Latino 6,237 19,827 19,715 6,004 5,518 Native Hawaiian/Other Pacific Islander Two or more races 668 664 672 642 462 White 59,353 188,028 187,590 53,639 51,391 Gender Female 53,095 161,944 161,748 48,965 45,095 Male 48,064 161,302 161,250 44,137 42,543 English language learner 1,310 1,500 1,482 1,314 1,261 Economically disadvantaged 6,685 9,927 9,943 6,618 5,778 Special education 2,377 4,608 4,609 2,469 2,190

Table 14.6. Mean SGP by Subject Area and Student Subgroup

Subgroup English Mathematics Reading Science Writing Race/ethnicity Black/African American 44.8 44.1 45.7 44.1 46.5 American Indian/Alaska Native 49.9 50.6 51.4 49.2 49.6 Asian 55.7 61.3 58.3 56.4 57.6 Hispanic or Latino 45.8 48.3 47.9 47.1 48.3 Native Hawaiian/Other Pacific Islander Two or more races 47.5 48.5 48.0 47.7 49.8 White 51.6 52.8 52.0 51.9 53.2 Gender Female 51.7 50.9 51.7 50.8 55.2 Male 47.7 49.2 48.2 48.7 47.7 English language learner 42.9 47.0 45.2 46.1 46.5 Economically disadvantaged 43.9 46.2 47.6 45.1 49.6 Special education 36.7 38.4 37.8 38.0 36.0

14.13 ACT Aspire Growth

14.7 Additional Growth Modeling Resources For additional information on student growth percentiles, see ACT Growth Modeling Resources online at http://www.act.org/content/act/en/research/act-growth- modeling-resources.html.

For information that can help parents, teachers, and administrators interpret growth data, see “Answers to questions about student growth” (parent, teacher, and administrator versions), available at:

http://www.act.org/content/dam/act/unsecured/documents/Answers-to- Administrators-Questions-about-Student-Growth.pdf

http://www.act.org/content/dam/act/unsecured/documents/Answers-to-Teachers- Questions-about-Student-Growth.pdf

http://www.act.org/content/dam/act/unsecured/documents/Answers-to-Parents- Questions-about-Student-Growth.pdf

14.14 CHAPTER 15 Progress toward Career Readiness

This chapter describes the procedures and results of a study used to develop predicted performance levels on the ACT National Career Readiness Certificate using ACT Aspire scores. This predicted performance was used to develop the Progress toward Career Readiness indicator included on ACT Aspire score reports.

15.1 Introduction For decades it has been a commonly held belief that high school students planning to go to college need to take more rigorous coursework than those going directly into the workforce. Today, however, many employers report that in an expanding global economy, entry-level workers need many of the same types of knowledge and skills as college-bound students (ACT, 2006). To help students predict and monitor whether they are on track to be career ready toward the end of high school using ACT Aspire performance, students at grade 8 through EHS with Composite scores (i.e., receiving scores on English, mathematics, reading, and science tests) receive a Progress toward Career Readiness indicator.

The ACT National Career Readiness Certificate™ (ACT NCRC®), awarded based on ACT WorkKeys® test results, is a portable credential that demonstrates achievement and is based on assessment level scores associated with workplace employability skills in three areas: applied mathematics, locating information, and reading for information. Assessment level scores typically run from Level 3 to Level 6 or 7. In addition, the ACT NCRC has four levels of certificate: Bronze, Silver, Gold, and Platinum. A Platinum certificate is earned if the three assessment level scores a student earns are all Level 6 or higher. A Gold certificate is earned if the three assessment level scores a student earns are all Level 5 or higher. A Silver certificate 15.1 Progress toward Career Readiness

is earned if the three assessment level scores a student earns are all Level 4 or higher. A Bronze certificate is earned if the three assessment level scores a student earns are all Level 3 or higher. If any of the three assessment level scores are less than Level 3, no certificate is achieved (ACT, 2007).

The actual knowledge and skills required across careers and career domains may differ greatly. Further, the ACT and ACT Aspire do not measure all such knowledge and skills. Rather, the ACT and ACT Aspire measure academic achievement, or foundation skills which are a subset of skills associated with career readiness. The linkage between ACT Aspire and the ACT NCRC is based only on these academic skills and provides a prediction of future performance. Because the constructs and content across the ACT and ACT Aspire differ somewhat from that of the ACT WorkKeys used in the ACT NCRC, a prediction method was used to indicate if a student is likely to meet Bronze, Silver, or Gold ACT NCRC level at the completion of high school.

The Progress toward Career Readiness cut scores were created for grades 8–11 to define indicators of progress towards career readiness to predict future ACT NCRC performance (see Figure 15.1). The first step in creating the cut scores was linking ACT NCRC level to the ACT Composite scores using a prediction method. A concordance was then used to find the ACT Aspire Composite scores that corresponded to each of the cut scores on the EPAS scale. Finally, backmapping was used to obtain the cut scores associated with Bronze, Silver, and Gold ACT NCRC levels for grades 8–10.

Figure 15.1. Sample Progress toward Career Readiness indicator from ACT Aspire report

15.2 Progress toward Career Readiness

15.2 Link ACT NCRC Levels to the ACT Composite Scores Data from over 110,000 grade 11 students who took ACT WorkKeys and the ACT were used to establish the link between the ACT Composite scores and the ACT NCRC levels. Four separate logistic regressions were conducted to determine the ACT Composite score that corresponded to a 50% chance of obtaining each of the four ACT NCRC levels, respectively. The ACT Composite score was the independent variable, and the status of whether a student was at or above a specific ACT NCRC level was the dependent variable (similar methods have been used for setting ACT College Readiness Benchmarks; e.g., Allen & Sconing, 2005).

Table 15.1 provides descriptive statistics of the ACT Composite scores for students at different ACT NCRC levels. As ACT NCRC level increases, the mean and median of ACT Composite scores increase, which is indicative of the positive relationship between ACT NCRC level and ACT Composite scores. The variability in performance on the ACT increases as ACT NCRC level increases through the Gold level. Between the Gold and Platinum levels, variability decreases slightly, which is likely the result of instability in the standard deviation due to the relatively small number of students in the Platinum group compared to the others.

Table 15.2 presents the logistic regression results—unrounded and rounded ACT Composite scores that are associated with a 50% chance of obtaining different ACT NCRC levels. For example, an ACT Composite score of 17 is required for a 50% chance of obtaining the ACT NCRC Silver certificate or higher while an ACT Composite score of 25 is required for a 50% chance of obtaining the ACT NCRC Gold certificate or higher.

Table 15.1. Descriptive Statistics of ACT Composite Scores by ACT NCRC Level

ACT Composite Score ACT NCRC level N Mean Median SD Min Max No Certificate 13,628 13.69 13 2.28 4 32 Bronze 23,726 15.86 16 2.64 6 33 Silver 50,087 19.76 20 3.56 7 35 Gold 25,669 25.20 25 3.81 11 36 Platinum 1,497 30.33 31 3.15 18 36

15.3 Progress toward Career Readiness

Table 15.2. ACT Composite Scores Corresponding to a 50% Chance of Obtaining Different ACT NCRC Levels

ACT Composite Cut Score ACT NCRC levels Unrounded Rounded

Bronze 12.74 13 Silver 16.43 17 Gold 24.15 25 Platinum 34.80 35

To ensure each final ACT Composite cut score corresponds to at least a 50% chance of receiving an ACT NCRC level, all unrounded Composite scores were rounded up to the final reported cut values. These four cut values are based on a sample of grade 11 students, so they should be interpreted as an indicator of career readiness for grade 11 students.

15.3 Link the EPAS Composite Scores to ACT Aspire Composite Scores The second step in the linkage between ACT Aspire and ACT NCRC certificate levels was to establish a concordance between the EPAS Composite scores on the EPAS tests (ACT Explore, ACT Plan, or the ACT) and ACT Aspire Composite scores. Data from spring 2013 ACT Aspire tests were merged with historical EPAS student data files to obtain a sample of students with EPAS and ACT Aspire Composite scores. Table 15.3 presents means, standard deviations, and the correlation between the EPAS and ACT Aspire Composite scores for the 13,528 students included in the sample.

Table 15.3. Descriptive Statistics of the Analysis Sample (N=13,528)

EPAS ACT Aspire Composite Score Composite Score

Mean 16.66 424.05 SD 3.95 7.72 Correlation .86

The concordance relationship between EPAS and ACT Aspire Composite scores was estimated using the equipercentile method described by Kolen and Brennan (2014). The same methodology was used to find the concordance relationship between EPAS and ACT Aspire for the individual subjects. See Chapter 12 for details on the methodology.

15.4 Progress toward Career Readiness

15.4 Identify the ACT Aspire Composite Scores Corresponding to Each ACT NCRC Level Once the concordance between the EPAS Composite scores and the ACT Aspire Composite scores was established, the Composite score on the ACT with constructed-response tests that best predicts each ACT NCRC level for grade 11 was taken directly from the concordance table. Table 15.4 presents the EPAS Composite scores corresponding to ACT NCRC Levels and their concorded Composite scores on the ACT with constructed-response tests in grade 11. The Platinum level was excluded due to small sample sizes.

To obtain the Progress toward Career Readiness indicator for each ACT NCRC level for grades 8, 9, and 10, the grade 11 Progress toward Career Readiness cut scores were backmapped to lower grades using z-score method, the same procedure used to backmap the ACT Readiness Benchmarks in Chapter 13. The ACT Aspire Composite score associated with each ACT NCRC level for a particular grade was the score that corresponds to the same standardized z-score of the EPAS Composite Score for each grade 11 ACT NCRC level in the sample used for the EPAS-to-ACT-Aspire concordance analysis. Grade 11 z-scores associated with Bronze, Silver, and Gold ACT NCRC levels were first computed. Then for each of grades 8-10, the ACT Aspire Composite score associated with the z-score was identified by multiplying grade specific scale score standard deviation and adding grade specific scale score mean to determine the backmapped Composite score corresponding to Bronze, Silver, and Gold level certificates. Table 15.5 presents the ACT Aspire Composite score for each of the three ACT NCRC levels for grades 8–10.

Table 15.4. Grade 11 EPAS and ACT Aspire Composite Scores Corresponding to ACT NCRC Levels

Corresponding Corresponding Composite Score on the EPAS Composite ACT with Constructed-Response Tests ACT NCRC Level Score (Grade 11)

Bronze 13 416 Silver 17 425 Gold 25 439

Table 15.5. ACT Aspire Composite Scores Corresponding to a 50% Chance of Obtaining Different ACT NCRC Levels

ACT NCRC Level Grade Bronze Silver Gold

8 415 422 434 9 415 423 436 10 416 425 439

15.5 Progress toward Career Readiness

15.5 Report Progress toward Career Readiness using ACT Aspire Cut Scores Students who take the four 8th-grade or EHS tests of English, mathematics, reading, and science obtain a Composite score and receive an indicator of their Progress toward Career Readiness (see Figure 15.1 for a sample from the score report). The student’s Composite score is listed along with the Bronze, Silver, and Gold level ACT NCRC certificates. The indicator reported to the student is a verbal statement indicating which certificate level the student is making progress toward. A purpose of the indicator is to encourage students to think about the knowledge and skills future job training will require. Whether or not a student plans to enter college immediately after high school, there is likely a need for some type of postsecondary training. It is not possible to profile the requirements for all possible postsecondary paths a student may consider, but the ACT NCRC and the ACT WorkKeys job profiles provide a starting point for students to understand that even if they are not planning to go to college, there are still knowledge and skills they need to attain before they leave high school. The specific foundational knowledge and skills required across careers and career paths differ, and ACT Aspire may not measure all such skills.

The Progress toward Career Readiness indicator provides a statistical prediction of the likely ACT NCRC level a student would obtain toward the end of high school given the student’s current performance, but caution should be used in its interpretation. The Progress toward Career Readiness indicator is not a substitute for actual ACT NCRC level obtained by taking ACT WorkKeys. Actual performance could differ from the statistically predicted performance for a variety of reasons, including such factors as statistical uncertainty in the prediction, a student’s individual educational achievement, and a student’s growth trajectory.

15.6 CHAPTER 16 ACT Aspire Reliability

16.1 Introduction This chapter presents the methods and results of ACT Aspire raw score, scale score, and composite score reliabilities and conditional standard errors of measurement.

Some degree of inconsistency or error is contained in the measurement of cognitive characteristics. A student took one form of a test on one occasion and a second, parallel form on another occasion would likely earn somewhat different scores on the two administrations. These differences might be due to the student or the testing situation, such as different motivations, different levels of distractions across occasions, or student growth between testing events. Differences across testing occasions might also be due to the particular sample of test items or prompts included on each test form. While procedures are in place to reduce differences across testing occasions, differences cannot be eliminated.

Reliability coefficients are estimates of the consistency of test scores. They typically range from zero to one, with values near one indicating greater consistency and those near zero indicating little or no consistency. The standard error of measurement (SEM) is closely related to test reliability. The SEM summarizes the amount of error or inconsistency in scores on a test. As the Standards for Educational and Psychological Testing stated: “For each total score, subscore, or combination of scores that is to be interpreted, estimates of relevant indices of reliability/precision should be reported.” (AERA, APA, & NCME, 2014, p. 43).

16.1 ACT Aspire Reliability

16.2 Raw Score Reliability Reliability coefficients are usually estimated based on a single test administration by calculating the inter-item covariances. These coefficients are referred to as internal consistency reliability. Cronbach’s coefficient alpha (Cronbach, 1951) is one of the most widely used estimates of test reliability and was computed for all of the ACT Aspire tests. Coefficient alpha can be computed using the following formula:

k k k s2 s k ∑i=1 i k ∑i=1 ∑ j≠i ij αˆ = 1− = k −1 s2 k −1 s2 x x ,

2 where k is the number of test items, si is the sample variance of the ith item, sij is the 2 sample covariance between item i and item j, and sx is the sample variance of the observed total raw score.

Although coefficient alpha is often used to estimate internal consistency reliability, it can, at times, underestimate the true value of test reliability depending on the characteristics of the specific test under consideration. In computing test reliabilities for the ACT Aspire tests, stratified coefficient alpha (Cronbach, SchÖnemann, & McKie, 1965) and congeneric reliability coefficients (Gilmer & Feldt, 1983) were also computed as a check on coefficient alpha. All three reliability coefficients produced nearly equal values for ACT Aspire subject tests at each grade level. As a consequence, Cronbach’s coefficient alpha was used in reporting raw score reliability in Tables 16.1 and 16.2.

Table 16.1. Raw and Scale Score Reliability Coefficient Ranges by Grade for Subject Tests and Composite Score from Spring 2013 Special Studies Data

Grade

Subject Score 3 4 5 6 7 8 EHS

English Raw .67–.79 .75–.80 .76–.79 .78–.81 .78–.85 .84–.87 .90–.91

Scale .70–.79 .76–.80 .77–.79 .80–.81 .79–.85 .84–.86 .90–.91

Mathematics Raw .73–.79 .55–.76 .57–.77 .61–.74 .66–.83 .84–.87 .86–.89

Scale .75–.79 .62–.75 .65–.77 .74–.77 .70–.85 .86–.89 .87–.90

Reading Raw .83–.85 .83–.84 .81–.84 .81–.84 .77–.83 .81–.86 .87–.87

Scale .83–.85 .83–.85 .81–.84 .82–.85 .79–.84 .82–.87 .88–.88

Science Raw .85–.88 .83–.84 .84–.87 .85–.88 .85–.89 .84–.88 .87–.90

Scale .86–.88 .83–.85 .84–.88 .86–.89 .85–.90 .85–.89 .86–.89

Composite* Scale .95–.96 .96–.97

* Composite scores are not reported below grade 8.

16.2 ACT Aspire Reliability

Table 16.2. Raw Score Reliability Coefficient Ranges by Grade for Subject Tests from Spring 2014 Operational Data

Grade

Subject 3 4 5 6 7 8 EHS

English .78–.82 .74–.78 .75–.78 .80–.82 .78–.83 .84–.85 .88–.90

Mathematics .78–.79 .67–.68 .67–.71 .77–.79 .81–.84 .86–.87 .82–.88

Reading .83–.85 .82–.84 .82–.84 .82–.84 .79–.80 .81–.83 .85–.87

Science .86–.89 .82–.84 .83–.84 .86–.87 .87–.89 .86–.87 .85–.89

The raw score reliabilities from the spring 2013 special studies listed in Table 16.1 for English, mathematics, reading, and science tests ranged from .55 (grade 4 mathematics) to a high of .91 (EHS English). Mathematics reliabilities tended to be quite low in some cases, particularly for grades 4–7.

The raw score reliabilities from the spring 2014 operational administration are listed in Table 16.2 for English, mathematics, reading, and science tests. Compared to 2013, reliabilities improved and ranged from .67 (grades 4 and 5 mathematics) to .90 (EHS English). Mathematics reliabilities still tended to be relatively low, specifically in grades 4 and 5.

Tests that consist of a single item typically use different types of information to estimate raw score reliability. For a test like ACT Aspire writing, which consists of a single writing prompt scored mostly by a single rater, one aspect of score reliability is rater consistency. While rater consistency should not be confused with score reliability due to task (in this case, the particular writing prompt administered), rater consistency is an important contributor to the reliability of writing prompt scores. One way to analyze rater consistency is by estimating correlations between two raters. In the spring 2013 ACT Aspire special studies, a subsample of students at each grade were rated by two raters. Table 16.3 lists the sample sizes and percentages of students in demographic groupings and Table 16.4 contains Pearson correlations between raters. The writing prompt is rated on four domains, or traits, which include Ideas and Analysis, Development and Support, Organization, and Language Use and Conventions. Scores on these domains are combined to obtain a total score.1 Correlations are reported for domain scores and total scores between raters. Writing total score correlations ranged from .70 (grade 7) to .81 grade 3 online and EHS online. Domain correlations ranged from .61 to .77.

1 For additional details, see Chapter 3, Section 3.2.3. 16.3 ACT Aspire Reliability

Table 16.3. Percentages of Students for Writing Rater Analysis in the Spring 2013 Special Studies by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (2,141) (1,902) (2,041) (2,190) (2,243) (1,701) (2,033) Gender Not Specified 7 8 6 4 5 6 4 Female 47 46 46 49 48 48 51 Male 45 46 48 47 47 46 45 Race/Ethnicity Not Specified 36 38 34 27 30 29 38 Black/African American 20 19 21 22 19 21 22 American Indian/Alaskan 1 1 1 1 1 1 1 Native White 34 34 33 31 29 31 24 Hispanic/Latino 6 7 8 16 15 15 11 Asian 2 1 2 2 2 2 3 Native Hawaiian/Other 1 1 1 1 1 0 Pacific Islander Two or More Races 1 0 1 1 3 1 1 Geographic Region East 46 51 39 34 33 40 26 Midwest 31 30 36 39 41 38 53 Southwest 17 13 18 12 15 16 15 West 6 6 7 14 11 6 6

Note. Zero indicates a percentage fewer than 1%. Missing values indicate no students. Percentages within a demographic grouping may not sum to 100% due to rounding.

16.4 ACT Aspire Reliability

Table 16.4. Writing Test Correlations between Rater 1 and Rater 2 by Domain and by Form from Spring 2013 Special Studies Data

Trait Average Correlation among the Total Score Grade N 1 2 3 4 Four Traits Correlation

3 2,141 .68 .72 .73 .66 .70 .81

4 1,902 .74 .72 .70 .71 .72 .78

5 2,041 .63 .65 .65 .69 .66 .73

6 2,190 .65 .66 .68 .65 .66 .72

7 2,243 .61 .62 .62 .65 .62 .70

8 1,701 .70 .72 .72 .65 .70 .77

EHS 2,033 .76 .76 .77 .72 .75 .81

Table 16.5. Writing Test Reliability Coefficients Based on Four Domain Scores from Spring 2013 Special Studies Data

Grade N Reliability

3 5,307 .91

4 4,709 .96

5 5,046 .95

6 5,407 .96

7 5,563 .95

8 4,717 .96

EHS 5,073 .96

Although ACT Aspire writing consists of a single writing prompt, ratings on the four domains that contribute to a final writing score can be used to obtain an internal consistency reliability estimate, like the coefficient alpha, for a writing prompt. These reliabilities are listed in Table 16.5 for writing prompts from spring 2013 and ranged from .91 to .96, which indicates that the four traits within a single writing prompt are quite reliable. However, these coefficients do not account for rater, prompt, or occasion variability, each of which is likely to be a more important contributor to the precision of the ACT Aspire writing scores than the domains within a prompt. The large internal consistency reliability based on domain scores is a reflection of the large correlations among domains within a writing prompt.

ACT Aspire also reports students' performance (percent and number of points students earn out of the total number of points possible) in each reporting category under each subject test. The total number of points possible in each reporting category may vary across forms (see Chapter 3). Raw score reliabilities (i.e., Cronbach's alpha) and standard error of measurement for each reporting category 16.5 ACT Aspire Reliability

on the forms administered in spring 2014 are reported in Appendix D. Standard error measurement is computed as raw score standard deviation multiplied by square root of one minus reliability.

16.3 Scale Score Reliability and Conditional Standard Error of Measurement The ACT Aspire scale was developed under the framework of the unidimensional item response theory (IRT) models, which involves statistical models that can be used to obtain estimate of scale score reliabilities and conditional standard errors of measurement (CSEMs).

In the unidimensional IRT models, the person trait parameter representing an examinee' proficiency is commonly denoted theta ( ). In addition to this person parameter, IRT models contain item parameters that represent particular characteristics of the items under the specific model. Additional details regarding IRT models applied to ACT Aspire can be found in Chapter 10. For more information regarding the IRT models in general, see Baker and Kim (2004), de Ayala (2009), or Yen and Fitzpatrick (2006).

To compute the CSEM for ACT Aspire scale scores, the Lord-Wingersky recursive algorithm and its extension (Lord & Wingersky, 1984; Hanson, 1994; Kolen & Brennan, 2014) were first used to estimate , , the conditional distribution of the expected raw scores X given a theta value . With a maximum number of raw score points K and the observed scale score sc(X), the true scale score

given is , and the CSEM of scale

scores given is

To estimate the scale score reliability , the conditional measurement error variance of scale scores given , is first obtained by squaring the CSEM, and the aggregate error variance can then be calculated as , where is the distribution of in the

population. Using a discrete distribution on a finite number of equally spaced points, such as quadrature points and weights to express , the aggregate error variance can be approximated by a summation form, , where q denotes the quadrature points and Q is the maximum number of quadrature points. Finally, letting be the variance of observed scale scores, the scale score

reliability can be estimated as . For more details regarding the procedures of estimating the CSEM and reliability for scale scores, see Kolen and Brennan (2014). 16.6 ACT Aspire Reliability

ACT Aspire writing scale score reliability and measurement error variance were computed based on a sample of examinees with scores from two raters in the spring 2014 operational administration. Two methods were used to compute reliability and measurement error variance, with one using the G theory method (Brennan, 2001) and the other using the Pearson correlation between scale scores from two raters as an estimate for the inter-rater reliability. Since both methods produced highly similar estimates, only G theory results are reported.

Under the G theory framework, a generalizability (G) study was designed to isolate and estimate various sources of error. Then a decision (D) study was conducted that uses variance components estimated in the G study to provide information for making decisions on the objects of measurement–examinees in this case. In the available dataset, all students took one prompt and the same prompt was given to different students in two modes: online and paper. A single-facet p x r' design was used for the G study, where p denotes examinees and r' ratings from different raters. In the p x r' design, each examinee’s response was graded by two raters, with the pair of raters possibly differing across all examinees. The variance components in this design included examinee effect, rating effect, and effects due to the interaction between examinees and ratings. Under the G theory framework, reliability is estimated using the following formula that involves absolute error variance estimated in the D study.

where 2 (Δ) is the absolute error variance and 2 (p) is variance component for examinees (Brennan, 2001, p.31).  (Δ) is reported as the standard error of measurement for the writing test.

Other than scale scores for the five subjects (English, mathematics, reading, science, and writing), ACT Aspire also reports scale scores for three types of combined scores: Composite, ELA, and STEM scores. The Composite score is computed as the average of English, mathematics, reading, and science scale scores; the ELA score is computed as the average of English, reading, and writing scale scores; and the STEM score is computed as the average of mathematics and science scale scores. Assuming that measurement error is independent across the subject tests contributing to the combined scores, the standard error of measurement for the combined scores is computed using the formula

where J is the number of subject tests contributing to the combined scores (i.e., J =4 for Composite scores; J=3 for ELA scores and J = 2 for STEM scores) and refers to the scale score measurement error variance for subject test j. Reliability for each of the combined scores is calculated using the following formula

16.7 ACT Aspire Reliability

where is the the observed score variance and is the measurement error variance of the combined score. The reliability for the combined score is expected to be higher than that for each contributing subject.

The scale score reliabilities and standard error of measurement for the five subjects, Composite, ELA, and STEM scores for the base online forms administered operationally in spring 2014 are presented in Table 16.6. Scale score reliabilities were very close to those derived in the 2013 special studies for the four subject tests which are shown below the raw score reliability in Table 16.1. English scale score reliabilities ranged from .76 (grade 4) to .89 (EHS). Mathematics scale score reliabilities ranged from .67 (grade 4) to .88 (grades 8 and EHS). Reading scale score reliabilities ranged from .81 (grade 7) to .87 (EHS). Science scale score reliabilities ranged from .84 (grade 4) to .89 (grade 7).

Scale score reliabilities are useful because they are an estimate of the precision of the scores reported to students. Raw score reliabilities are also useful for obtaining an estimate of score consistency, but raw number-of-point scores are not used for interpreting student performance on ACT Aspire. Therefore, where possible, scale score reliabilities are preferable.

Table 16.6. Scale Score Reliability Coefficient and Standard Error of Measurement by Grade and Subject from Spring 2014 Operational Data

Grade Subject 3 4 5 6 7 8 EHS English Reliability 0.79 0.76 0.78 0.82 0.82 0.84 0.89 SEM 2.67 2.97 3.24 3.36 3.70 3.53 3.32 Mathematics Reliability 0.80 0.67 0.71 0.79 0.84 0.88 0.88 SEM 1.82 2.34 2.75 2.81 2.74 2.82 2.93 Reading Reliability 0.85 0.85 0.84 0.84 0.81 0.83 0.87 SEM 2.04 2.35 2.49 2.60 2.83 2.91 2.73 Science Reliability 0.88 0.84 0.85 0.88 0.89 0.87 0.87 SEM 2.04 2.49 2.44 2.40 2.47 2.74 3.02 Writing Reliability 0.64 0.67 0.68 0.65 0.75 0.67 0.73 SEM 3.45 3.14 4.11 4.58 3.65 4.52 4.03 Composite* Reliability 0.95 0.96 SEM 1.51 1.50 ELA Reliability 0.88 0.87 0.87 0.88 0.90 0.88 0.93 SEM 1.60 1.64 1.93 2.08 1.97 2.15 1.96 STEM Reliability 0.91 0.86 0.87 0.90 0.92 0.92 0.93 SEM 1.37 1.71 1.84 1.85 1.84 1.97 2.10 Note. Composite scores were not reported below grade 8. 16.8 CHAPTER 17 ACT Aspire Validity

Validity involves collecting evidence regarding the degree to which score interpretations for proposed uses are supported. Validity evidence for ACT Aspire is organized into six areas, including content-oriented evidence, cognitive processes, internal structure, relationships to other constructs, relationships with criteria, and consequences. The validity evidence regarding each area is addressed in this chapter.

According to the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014), “Validity refers to the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (p. 11). Validation is the process of justifying a particular interpretation or use and may involve logical, empirical, or theoretical components. In the fourth edition of Educational Measurement, Kane (2006) stated that

Measurement uses limited samples of observations to draw general and abstract conclusions about persons or other units (e.g., classes, schools). To validate an interpretation or use of measurements is to evaluate the rationale, or argument, for the claims being made, and this in turn requires a clear statement of the proposed interpretations and uses and a critical evaluation of these interpretations and uses. Ultimately, the need for validation derives from the scientific and social requirement that public claims and decisions be justified (p. 17).

The potential interpretations and uses of ACT Aspire scores are numerous and diverse. Some interpretations and uses are anticipated and others are not, but each needs to be justified by a validity argument. Every interpretation and use of ACT Aspire scores cannot be fully validated in a single chapter of a technical manual. For example, evidence and information that can feed validation is contained in each artifact of technical documentation, research study, research report, and piece of

17. 1 ACT Aspire Validity

documentation related to ACT Aspire. In addition, test users may develop particular interpretations and uses that are not covered explicitly in this validity chapter or this technical manual.

The evidence summarized here should be viewed as necessary for validating interpretations of ACT Aspire scores but may not be sufficient for validating a particular interpretation for a proposed use. Test users should consider their specific interpretations and uses of ACT Aspire scores and may need to develop interpretive and/or validity arguments to support their uses. In addition, as the ACT Aspire assessments mature, validity evidence will continue to be gathered and evaluated once additional evidence becomes available and the interpretations and uses of ACT Aspire scores evolve.

In this section, two primary and three secondary intended interpretations of ACT Aspire scores are reviewed and evidence regarding the validity of ACT Aspire scores for particular interpretations is described. Validity evidence is organized into the following six areas, as described by The Standards (AERA et al., 2014): 1. content-oriented evidence, 2. evidence regarding cognitive processes, 3. evidence regarding internal structure, 4. evidence regarding relationships with conceptually related constructs, 5. evidence regarding relationships with criteria, and 6. evidence based on consequences of tests.

17.1 Interpretations of ACT Aspire Scores ACT Aspire scores include two primary interpretations and three secondary interpretations. The two primary interpretations are to identify students’ readiness on (a) a college readiness trajectory and (b) a career-readiness trajectory. The three secondary interpretations are to provide instructionally actionable information to educators, empirical data for inferences related to accountability, and empirical support for inferences about international comparisons.

Fundamental to the five interpretations of ACT Aspire scores is the assumption that scores are indicative of performance on a particular set of constructs, specifically the subject areas assessed by ACT Aspire: English, mathematics, reading, science, and writing. Scores on each subject area are intended to provide inferences about students’ knowledge and skills (achievement or performance) in these subjects. Scores obtained (or some combination of them) are then interpreted as indicators of readiness for college and career and are intended to be used to identify student status on the path to readiness for college and career. Therefore, one aspect of validation for ACT Aspire is gathering evidence for the foundational interpretation that ACT Aspire scores are indicative of academic performance in English, mathematics, reading, science, and writing. Gathering evidence to validate the five intended 17. 2 ACT Aspire Validity

interpretations of ACT Aspire scores builds on the validation of this foundational interpretation.

Some uses and interpretations of individual student scores and aggregate score results have been commonly implemented with assessments of student achievement and are likely to continue. At the student level some of these have included (a) student proficiency against pre-defined performance levels, (b) student growth across grade levels within a particular subject, (c) predicting future performance, (d) student level diagnostic information for instructional purposes, and (e) ranking students (e.g., for selection into special programs). Some uses and interpretations of aggregate scores at the group level have included (a) school and district accountability (e.g., percentage of students above a “satisfactory” cut score), (b) school, district, and classroom growth, and (c) evaluating the impact of the curriculum or special programs. These are examples of additional interpretations and uses that may be implemented with ACT Aspire scores. Some may closely align with the primary and secondary intended interpretations of ACT Aspire scores (e.g., predicting future performance, instructionally actionable information, etc.), and others may not (e.g., using scores for evaluating the impact of curriculum). Each interpretation and use should be carefully evaluated against the available evidence.

17.2 Validity Evidence 17.2.1 Content-Oriented Evidence Content-oriented evidence for validity is based on the appropriateness of test content and the procedures used to specify and create test content. For the ACT Aspire forms, this evidence is based on the justifications for and the connections between • the content domain, • the knowledge and skills implied by the ACT Aspire content specifications, • the development of items, • the development of forms, • test administration conditions, and • item and test scoring.

Most of the descriptions and justifications for content were provided in earlier chapters, but a brief review is included below.

The content domains for ACT Aspire include (a) English, reading, and writing (ELA) and (b) mathematics and science (STEM) across grades 3-10. These content domains include knowledge and skills needed by students for success in school (i.e., academic achievement) and for success in college and career. ACT Aspire test forms in a particular subject area include samples of test items intended to represent content from the domain implied by a subject area (see Chapter 2).

17. 3 ACT Aspire Validity

In addition, the ACT College and Career Readiness Standards (grades 8 and above) and the ACT Readiness Standards (grades 3-7) provide statements of what students should know and be able to do to be on-track to attaining college and career readiness in English, mathematics, reading, science, and writing at each grade level. These statements were developed by content and measurement experts based on research that included the ACT National Curriculum Survey. These standards are consistent with many states’ standards that focus on college and career readiness (see Chapter 3).

ACT Aspire content specifications identify the number, type, and depth-of-knowledge (DOK) level of items required when each form was developed. These specifications determine the type and amount of evidence obtained within the content domain. The extent to which the type and selection of content included in the content specifications represent the content domain determines how representative test form content is of the intended content domain (see Chapter 2).

Item writers are carefully chosen from a pool of qualified educators and follow standardized procedures and templates when creating items. All items undergo rigorous internal reviews and external audits to ensure that they are accurate, appropriate, and fair for the intended population of students. Items are pretested with students drawn from the intended ACT Aspire population to verify item quality and item characteristics (see Chapters 2 and 4).

Initial ACT Aspire forms are developed to follow the content specifications. Initial forms are reviewed by content experts to determine the accuracy and appropriateness of the collection of items. If experts determine that adjustments should be made to the initial test forms, items may be swapped that maintain consistency with the content specifications. A form is finalized once it passes these reviews (see Chapter 2).

ACT Aspire is a standardized test administered on paper or online under time limits. The standardization of the ACT Aspire test administration is intended to ensure that test administration characteristics do not contribute adversely to some students’ performance compared to others. Documentation of test administration practices is included in the ACT Aspire Room Supervisor Manual. However, where possible, students are allowed some flexibility in the test administration to participate in a way that is consistent with their needs but that does erode the intended and relevant characteristics of the test or test administration. In addition, a standardized process is followed for documenting and provided students who need particularly strong supports, such as testing accommodations, the flexibility they need. See Chapter 5 for additional details.

ACT Aspire standardized test administrations on paper and online are established to be as similar as possible. However, test administration conditions are different because the mode of administration is different. ACT Aspire forms across mode are not assumed to be the same, in part due to potential differences in performance

17. 4 ACT Aspire Validity

due to mode and the availability of technology-enhanced items only in online tests. Chapter 19 addresses performance differences across online and paper forms and Chapter 20 addresses how scores are equated across online and paper forms so that scale scores are reported on the ACT Aspire scale. Treating forms across mode separately when linking them to the ACT Aspire scale helps ensure that scores from online forms and paper forms are comparable.

Descriptions of item scoring procedures are included in Chapter 4 and test scoring procedures are included in Chapter 9. ACT Aspire item and test scoring is intended to be applied uniformly across students and forms so that scores consistently reflect student performance across various student, item, and test characteristics. For example, rater training and quality control procedures for scoring constructed response items are explained in Chapters 2 and 4.

17.2.2 Evidence Regarding Cognitive Processes Evidence regarding cognitive processes includes verifying that an intended interpretation assuming a particular psychological process or cognitive operation is indeed used by test takers. Two sources of validity evidence are commonly used to evaluate cognitive processes. The first is to use think aloud or cognitive lab procedures, where students say what they are thinking while working on test materials. This procedure is used to gather evidence that the intended cognitive processes for a test item are consistent with the processes that students verbalize that they use to respond to an item. These procedures are applicable to a variety of artifacts collected and scored as part of an assessment, but can be particularly helpful for evaluating artifacts that require students to construct responses, which often requires stronger assumptions about the cognitive processes used to provide a response. The second source of evidence commonly used to evaluate cognitive processes applies to assessments where student responses are rated. In this case, the extent to which the raters accurately interpret and apply scoring rubrics, rules, and/or criteria contributes to evidence regarding cognitive processes.

As part of piloting constructed response (CR) items for ACT Aspire in 2012 and 2013, students at grades 8, 10, and 11 from several schools were recruited to respond to two or three CR tasks in mathematics or science. Students completed tasks in pairs or small groups using a think aloud protocol that involved discussing with their partners/group their strategies and reasoning processes as they completed the CR items. After completing items, students were asked to fill out a brief survey (e.g., what the student liked, disliked, and found confusing about the items) and engage in a brief group interview regarding their experiences with the items.

These pilots helped verify that ACT Aspire CR items required students to draw on the intended knowledge and skills to produce their answer. Student responses from think aloud tasks, surveys, and interviews indicated that the CR items required the intended knowledge and skills for students to supply an answer. In some cases, students found

17. 5 ACT Aspire Validity

tasks challenging, but it was apparent that responding to items required students to use intended knowledge and skills in mathematics or science to respond to each CR item.

Another source of evidence regarding cognitive processes can be found in Chapters 2 and 4 describing the training of raters and scoring of items. This process involves carefully defined procedures, checks, and reviews to ensure the quality and validity of scores from ratings obtained for constructed response items. The procedures and processes include checks of the validity of scores during the scoring process, where initial scores may be rejected if they do not meet quality checks. In addition, during the process of rubric development and preliminary application of the rubric to student responses, the coherence and consistency of rubrics are considered against the content intended to be measured by each item to help ensure that the rubric leads to ratings that indeed reflect the intended knowledge and skills for the item.

17.2.3 Evidence Regarding Internal Structure Evidence regarding internal structure is based on evaluating intended score interpretations from the perspective of expected characteristics among test items or parts of the test. ACT Aspire is intended to measure academic achievement in English, mathematics, reading, science, and writing. In addition, Composite scores1, ELA scores, and STEM scores are reported for students. Indicators of on-track to college readiness and on-track to career readiness are also provided to students along with additional scores describing the percentages of points obtained in reporting categories (a sub-set of items within a subject area) and an indicator regarding progress with text complexity.

Dimensionality. The expectation for scores in English, mathematics, reading, science, and writing, is that they are reasonably essentially unidimensional. Even though reporting category scores (in percentage of points correct) are included in ACT Aspire reports, which would suggest distinct sub-dimensions within a subject area, ACT Aspire was developed to measure academic achievement in English, mathematics, reading, science, and writing, so a dominant dimension would be expected in empirical analysis of dimensionality.

A study of the dimensionality was conducted to evaluate the extent to which ACT Aspire item performance appeared to be best represented by a single dimension. Analysis included conducting an exploratory factor analysis (EFA)2, which is often used to evaluate the dimensionality of performance on assessments like ACT Aspire (e.g., Sireci, 2009, p. 30). Data included item scores from students participating in the spring 2013 comparability study3 in English, mathematics, reading, and science4.

1 Grade 8 and above only. 2 Mplus software (Muthén & Muthén, 2008) was used to conduct the exploratory factor analysis on the inter-item correlation matrices. Settings used to extract factors included weighted least squares estimation and the Geomin (oblique) factor rotation. 3 For additional details regarding the comparability study, see Chapter 19. 17. 6 4 Writing was excluded from the dimensionality analysis because the writing test consists of a single writing prompt. ACT Aspire Validity

Scree plots of eigenvalues, model fit, and factor loadings were examined to evaluate dimensionality.

Scree plots display factor analysis eigenvalues against the number of extracted factors, and evaluation of scree plots involves identifying the “elbow” in the plot to indicate the number of dimensions to retain (Cattell, 1966). The scree plots for each subject listed in Figure 17.1 show that for all reading forms and the majority of the English, mathematics and science forms, the “elbow” appears after the first eigenvalue, which is evidence for a single dimension. For a few English forms (paper forms for grades 3 and 6), mathematics forms (paper forms for grade 4 and 6), and science (paper forms for grade 3, 4, and 6), the “elbow” appears after the second eigenvalue indicating that a two-factor model might fit the data better. Tables 17.1- 17.4 show the eigenvalue and the proportion of accounted variance for each factor in forms from different subjects. As shown in these tables, the percentage of variance accounted for by the second factor for these forms did not exceed 10% (except in two cases where the percentages were 10.1 and 10.3). Hatcher (1994) suggested that factors accounting for less than 10% of the variance should not be retained, so we could argue for not retaining second factors based on a combination of scree plots and percentage of variance accounted for. The scree plots generally showed evidence that item performance was best represented by a unidimensional model although in some cases, two factors may be better represent the data.

Model fit was also evaluated. Based on the scree plot results, which indicated that up to two factors may be justifiable for data in some subjects at certain grade levels, model fit was evaluated by comparing one-factor and two-factor models for each subject and grade-level to determine whether the one-factor or two-factor model appeared to better represent students’ item-level performance on ACT Aspire assessments. The goodness-of-fit for each model was examined using various model fit statistics. The fit statistics included the chi-square statistic (CHISQ) and several other fit statistics, including (along with cutoff values) comparative fit index (CFI; .95), Tucker-Lewis index (TLI; .95), root mean square error of approximation (RMSEA; .06), and standardized root mean square residual (SRMR; .08) (see Hu and Bentler, 2009). The difference in chi-square statistics between one- and two-factor models was also evaluated using a statistical significance test. The correlation between Factors 1 and 2 was used to evaluate how well the two factors, when modeled, could be distinguished from each other. Fit indices were evaluated considering the goodness of fit of the one-factor and two-factor models.

Tables 17.5–17.8 display the fit statistics for the one-factor and two-factor models. The statistical significance of the change in chi-square statistics between the one- and two-factor models for each grade and subject is also included in the tables (i.e. the rows associated with p-value). These chi-square difference tests are statistically significant for all forms, as indicated by the * in the rows associated with chi-square although this is likely due to the well-known sensitivity of chi-square (e.g., Bollen,

17.7 ACT Aspire Validity

1989), where relatively small changes in chi-square statistics are identified as statistically significant when sample sizes are large. For this reason, dimensionality analysis also typically includes other fit indices. Fit indices were flagged with a * if they showed inadequate fit (i.e., the CFI, TLI, RMSEA, SRMR rows). Forms that showed inadequate fit with the one-factor model generally showed adequate fit with the two-factor model.

Fit statistics for the English, mathematics, reading, and science showed evidence for unidimensionality in most cases; the two-factor model did not generally show meaningful improvement in model fit compared to the one-factor model. The most noticeable improvement in fit for two-factor models was observed for mathematics grade 4 paper, mathematics grade 5 paper, and mathematics grade 6 online and paper. Interestingly, for these forms the inter-factor correlations were relatively small compared to many of the other forms, indicating that the second factor for these forms, if meaningful, appeared to be measuring something different than the first factor.

The factor loadings were examined to investigate the substantive meaning of the constructs defined by the two factors. Factor loadings are of particular interest for data from forms showing noticeable improvement in fit under the two-factor model because the patterns of loadings across Factor 1 and Factor 2 may provide us with an idea of what each factor appears to be measuring (e.g., Gorsuch, 1983 , pp. 206- 212). The loadings for the two-factor model are shown in Table 17.9 for the forms that showed particularly strong evidence of improvement in fit between the one- and two-factor models (i.e., mathematics grade 4 paper, mathematics grade 5 paper, and mathematics grade 6 online and paper). Given that the items with the largest loadings on the second factor tended to occur toward the end of the test, these data may have exhibited a degree of speededness. This pattern was particularly evident for mathematics grades 4 , 5, and 6 paper forms although the last two items (and other items in the second half of the test) on the grade 6 online form also had relatively large loadings on a second factor5.

The exploratory factor analysis results provided evidence for unidimensionality with most of the ACT Aspire data. However, grades 4 and 6 mathematics in particular may require monitoring to determine whether a second dimension is observed for future administrations of ACT Aspire.

Relationships among scores. Because English, reading, and writing scale scores are combined to obtain ELA scores and mathematics and science are combined to obtain STEM scores, the expectation would be to observe larger correlations among scores

5 While grade 4 online did not show particularly strong evidence for a two dimensional solution, the factor loadings under a two-factor solution showed a similar pattern, with the last few items having strong positive loadings on a second factor.

17. 8 ACT Aspire Validity

contributing to ELA and STEM than across these reported scores. For example, the expectation would be that English and reading scores are more highly correlated than mathematics and reading.

Table 17.10 lists the means, standard deviations, and sample sizes for English, mathematics, reading, science, writing scale scores, and those combined scores from all data available from spring 2013 ACT Aspire special studies. Table 17.11 lists the correlations among ACT Aspire scores for students taking English, mathematics, reading, science, and writing tests by grade level. Patterns of correlations at each grade are generally consistent with the expectation that English, reading, writing and ELA show higher correlations among each other and mathematics, science, and STEM show higher correlations among each other with two exceptions. In grades 3, 5, and 6, English and science had correlations similar to or higher than the correlation between English and reading. Also, in grades 3 to 9, the correlation between science and reading was the same as or greater than the correlation between science and mathematics.

The pattern of relationships among scores is generally consistent with the interpretations of scores reflecting performance in English, mathematics, reading, science, writing, ELA, and STEM. In particular, the amount of reading required of the science test can be explained by the relatively strong relationships between (a) science and English and (b) science and reading.

17. 9 ACT Aspire Validity

Figure 17.1. Scree plots of ACT Aspire online and paper forms by subject and grade using data from the spring 2013 comparability study

English Grade 3 English Grade 4

English Grade 5 English Grade 6

17. 10 ACT Aspire Validity

English Grade 7 English Grade 8

English Early High School

17.11 ACT Aspire Validity

Mathematics Grade 3 Mathematics Grade 4

Mathematics Grade 5 Mathematics Grade 6

17.12 ACT Aspire Validity

Mathematics Grade 7 Mathematics Grade 8

Mathematics Early High School

17.13 ACT Aspire Validity

Reading Grade 3 Reading Grade 4

Reading Grade 5 Reading Grade 6

17.14 ACT Aspire Validity

Reading Grade 7 Reading Grade 8

Reading Early High School

17.15 ACT Aspire Validity

Science Grade 3 Science Grade 4

Science Grade 5 Science Grade 6

17.16 ACT Aspire Validity

Science Grade 7 Science Grade 8

Science Early High School

17.17 ACT Aspire Validity

Table 17.1. English ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentage of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study % 5.6 3.7 3.2 2.6 2.5 2.4 2.3 2.3 2.1 2.0 1.9 1.8 1.8 1.8 1.7 1.6 1.6 1.6 1.6 1.5 1.5 1.4 1.3 1.3 28.3 Paper 2.8 1.9 1.6 1.3 1.2 1.2 1.1 1.1 1.1 1.0 1.0 0.9 0.9 0.9 0.9 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.6 Eig. 14.2 (N=1,209) % 4.6 3.3 2.9 2.5 2.4 2.4 2.3 2.3 2.2 2.0 2.0 1.9 1.9 1.8 1.7 1.7 1.6 1.6 1.6 1.5 1.4 1.4 1.4 1.3 28.9 Online Early High School 2.3 1.7 1.5 1.3 1.2 1.2 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.7 Eig. 14.4 (N=1,176) % 5.2 5.0 3.7 3.5 3.5 3.2 3.1 2.9 2.8 2.7 2.5 2.5 2.3 2.1 2.1 1.9 1.9 1.8 1.7 1.6 1.5 1.5 1.4 1.4 30.9 Paper (N=887) 1.8 1.7 1.3 1.2 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 Eig. 10.8 % Grade 8 5.9 4.9 4.0 3.7 3.4 3.2 3.1 3.0 2.9 2.7 2.7 2.5 2.5 2.3 2.2 2.1 2.0 1.9 1.8 1.7 1.6 1.5 1.5 1.4 26.8 Online (N=866) 9.4 2.1 1.7 1.4 1.3 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 Eig. % 6.2 4.3 4.2 4.0 3.5 3.4 3.2 3.1 3.0 2.8 2.7 2.6 2.6 2.5 2.4 2.3 2.2 2.1 1.9 1.8 1.8 1.7 1.6 1.5 21.9 Paper 7.7 2.2 1.5 1.5 1.4 1.2 1.2 1.1 1.1 1.0 1.0 0.9 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.5 Eig. (N=1,033) % Grade 7 5.5 4.4 3.9 3.6 3.5 3.4 3.1 2.9 2.9 2.9 2.7 2.6 2.5 2.4 2.3 2.2 2.2 2.1 2.0 1.9 1.8 1.8 1.7 1.6 22.9 Online 8.0 1.9 1.5 1.4 1.3 1.2 1.2 1.1 1.0 1.0 1.0 1.0 0.9 0.9 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 Eig. (N=1,019) % 8.3 4.9 4.0 3.8 3.5 3.1 3.1 3.0 2.9 2.8 2.6 2.6 2.5 2.4 2.3 2.2 2.1 2.0 1.9 1.8 1.8 1.7 1.6 1.4 21.7 Paper 7.6 2.9 1.7 1.4 1.3 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 Eig. (N=1,166) % Grade 6 6.4 4.1 4.0 3.6 3.4 3.3 3.1 3.1 2.9 2.7 2.7 2.6 2.6 2.4 2.3 2.2 2.2 2.1 1.9 1.8 1.7 1.7 1.6 1.6 22.3 Online 7.8 2.3 1.4 1.4 1.3 1.2 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.6 Eig (N=1,129) % 7.9 5.3 4.4 4.3 4.0 3.8 3.7 3.6 3.3 3.2 3.1 2.9 2.8 2.6 2.5 2.4 2.1 2.1 2.0 1.8 1.5 1.3 1.2 1.1 27.1 Paper 6.8 2.0 1.3 1.1 1.1 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.3 0.3 0.3 Eig. (N=1,346) % Grade 5 6.8 5.6 4.8 4.1 4.0 3.9 3.7 3.6 3.2 3.1 2.9 2.8 2.7 2.6 2.5 2.3 2.2 2.1 2.0 1.8 1.7 1.6 1.0 0.6 28.0 Online 7.0 1.7 1.4 1.2 1.0 1.0 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.3 0.2 Eig. (N=1,349) % 7.7 5.3 5.0 4.4 3.9 3.8 3.6 3.5 3.3 3.1 3.0 2.9 2.7 2.7 2.4 2.4 2.3 2.1 1.9 1.8 1.7 1.6 1.3 0.6 27.1 Paper 6.8 1.9 1.3 1.2 1.1 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.3 0.1 Eig. (N=1,489) % Grade 4 6.4 5.4 4.7 4.5 4.2 4.2 3.9 3.8 3.4 3.3 3.2 3.0 2.9 2.8 2.6 2.5 2.4 2.3 2.2 1.9 1.8 1.7 1.3 1.0 24.6 Online 6.2 1.6 1.3 1.2 1.1 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.3 0.2 Eig. (N=1,480) % 5.2 4.9 4.3 4.2 3.9 3.7 3.4 3.3 3.2 3.1 3.0 2.9 2.5 2.4 2.3 2.2 1.9 1.7 1.6 1.5 1.4 1.0 0.7 25.4 10.1 Paper 6.4 2.5 1.3 1.2 1.1 1.1 1.0 0.9 0.8 0.8 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.2 Eig. (N=1,405) % Grade 3 5.7 5.7 4.8 4.6 4.2 4.0 3.8 3.6 3.5 3.3 3.1 3.0 2.9 2.7 2.5 2.5 2.3 2.1 1.9 1.8 1.7 1.4 1.2 1.1 26.5 Online 6.6 1.4 1.4 1.2 1.1 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.3 0.3 Eig. (N=1,359) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Factor 17.18 ACT Aspire Validity

Table 17.1. (continued) % 1.2 1.2 1.2 1.2 1.1 1.1 1.1 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.5 0.4 0.4 0.4 0.4 Paper 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.3 0.2 0.2 0.2 0.2 Eig. (N=1,209) % 1.3 1.2 1.2 1.2 1.2 1.1 1.1 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 Online Early High School 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.3 0.3 0.2 0.2 0.2 Eig. (N=1,176) % 1.2 1.1 0.9 0.8 0.8 0.6 0.6 0.5 0.4 0.4 Paper (N=887) 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.1 Eig. % Grade 8 1.3 1.3 1.1 1.0 1.0 0.8 0.8 0.6 0.5 0.4 Online (N=866) 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.1 Eig. % 1.5 1.4 1.4 1.3 1.1 1.1 1.0 0.8 0.8 0.6 Paper 0.5 0.5 0.5 0.5 0.4 0.4 0.3 0.3 0.3 0.2 Eig. (N=1,033) % Grade 7 1.4 1.4 1.3 1.3 1.2 1.1 1.0 0.9 0.8 0.6 Online 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.2 Eig. (N=1,019) % 1.4 1.3 1.3 1.2 1.2 1.1 0.9 0.8 0.7 0.4 Paper 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.1 Eig. (N=1,166) % Grade 6 1.5 1.4 1.3 1.3 1.2 1.1 1.0 0.9 0.7 0.6 Online 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.3 0.3 0.2 Eig (N=1,129) % Paper Eig. (N=1,346) % Grade 5 Online Eig. (N=1,349) % Paper Eig. (N=1,489) % Grade 4 Online Eig. (N=1,480) % Paper Eig. (N=1,405) % Grade 3 Online Eig. (N=1,359) Eig. denotes eigenvalue. 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 Factor Note. 17.19 ACT Aspire Validity

Table 17.2. Mathematics ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentages of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study % 5.4 3.6 3.4 3.1 3.0 2.8 2.8 2.8 2.6 2.6 2.4 2.3 2.2 2.2 2.1 2.0 1.9 1.8 1.8 1.7 1.7 1.6 1.4 1.4 28.3 Paper 2.0 1.4 1.3 1.2 1.1 1.1 1.1 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 Eig. 10.7 (N=1,098) % 4.7 3.3 3.3 3.1 2.9 2.8 2.7 2.6 2.6 2.4 2.4 2.3 2.3 2.1 1.9 1.9 1.8 1.8 1.6 1.6 1.5 1.4 1.4 1.3 31.8 Online Early High School 1.8 1.3 1.2 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.9 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 Eig. 12.1 (N=1,075) % 4.8 4.7 3.4 3.3 3.2 3.2 3.0 2.8 2.6 2.4 2.4 2.3 2.2 2.1 2.1 1.9 1.8 1.8 1.6 1.6 1.6 1.5 1.4 1.4 29.4 Paper (N=927) 1.8 1.8 1.3 1.3 1.2 1.2 1.1 1.1 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 Eig. 11.2 % Grade 8 5.0 4.0 3.8 3.3 3.3 3.1 2.9 2.7 2.6 2.5 2.4 2.4 2.3 2.1 2.0 2.0 1.9 1.7 1.7 1.7 1.6 1.5 1.5 1.4 29.7 Online (N=896) 1.9 1.5 1.4 1.3 1.2 1.2 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.5 Eig. 11.3 % 7.6 4.4 4.0 3.7 3.4 3.4 3.3 3.0 3.0 2.8 2.8 2.5 2.5 2.4 2.2 2.2 2.2 2.0 1.9 1.8 1.7 1.7 1.6 1.5 22.6 Paper 7.7 2.6 1.5 1.4 1.3 1.2 1.2 1.1 1.0 1.0 1.0 1.0 0.9 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 Eig. (N=1,010) % Grade 7 5.4 4.3 4.0 4.0 3.7 3.3 3.1 3.0 3.0 2.9 2.7 2.7 2.6 2.4 2.2 2.2 2.1 2.0 1.9 1.8 1.7 1.6 1.5 1.4 25.1 Online (N=974) 8.5 1.8 1.5 1.4 1.4 1.3 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 Eig. % 9.7 5.1 4.3 3.8 3.5 3.5 3.3 3.2 3.2 2.9 2.9 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2.0 1.9 1.9 1.8 1.8 1.6 16.5 Paper 5.6 3.3 1.7 1.5 1.3 1.2 1.2 1.1 1.1 1.1 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 Eig. (N=1,194) % Grade 6 7.3 4.8 4.1 4.0 3.6 3.5 3.2 3.1 3.0 2.9 2.9 2.8 2.6 2.5 2.4 2.3 2.2 2.2 2.2 2.0 1.9 1.9 1.8 1.6 19.5 Online 6.6 2.5 1.6 1.4 1.4 1.2 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 Eig. (N=1,156) % 7.1 6.5 5.2 4.9 4.6 4.5 4.2 4.0 3.8 3.8 3.6 3.2 3.1 3.0 3.0 2.9 2.7 2.5 2.4 2.2 2.1 1.8 1.7 1.0 16.1 Paper 4.0 1.8 1.6 1.3 1.2 1.1 1.1 1.1 1.0 1.0 1.0 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.4 0.3 Eig. (N=1,427) % Grade 5 6.4 5.6 4.8 4.7 4.6 4.3 4.2 4.0 3.8 3.6 3.3 3.2 3.1 3.0 2.8 2.7 2.7 2.6 2.4 2.2 2.0 2.0 1.8 1.6 18.5 Online 4.6 1.6 1.4 1.2 1.2 1.1 1.1 1.1 1.0 0.9 0.9 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.5 0.5 0.4 Eig. (N=1,450) % 8.8 5.7 5.2 5.0 4.4 4.2 4.2 4.0 3.9 3.7 3.6 3.5 3.2 3.0 2.8 2.6 2.6 2.5 2.4 2.3 2.2 2.0 1.7 1.4 15.0 Paper 3.8 2.2 1.4 1.3 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.4 0.4 Eig. (N=1,532), % Grade 4 6.4 5.5 5.0 4.8 4.6 4.5 4.4 4.2 4.0 3.8 3.7 3.4 3.3 3.1 2.8 2.7 2.6 2.5 2.3 2.2 2.1 1.9 1.7 0.6 17.7 Online 4.4 1.6 1.4 1.3 1.2 1.2 1.1 1.1 1.1 1.0 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.4 0.1 Eig. (N=1,534) % 8.1 5.0 4.5 4.5 4.3 4.2 4.0 3.7 3.6 3.4 3.2 3.0 3.0 2.7 2.6 2.5 2.4 2.2 2.0 1.8 1.8 1.5 1.5 1.2 23.1 Paper 5.8 2.0 1.3 1.1 1.1 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 Eig. (N=1,539) % Grade 3 6.0 5.2 4.6 4.5 4.3 4.2 3.9 3.8 3.5 3.4 3.2 3.1 2.9 2.8 2.7 2.5 2.4 2.2 2.1 1.8 1.7 1.6 1.2 0.8 25.3 Online 6.3 1.5 1.3 1.1 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.2 Eig. (N=1,488) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Factor 17.20 ACT Aspire Validity

Table 17.2. (continued) % 1.4 1.3 1.3 1.2 1.1 1.1 1.0 0.9 0.9 0.8 0.8 0.7 0.6 Paper 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.2 Eig. (N=1,098) % 1.3 1.3 1.2 1.2 1.1 1.0 1.0 0.9 0.9 0.8 0.7 0.5 0.4 Online Early High School 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 Eig. (N=1,075) % 1.3 1.2 1.2 1.1 1.0 0.9 0.9 0.9 0.8 0.7 0.7 0.5 0.4 Paper (N=927) 0.5 0.5 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.3 0.2 0.2 Eig. % Grade 8 1.3 1.2 1.1 1.1 1.0 0.9 0.8 0.7 0.7 0.7 0.5 0.4 0.3 Online (N=896) 0.5 0.5 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 Eig. % 1.4 1.4 1.3 1.3 1.1 0.9 0.9 0.8 0.6 Paper 0.5 0.5 0.5 0.4 0.4 0.3 0.3 0.3 0.2 Eig. (N=1,010) % Grade 7 1.4 1.3 1.2 1.2 1.0 0.9 0.9 0.8 0.6 Online (N=974) 0.5 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.2 Eig. % 1.5 1.4 1.3 1.3 1.1 1.1 0.9 0.8 0.6 Paper 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.2 Eig. (N=1,194) % Grade 6 1.5 1.5 1.4 1.3 1.2 1.1 0.9 0.8 0.2 Online 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.1 Eig. (N=1,156) % Paper Eig. (N=1,427) % Grade 5 Online Eig. (N=1,450) % Paper Eig. (N=1,532), % Grade 4 Online Eig. (N=1,534) % Paper Eig. (N=1,539) % Grade 3 Online Eig. (N=1,488) Eig. denotes eigenvalue. 26 27 28 29 30 31 32 33 34 35 36 37 38 Factor Note. 17.21 ACT Aspire Validity

Table 17.3. Reading ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentages of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study % 5.8 4.6 4.0 3.6 3.4 3.2 3.2 3.1 2.6 2.5 2.4 2.2 2.1 2.0 1.8 1.8 1.5 1.4 1.2 1.1 1.0 0.8 0.6 44.0 Paper (N=919) 1.4 1.1 1.0 0.9 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 Eig. 10.6 % 5.4 4.5 4.1 3.9 3.8 3.5 3.1 2.9 2.8 2.8 2.5 2.3 2.2 2.1 2.0 1.7 1.7 1.5 1.4 1.3 1.2 1.0 0.9 41.6 Online Early High School (N=857) 1.3 1.1 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 Eig. 10.0 % 6.8 4.7 4.3 4.3 3.7 3.5 3.4 3.1 2.8 2.6 2.6 2.4 2.3 2.1 1.8 1.7 1.6 1.5 1.4 1.2 1.1 1.1 0.7 39.2 Paper (N=831) 9.4 1.6 1.1 1.0 1.0 0.9 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.5 0.5 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.2 Eig. % Grade 8 6.4 5.0 4.8 4.5 4.0 3.7 3.4 3.3 3.3 2.7 2.7 2.6 2.3 2.2 2.0 1.9 1.6 1.6 1.5 1.3 1.0 0.5 0.1 37.6 Online (N=800) 9.0 1.5 1.2 1.1 1.1 1.0 0.9 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.2 0.1 0.0 Eig. % 5.5 5.1 4.7 4.4 4.2 3.9 3.9 3.5 3.4 3.3 3.0 3.0 2.8 2.6 2.5 2.3 2.2 1.9 1.8 1.7 1.5 1.3 1.1 30.4 Paper 7.3 1.3 1.2 1.1 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.4 0.4 0.3 0.3 Eig. (N=1,037) % Grade 7 5.3 5.0 4.7 4.3 4.1 3.9 3.6 3.4 3.2 3.0 3.0 2.9 2.7 2.7 2.5 2.5 2.2 2.2 2.1 1.9 1.8 1.4 1.1 30.4 Online 7.3 1.3 1.2 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.3 0.3 Eig. (N=1,010) % 6.2 5.2 4.9 4.7 4.5 4.2 3.9 3.6 3.5 3.3 3.2 2.8 2.6 2.5 2.3 2.2 2.0 1.9 1.7 1.6 1.5 1.3 30.4 Paper 7.0 1.4 1.2 1.1 1.1 1.0 1.0 0.9 0.8 0.8 0.8 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 Eig. (N=1,038) % Grade 6 5.4 4.8 4.7 4.4 4.1 4.0 3.9 3.6 3.4 3.1 3.1 2.9 2.9 2.7 2.5 2.3 2.2 2.2 1.8 1.6 1.5 1.2 31.8 Online 7.3 1.2 1.1 1.1 1.0 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.3 0.3 Eig. (N=1,004) % 6.0 4.8 4.2 4.1 3.9 3.6 3.4 3.4 3.1 2.9 2.9 2.8 2.6 2.5 2.3 2.2 2.1 2.0 2.0 1.8 1.7 1.4 1.3 33.0 Paper 7.9 1.4 1.2 1.0 1.0 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.4 0.3 0.3 Eig. (N=1,294) % Grade 5 5.6 4.9 4.3 4.2 3.9 3.7 3.6 3.3 3.1 3.0 3.0 2.7 2.6 2.5 2.3 2.2 2.1 2.1 1.9 1.8 1.8 1.4 1.3 32.8 Online 7.9 1.4 1.2 1.0 1.0 0.9 0.9 0.9 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.4 0.3 0.3 Eig. (N=1,299) % 5.6 4.6 4.1 4.1 3.9 3.7 3.6 3.3 3.2 3.0 2.9 2.8 2.6 2.4 2.3 2.2 2.1 2.1 1.9 1.8 1.5 1.4 1.3 33.6 Paper 8.1 1.3 1.1 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.4 0.3 0.3 Eig. (N=1,401) % Grade 4 5.4 4.5 4.3 4.0 3.8 3.7 3.6 3.3 3.1 3.0 2.9 2.7 2.7 2.6 2.5 2.4 2.3 2.2 2.0 1.9 1.7 1.6 1.6 32.1 Online 7.7 1.3 1.1 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.4 0.4 Eig. (N=1,399) % 7.1 4.7 4.4 4.1 3.8 3.7 3.5 3.4 3.1 3.0 2.8 2.7 2.4 2.4 2.3 2.2 2.1 1.9 1.8 1.6 1.5 1.2 0.9 33.3 Paper 8.0 1.7 1.1 1.1 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.2 Eig. (N=1,513) % Grade 3 5.5 4.3 4.2 4.0 3.9 3.7 3.5 3.2 3.2 3.1 3.0 2.9 2.7 2.5 2.4 2.3 2.1 2.0 1.9 1.7 1.6 1.5 1.3 33.5 Online 8.0 1.3 1.0 1.0 1.0 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.3 Eig. (N=1,441) Eig. denotes eigenvalue. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Factor 17.22 Note. ACT Aspire Validity

Table 17.4. Science ACT Aspire Online and Paper Mode Exploratory Factor Analysis Eigenvalues and Percentages of Variance Explained, by Grade Level Using Data from the Spring 2013 Comparability Study % 6.3 4.1 4.0 3.6 3.4 3.2 3.0 2.9 2.9 2.6 2.5 2.4 2.3 2.2 2.0 2.0 1.9 1.8 1.7 1.6 1.5 32.3 Paper 2.0 1.3 1.3 1.2 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 Eig. (N=1,023) 10.3 % 5.5 4.0 3.7 3.6 3.3 3.2 3.1 2.9 2.8 2.7 2.5 2.5 2.4 2.3 2.3 2.1 2.0 1.8 1.8 1.7 1.7 31.7 Early High School Online (N=959) 1.7 1.3 1.2 1.1 1.1 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 Eig. 10.2 % 6.8 4.2 3.9 3.5 3.3 3.2 3.0 2.9 2.6 2.6 2.5 2.3 2.2 2.1 2.0 1.9 1.8 1.7 1.6 1.4 1.3 34.5 Paper (N=934) 2.2 1.3 1.3 1.1 1.1 1.0 1.0 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.4 Eig. 11.0 % Grade 8 6.1 4.0 3.9 3.4 3.3 3.2 3.0 2.9 2.7 2.6 2.5 2.4 2.3 2.1 2.0 1.9 1.8 1.8 1.7 1.5 1.5 34.1 Online (N=913) 1.9 1.3 1.3 1.1 1.1 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.5 0.5 Eig. 10.9 % 7.1 4.5 3.7 3.4 3.3 3.1 2.9 2.7 2.4 2.3 2.2 2.1 2.0 1.9 1.9 1.8 1.7 1.7 1.6 1.5 1.4 35.5 Paper 2.3 1.5 1.2 1.1 1.0 1.0 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 Eig. (N=1,042) 11.4 % Grade 7 4.8 4.3 3.7 3.5 3.3 3.1 2.8 2.7 2.5 2.3 2.3 2.3 2.1 2.1 1.9 1.9 1.9 1.8 1.7 1.5 1.5 37.7 Online 1.5 1.4 1.2 1.1 1.1 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.5 Eig. (N=1,028) 12.1 % 8.7 4.2 3.9 3.5 3.1 3.0 2.8 2.6 2.6 2.3 2.2 2.2 2.1 1.9 1.9 1.8 1.8 1.7 1.5 1.5 1.4 34.1 Paper 2.8 1.3 1.2 1.1 1.0 1.0 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.4 Eig. (N=1,040) 10.9 % Grade 6 5.4 4.4 3.6 3.5 3.3 3.1 2.9 2.8 2.7 2.6 2.6 2.4 2.3 2.3 2.1 1.9 1.9 1.8 1.7 1.6 1.5 33.7 Online 1.7 1.4 1.1 1.1 1.0 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 Eig. (N=1,047) 10.8 % 7.9 4.6 4.5 3.9 3.5 3.4 3.2 2.9 2.8 2.8 2.5 2.4 2.3 2.2 2.0 2.0 1.9 1.8 1.6 1.5 1.3 33.5 Paper 9.4 2.2 1.3 1.2 1.1 1.0 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.4 Eig. (N=1,379) % Grade 5 6.9 4.1 3.8 3.5 3.5 3.2 3.2 3.0 2.8 2.6 2.5 2.4 2.4 2.2 2.1 2.1 2.0 1.9 1.9 1.8 1.7 33.1 Online 9.3 1.9 1.1 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 Eig. (N=1,378) % 4.9 4.3 3.8 3.6 3.4 3.2 2.9 2.8 2.6 2.5 2.4 2.3 2.1 2.0 1.9 1.9 1.7 1.7 1.5 1.5 31.3 10.3 Paper 8.8 2.9 1.4 1.2 1.1 1.0 1.0 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.4 0.4 Eig. (N=1,444) % Grade 4 7.4 4.5 4.0 3.7 3.3 3.2 3.1 2.9 2.8 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2.0 1.9 1.9 1.7 1.5 32.5 Online 9.1 2.1 1.3 1.1 1.0 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 Eig. (N=1,435) % 9.8 4.3 4.3 3.5 3.2 3.0 2.8 2.7 2.6 2.4 2.4 2.1 2.1 2.0 1.9 1.7 1.7 1.6 1.5 1.4 1.3 36.3 Paper 2.7 1.2 1.2 1.0 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.4 Eig. (N=1,405) 10.2 % Grade 3 6.3 4.1 3.6 3.4 3.2 3.2 2.9 2.8 2.6 2.5 2.4 2.2 2.1 2.0 2.0 1.9 1.7 1.7 1.6 1.6 1.5 38.1 Online 1.8 1.1 1.0 1.0 0.9 0.9 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 Eig. (N=1,376) 10.7 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Factor 17.23 ACT Aspire Validity

Table 17.4. (continued) % 1.4 1.4 1.2 1.1 1.0 1.0 0.8 0.8 0.6 0.4 Paper 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.2 0.1 Eig. (N=1,023) % 1.5 1.4 1.3 1.2 1.2 1.1 1.0 0.9 0.7 0.2 Early High School Online (N=959) 0.5 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.2 0.1 Eig. % 1.3 1.2 1.0 0.9 0.9 0.8 0.7 0.7 0.6 0.4 Paper (N=934) 0.4 0.4 0.3 0.3 0.3 0.3 0.2 0.2 0.2 0.1 Eig. % Grade 8 1.4 1.3 1.2 1.1 1.0 0.9 0.8 0.7 0.4 0.3 Online (N=913) 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.1 0.1 Eig. % 1.4 1.3 1.2 1.1 1.0 1.0 0.9 0.7 0.5 0.2 Paper 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.2 0.2 0.1 Eig. (N=1,042) % Grade 7 1.3 1.3 1.2 1.1 1.1 0.9 0.8 0.8 0.5 -0.8 Online 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.2 Eig. -0.2 (N=1,028) % 1.3 1.3 1.2 1.0 1.0 0.9 0.8 0.8 0.5 0.4 Paper 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.2 0.2 0.1 Eig. (N=1,040) % Grade 6 1.3 1.3 1.2 1.1 1.0 1.0 0.9 0.8 0.8 0.7 Online 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.2 0.2 Eig. (N=1,047) % 1.3 1.1 1.1 0.9 0.7 0.6 Paper 0.4 0.3 0.3 0.3 0.2 0.2 Eig. (N=1,379) % Grade 5 1.6 1.5 1.3 1.1 1.0 0.9 Online 0.5 0.4 0.4 0.3 0.3 0.2 Eig. (N=1,378) % 1.2 1.2 1.1 1.0 0.7 0.2 Paper 0.3 0.3 0.3 0.3 0.2 0.0 Eig. (N=1,444) % Grade 4 1.4 1.4 1.3 1.2 0.8 0.6 Online 0.4 0.4 0.4 0.3 0.2 0.2 Eig. (N=1,435) % 1.3 1.1 1.0 0.8 0.7 0.5 Paper 0.4 0.3 0.3 0.2 0.2 0.1 Eig. (N=1,405) % Grade 3 1.4 1.2 1.1 1.0 0.9 0.6 Online 0.4 0.3 0.3 0.3 0.3 0.2 Eig. (N=1,376) Eig. denotes eigenvalue. 23 24 25 26 27 28 29 30 31 32 Factor

17.24 Note. ACT Aspire Validity

Table 17.5. English Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study * * 0 0 24 34 0.52 0.53 222.30 447.03 DIFF DIFF * * 0 0 2 51 526 2 2 0.02 0.03 0.05 0.05 0.98 0.98 0.97 0.96 434.92 861.31 * * * * Grade 7 Paper Grade 5 Online 0 0 275 560 0.03 0.04 0.06 0.07 0.96 0.96 0.93 0.92 1 1 657.22 1308.33 * * * 0 0 0 49 24 34 0.60 0.58 0.55 DIFF DIFF DIFF 393.27 317.85 1451.23 * * * 0 0 0 2 51 526 0.03 0.05 0.98 0.98 0.03 0.02 0.05 0.05 0.97 0.98 0.97 0.98 2 2 2 112 6 715.36 513.66 1946.53 * * * * * Grade 4 Paper Grade 7 Online Grade 9 Paper 0 0 0 275 560 0.04 0.06 0.95 0.95 0.04 0.03 0.07 0.06 0.94 0.96 0.93 0.95 1 1 1 117 5 906.92 3397.76 1033.20 * * * * 0 0 0 49 24 34 0.54 0.20 0.43 DIFF DIFF DIFF 735.98 235.28 1115.02 * * * 0 0 0 2 51 526 0.02 0.04 0.99 0.99 0.02 0.03 0.05 0.05 0.98 0.97 0.97 0.96 2 112 6 2 2 900.40 428.16 1685.60 * Grade 4 Online Grade 6 Paper Grade 9 Online * * * * * * 0 0 0 275 560 0.03 0.06 0.97 0.97 0.03 0.05 0.06 0.08 0.95 0.87 0.94 0.87 1 1 1 117 5 663.43 2421.58 2015.42 * * * 0 0 0 34 24 34 0.59 0.46 0.47 DIFF DIFF DIFF 248.40 996.26 449.69 * * * 0 0 0 526 2 51 526 2 0.03 0.06 0.97 2 2 0.98 0.03 0.02 0.05 0.05 0.97 0.98 0.97 0.98 917. 21 529.01 763.99 * * * * * * * * Grade 3 Paper Grade 6 Online Grade 8 Paper 0 0 0 560 275 560 0.04 0.07 0.96 0.97 0.06 0.03 0.09 0.06 0.88 0.94 0.87 0.94 1 1 1 1165.62 1525.28 1213.68 * * * 0 0 0 34 24 24 0.51 0.65 0.48 DIFF DIFF DIFF 295.27 213.70 490.28 * * * 0 0 0 526 2 51 2 51 0.02 0.06 0.98 0.98 0.03 0.02 0.05 0.04 0.98 0.99 0.97 0.99 2 2 2 721.89 483.92 365.61 * * * * * Grade 5 Paper Grade 3 Online Grade 8 Online 0 0 0 560 275 275 0.03 0.07 0.96 0.96 0.03 0.04 0.06 0.07 0.96 0.94 0.95 0.94 1 1 1 697.63 855.89 1017.16 DF is the degree of freedom; CFI is the comparative fit index; TLI is the Tucker-Lewis index; index; RMSEATucker-Lewis is the root mean square error of approximation; SRMRDF is the standardized is the degree of freedom; CFI is the TLI is the comparative fit index; 2 2 2 χ χ χ DF TLI DF DF TLI TLI CFI CFI CFI CORR CORR CORR SRMR SRMR SRMR p-value p-value p-value Fit Stats Fit Stats Fit Stats RMSEA RMSEA RMSEA Note. root mean square residual; and CORR 2 loadings. Flag criteria included CFI 1 and Factor < 0.95, RMSEA is the correlation between Factor < 0.95, TLI > 0.06, SRMR > 0.08, |DIFF CFI| > 0.1, |DIFF > 0.1, |DIFF TLI| RMSEA| > 0.05, |DIFF SRMR| > 0.05, and CORR < 0.3. |DIFF| for an index. represents the absolute difference

17.25 ACT Aspire Validity

Table 17.6. Mathematics Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study * * * * 0 0 24 33 0.26 0.59 DIFF DIFF 141.08 1256.08 * * 0 0 2 2 2 51 494 0.02 0.05 0.98 0.02 0.05 0.95 0.98 0.96 405.29 748.06 * * * * * * Grade 7 Paper Grade 5 Online 0 0 1 1 275 527 0.05 0.07 0.87 0.03 0.06 0.92 0.88 0.93 546.37 2004.14 * * * * * 0 0 0 37 24 33 0.49 0.53 0.24 DIFF DIFF DIFF 436.78 895.54 361.12 * * * * * 0 0 0 2 2 2 628 2 51 494 0.04 0.02 0.02 0.05 0.99 0.98 0.02 0.05 0.99 0.93 0.99 0.94 943.23 460.17 694.45 * * * * * Grade 4 Paper Grade 9 Paper Grade 7 Online 0 0 0 1 1 1 665 275 527 0.06 0.03 0.03 0.06 0.97 0.96 0.05 0.07 0.97 0.69 0.96 0.71 1380.01 1355.71 1055.57 * * * * * 0 0 0 37 24 33 0.56 0.14 0.39 DIFF DIFF DIFF 243.17 286.84 2154.74 * * * * * * 0 0 0 2 2 2 628 2 51 494 0.04 0.02 0.03 0.06 0.99 0.94 0.02 0.05 0.99 0.94 0.95 0.95 454.20 846.19 890.03 * * * * * * * * * Grade 6 Paper Grade 4 Online Grade 9 Online 0 0 0 1 1 1 665 275 527 0.05 0.02 0.06 0.10 0.99 0.66 0.03 0.06 0.99 0.87 0.68 0.88 741.03 1089.36 3044.77 * * * * * * 0 0 0 37 24 33 0.54 0.19 0.35 DIFF DIFF DIFF 422.06 738.93 953.44 * * * 0 0 0 2 2 2 628 2 51 494 0.05 0.02 0.02 0.05 0.98 0.98 0.03 0.05 0.99 0.95 0.98 0.96 974.08 643.30 647.68 * * * * * * * Grade 3 Paper Grade 6 Online Grade 8 Paper 0 0 0 1 1 1 665 275 527 0.06 0.03 0.04 0.07 0.97 0.84 0.05 0.07 0.97 0.88 0.85 0.89 1396.14 1382.23 1601.12 * * * * 0 0 0 37 24 24 0.73 0.29 0.63 DIFF DIFF DIFF 296.35 337.26 324.63 * * * * * 0 0 0 2 2 2 628 2 51 2 51 0.06 0.03 0.02 0.06 0.98 0.93 0.02 0.05 0.98 0.98 0.94 0.98 461.48 459.34 1037.57 * * * * * Grade 3 Online Grade 5 Paper Grade 8 Online 0 0 0 1 1 1 665 275 275 0.06 0.03 0.04 0.07 0.97 0.04 0.06 0.84 0.95 0.97 0.95 0.86 798.73 783.97 1333.91 DF is the degree of freedom; CFI is the comparative fit index; TLI is the Tucker-Lewis index; index; RMSEATucker-Lewis is the root mean square error of approximation; SRMRDF is the standardized root is the degree of freedom; CFI is the TLI is the comparative fit index; 2 2 2 χ χ χ TLI TLI TLI DF DF DF CFI CFI CFI CORR CORR CORR SRMR SRMR SRMR p-value p-value p-value Fit Stats Fit Stats Fit Stats RMSEA RMSEA RMSEA Note. mean square residual; and CORR 2 loadings. Flag criteria included CFI 1 and Factor < 0.95, RMSEA is the correlation between Factor < 0.95, TLI > 0.06, SRMR > 0.08, |DIFF CFI| > 0.1, |DIFF > 0.1, |DIFF TLI| RMSEA| > 0.05, |DIFF SRMR| > 0.05, and CORR < 0.3. |DIFF| for an index. represents the absolute difference 17.26 ACT Aspire Validity

Table 17.7. Reading Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study * * 0 0 23 23 0.76 0.70 DIFF DIFF 129.16 101.27 * * 0 0 2 2 229 229 0.02 0.04 0.03 0.04 0.99 0.99 0.98 0.99 351.46 387.33 * * Grade 7 Paper Grade 5 Online 0 0 1 1 252 252 0.03 0.04 0.03 0.05 0.98 0.99 0.98 0.98 480.62 488.61 * * * 0 0 0 23 23 23 0.67 0.80 0.67 0.02 0.01 DIFF DIFF DIFF -0.01 -0.01 77.32 244.45 285.46 * * 0 0 2 2 2 229 229 229 0.02 0.04 0.03 0.04 0.01 0.04 0.99 0.99 0.99 0.02 1.00 0.99 1.00 312.35 275.26 392.32 * * * Grade 4 Paper Grade 9 Paper Grade 7 Online 0 0 0 1 1 1 252 252 252 0.03 0.05 0.04 0.05 0.02 0.05 0.98 0.98 0.98 0.98 0.99 0.99 556.80 352.58 677.77 * * * * 0 0 0 0 23 23 22 0.73 0.68 0.18 0.01 0.01 DIFF DIFF DIFF -0.01 92.33 203.91 111.45 * * 0 0 2 2 2 229 229 208 0.01 0.03 0.03 0.04 0.01 0.99 0.99 1.00 0.03 0.04 1.00 0.98 0.98 395.99 380.64 280.55 * * * Grade 4 Online Grade 6 Paper Grade 9 Online 0 0 0 1 1 1 252 252 230 0.02 0.04 0.04 0.06 0.99 0.99 0.99 0.03 0.05 0.99 0.96 0.96 599.90 492.09 372.88 * * * * 0 0 0 23 23 22 0.60 0.76 0.57 0.01 0.02 DIFF DIFF DIFF -0.01 -0.01 72.25 515.23 222.79 * * * 0 0 0 2 2 2 229 229 208 0.02 0.04 0.02 0.04 0.99 0.99 0.99 0.03 0.05 0.99 0.99 1.00 417. 47 268.48 352.76 * * * Grade 3 Paper Grade 6 Online Grade 8 Paper 0 0 0 1 1 1 252 252 230 0.04 0.06 0.02 0.04 0.98 0.98 0.96 0.04 0.06 0.97 0.99 0.99 932.69 340.72 575.55 * * * 0 0 0 23 23 23 0.38 0.68 0.60 0.01 0.01 DIFF DIFF DIFF -0.01 -0.01 118.27 230.74 121.15 * * * 0 0 0 2 2 2 229 229 229 0.02 0.03 0.02 0.04 0.99 0.99 0.99 0.03 0.05 1.00 0.99 0.99 335.74 317.77 374.81 * * * Grade 3 Online Grade 5 Paper Grade 8 Online 0 0 0 1 1 1 252 252 252 0.02 0.04 0.03 0.05 0.98 0.98 0.99 0.04 0.07 0.99 0.98 0.98 454.00 548.51 495.96 2 2 2 DF is the degree of freedom; CFI is the comparative fit index; TLI is the Tucker-Lewis index; index; RMSEATucker-Lewis is the root mean square error of approximation; SRMR DF is the standardized is the degree of freedom; CFI is the TLI is the comparative fit index; χ χ χ DF DF DF TLI TLI TLI CFI CFI CFI CORR CORR SRMR SRMR SRMR CORR p-value p-value p-value Fit Stats Fit Stats Fit Stats RMSEA RMSEA RMSEA Note. root mean square residual; and CORR 2 loadings. Flag criteria included CFI 1 and Factor < 0.95, RMSEA is the correlation between Factor < 0.95, TLI > 0.06, SRMR > 0.08, |DIFF CFI| > 0.1, |DIFF > 0.1, |DIFF TLI| RMSEA| > 0.05, |DIFF SRMR| > 0.05, and CORR < 0.3. |DIFF| for an index. represents the absolute difference 17. 27 ACT Aspire Validity

Table 17.8. Science Test Fit Statistics for One- and Two-Factor Models Using Data from the 2013 ACT Aspire Mode Comparability Study * * 0 0 27 31 0.55 0.62 DIFF DIFF 444.16 901.68 * * 0 0 323 433 2 2 0.04 0.02 0.99 0.99 0.03 0.05 0.99 0.98 501.38 839.85 * * Grade 7 Paper Grade 5 Online 0 0 350 464 0.06 0.04 0.97 0.97 0.05 0.07 0.96 0.95 1 1 945.54 1741.53 * * * * 0 0 0 27 31 31 0.57 0.45 0.22 DIFF DIFF DIFF 240.17 476.94 2109.21 * * * 0 0 0 323 433 433 0.05 0.03 0.98 0.98 0.05 0.03 0.98 0.98 0.03 0.05 0.99 0.99 2 2 2 815.08 879.85 891.01 * * * * * * * Grade 4 Paper Grade 7 Online Grade 9 Paper 0 0 0 350 464 464 0.06 0.04 0.96 0.96 0.10 0.07 0.88 0.89 0.04 0.06 0.98 0.98 1 1 1 2924.30 1120.01 1367.95 * * * * 0 0 0 27 31 31 0.29 0.58 0.47 DIFF DIFF DIFF 843.09 250.56 1664.18 * * * 0 0 0 323 433 433 0.05 0.02 0.99 0.99 0.04 0.02 0.99 0.99 0.03 0.05 0.98 0.98 2 2 2 506.33 605.68 929.10 * * * * * * * Grade 4 Online Grade 6 Paper Grade 9 Online 0 0 0 350 464 464 0.06 0.03 0.98 0.98 0.07 0.05 0.07 0.08 0.93 0.92 0.95 0.95 1 1 1 856.24 1349.42 2593.28 * * * 0 0 0 27 31 31 0.50 0.47 0.67 DIFF DIFF DIFF 575.15 344.96 2055.92 * * * 0 0 0 323 433 433 0.05 0.03 0.98 0.99 0.05 0.04 0.03 0.05 0.99 0.99 0.98 0.98 2 2 2 801.27 779.30 1049.07 * * * * * * * Grade 3 Paper Grade 8 Paper Grade 6 Online 0 0 0 350 464 464 0.07 0.05 0.96 0.09 0.08 0.91 0.92 0.96 0.04 0.06 0.97 0.98 1 1 1 1376.42 3105.00 1124.26 * * * 0 0 0 27 27 31 0.59 0.67 0.46 DIFF DIFF DIFF 3 57.74 572.97 954.17 * * * 0 0 0 323 323 433 2 2 2 0.05 0.02 0.99 0.04 0.03 0.99 0.99 0.99 0.03 0.05 0.98 0.98 614.65 657.07 814.33 * * * * * Grade 3 Online Grade 5 Paper Grade 8 Online 0 0 0 350 350 464 0.06 0.04 0.98 0.06 0.04 0.97 0.98 0.98 0.05 0.08 0.94 0.94 1 1 1 972.39 1230.04 1768.50 2 2 2 DF is the degree of freedom; CFI is the comparative fit index; TLI is the Tucker-Lewis index; index; RMSEATucker-Lewis is the root mean square error of approximation; SRMR DF is the standardized is the degree of freedom; CFI is the TLI is the comparative fit index; χ χ χ TLI TLI DF TLI DF DF CFI CFI CFI CORR CORR CORR SRMR SRMR SRMR p-value p-value p-value Fit Stats Fit Stats Fit Stats RMSEA RMSEA RMSEA Note. root mean square residual; and CORR 2 loadings. Flag criteria included CFI 1 and Factor < 0.95, RMSEA is the correlation between Factor < 0.95, TLI > 0.06, SRMR > 0.08, |DIFF CFI| > 0.1, |DIFF > 0.1, |DIFF TLI| RMSEA| > 0.05, |DIFF SRMR| > 0.05, and CORR < 0.3. |DIFF| for an index. represents the absolute difference 17.28 ACT Aspire Validity

Table 17.9. Two-Factor Exploratory Factor Model Loadings for Four ACT Aspire Forms Identified as Showing Evidence of Two Factors Based on Analysis of Fit Statistics

Mathematics Mathematics Mathematics Mathematics Grade 4 Paper Grade 5 Paper Grade 6 Online Grade 6 Paper

Item Factor 1 Factor 2 Factor 1 Factor 2 Factor 1 Factor 2 Factor 1 Factor 2

1 0.32 0.02 0.21 -0.07 0.55 -0.09 0.48 0.09 2 0.33 -0.03 0.57 -0.04 0.43 -0.09 0.52 -0.10 3 0.46 0.00 0.44 -0.12 0.10 -0.03 0.27 -0.01 4 0.59 -0.03 0.19 -0.06 0.14 0.01 -0.18 0.07 5 0.35 -0.08 0.39 0.04 0.58 -0.06 0.60 0.00 6 0.61 0.00 0.14 -0.02 -0.08 0.11 -0.01 -0.02 7 0.46 -0.10 0.15 -0.04 0.55 -0.07 0.59 -0.04 8 0.22 0.02 0.04 -0.07 0.61 -0.05 0.60 0.00 9 0.15 -0.01 0.30 -0.03 0.37 0.00 0.38 -0.06 10 0.22 0.02 0.56 -0.20 0.45 -0.06 0.45 0.01 11 0.09 0.01 0.03 0.09 0.44 -0.05 0.54 -0.08 12 0.52 0.10 0.52 0.00 0.45 -0.08 0.42 0.07 13 0.05 0.15 0.24 0.04 -0.04 0.14 0.54 -0.02 14 0.17 0.14 0.03 0.11 0.41 0.07 0.44 -0.12 15 0.18 0.16 0.28 0.06 0.48 0.05 0.04 -0.03 16 -0.05 0.21 0.46 0.11 0.39 -0.02 0.27 0.04 17 0.32 0.47 0.38 0.27 0.13 0.10 0.44 0.11 18 0.13 0.42 0.26 0.25 0.32 0.12 0.45 0.07 19 0.04 0.44 0.00 0.20 0.52 0.35 0.58 0.24 20 -0.11 0.31 0.23 0.40 0.17 0.28 0.20 0.08 21 -0.01 0.36 0.18 0.40 0.14 0.41 0.22 0.17 22 0.54 0.02 0.39 0.30 0.01 0.65 0.17 0.22 23 0.39 0.20 0.44 0.17 -0.03 0.41 0.34 0.27 24 0.00 0.69 0.25 0.53 0.20 0.33 0.08 0.34 25 -0.18 0.67 -0.02 0.79 0.33 0.02 0.24 0.50 26 0.58 0.03 0.07 0.40 27 0.42 0.22 0.01 0.38 28 0.64 0.00 -0.04 0.73 29 0.47 0.17 -0.06 0.47 30 0.57 0.30 -0.01 0.43 31 0.29 0.26 0.29 0.38 32 0.45 0.19 0.27 0.37 33 0.31 0.42 0.09 0.74 34 -0.06 0.84 -0.21 0.76 17.29 ACT Aspire Validity

Table 17.10. Summary Statistics for ACT Aspire Scale Scores by Grade

Grade Statistic English Mathematics Reading Science Writing ELA STEM Composite Mean 416.9 412.37 412.17 414.27 422.76 417.27 413.57 414.05 3 SD 5.97 3.74 5.27 5.95 6.65 5 4.45 4.61 N 5,534 5,534 5,534 5,534 5,534 5,534 5,534 5,534 Mean 420 415.19 414.46 417.12 423.09 419.18 416.41 416.82 4 SD 6.07 3.92 5.6 6.49 7 5.15 4.6 4.69 N 5,354 5,354 5,354 5,354 5,354 5,354 5,354 5,354 Mean 422.33 417.24 416.65 419.27 423.02 420.66 418.5 419 5 SD 6.95 4.53 6.16 6.64 7.03 5.59 5.01 5.19 N 5,300 5,300 5,300 5,300 5,300 5,300 5,300 5,300 Mean 423.89 419.09 417.88 419.78 426.76 422.84 419.69 420.28 6 SD 7.86 5.64 6.75 7.26 7.81 6.28 5.82 5.98 N 5,087 5,087 5,087 5,087 5,087 5,087 5,087 5,087 Mean 425.37 420.24 419.26 420.8 425.43 423.36 420.78 421.55 7 SD 8.51 6.39 6.67 7.71 7.09 6.25 6.5 6.39 N 5,119 5,119 5,119 5,119 5,119 5,119 5,119 5,119 Mean 426.81 422.68 420.36 422.62 423.13 423.44 422.89 423.24 8 SD 8.86 7.36 7.17 7.76 6.59 6.5 7.11 6.94 N 3,995 3,995 3,995 3,995 3,995 3,995 3,995 3,995 Mean 427.86 424.53 421.1 423.67 423.67 424.21 424.35 424.41 9 SD 10.46 8.21 7.78 8.11 7.43 7.6 7.6 7.73 N 2,287 2,287 2,287 2,287 2,287 2,287 2,287 2,287 Mean 431.2 427.22 422.49 425.6 425.74 426.47 426.66 426.75 10 SD 11.26 9.1 8.37 8.96 8.07 8.21 8.52 8.51 N 1,778 1,778 1,778 1,778 1,778 1,778 1,778 1,778

Note. Sample includes students taking English, mathematics, reading, science, and writing tests from the spring 2013 special studies.

17. 3 0 ACT Aspire Validity

Table 17.11. Correlations among ACT Aspire Scale Scores by Grade

Grade English Mathematics Reading Science Writing ELA STEM English Mathematics 0.62 Reading 0.72 0.63 Science 0.72 0.67 0.76 3 Writing 0.47 0.46 0.49 0.45 ELA 0.86 0.67 0.85 0.75 0.80 STEM 0.74 0.86 0.77 0.95 0.49 0.79 Composite 0.89 0.80 0.89 0.91 0.53 0.90 0.94 English Mathematics 0.47 Reading 0.69 0.49 Science 0.67 0.53 0.75 4 Writing 0.46 0.34 0.45 0.42 ELA 0.85 0.52 0.84 0.72 0.80 STEM 0.67 0.80 0.73 0.93 0.44 0.73 Composite 0.86 0.69 0.88 0.90 0.50 0.88 0.93 English Mathematics 0.49 Reading 0.70 0.53 Science 0.70 0.58 0.75 5 Writing 0.45 0.36 0.47 0.46 ELA 0.86 0.55 0.85 0.76 0.78 STEM 0.69 0.84 0.74 0.93 0.47 0.75 Composite 0.87 0.73 0.88 0.90 0.52 0.90 0.93 English Mathematics 0.60 Reading 0.70 0.57 Science 0.72 0.62 0.78 6 Writing 0.50 0.40 0.48 0.48 ELA 0.87 0.62 0.85 0.77 0.80 STEM 0.74 0.87 0.76 0.92 0.49 0.78 Composite 0.88 0.78 0.88 0.90 0.54 0.91 0.94 Note. Sample included students taking English, mathematics, reading, science, and writing tests from the spring 2013 special studies.

17.31 ACT Aspire Validity

Table 17.11. (continued) Grade English Mathematics Reading Science Writing ELA STEM English Mathematics 0.62 Reading 0.69 0.64 Science 0.68 0.69 0.74 7 Writing 0.49 0.45 0.47 0.47 ELA 0.89 0.68 0.85 0.75 0.77 STEM 0.71 0.90 0.75 0.93 0.50 0.78 Composite 0.88 0.83 0.87 0.90 0.54 0.91 0.94 English Mathematics 0.69 Reading 0.72 0.68 Science 0.72 0.76 0.76 8 Writing 0.54 0.52 0.54 0.53 ELA 0.90 0.74 0.88 0.79 0.78 STEM 0.75 0.94 0.77 0.94 0.56 0.81 Composite 0.89 0.87 0.88 0.91 0.60 0.93 0.95 English Mathematics 0.73 Reading 0.77 0.69 Science 0.73 0.73 0.73 9 Writing 0.64 0.57 0.62 0.57 ELA 0.93 0.75 0.89 0.77 0.83 STEM 0.79 0.93 0.76 0.93 0.61 0.82 Composite 0.92 0.88 0.88 0.89 0.67 0.94 0.95 English Mathematics 0.74 Reading 0.77 0.72 Science 0.74 0.78 0.75 10 Writing 0.65 0.60 0.60 0.59 ELA 0.93 0.78 0.89 0.79 0.83 STEM 0.79 0.94 0.78 0.94 0.63 0.83 Composite 0.91 0.90 0.89 0.90 0.68 0.94 0.95 Note. Sample included students taking English, mathematics, reading, science, and writing tests from the spring 2013 special studies.

17.32 ACT Aspire Validity

17.2.4 Evidence Regarding Relationships with Conceptually Related Constructs Evidence regarding relationships with conceptually related constructs refers to evaluating the relationships between scores on test items compared to other variables. Two studies summarized below provide evidence regarding the relationships of ACT Aspire scale scores with scale scores on other assessments of similar constructs. One study compares grade 8 ACT Aspire scores to ACT Explore scores and early high school ACT Aspire scores to ACT Plan scores. A second study compares grades 3 to 8 ACT Aspire scores to scores on a state assessment.

17.2.4.1 Study 1: Comparison of ACT Explore and ACT Plan Scores to ACT Aspire Scores In this study the relationships between ACT Explore, ACT Plan, and ACT Aspire English, mathematics, reading, and science scale scores and Composite scores were compared using samples of students taking ACT Explore and ACT Aspire (grades 8 and 9) or ACT Plan and ACT Aspire assessments (grade 10). Samples included students participating in the special 2013 ACT Aspire studies described in Chapter 12 and who had also taken ACT Plan or ACT Explore separately. The sample sizes and demographics by grade and subject in each sample are listed in Table 17.12. The samples included students from a total of 122 districts and 263 schools.

ACT Explore and ACT Plan are both intended to measure academic achievement, similar to ACT Aspire. In fact, ACT Aspire is intended to serve similar purposes and share similar interpretations as ACT Explore and ACT Plan: ACT Aspire is the successor to those assessments. ACT Explore and ACT Plan scale scores are intended to be on the same scale but have different scale ranges. ACT Explore scale scores range from 1 to 25 and ACT Plan scale scores range from 1 to 32. ACT Aspire scores are also intended to be on the same scale and have different scale score ranges across grades (see Table 17.13).

17.33 ACT Aspire Validity

Table 17.12. Percentages of Students in the ACT Explore, ACT Plan, and ACT Aspire Matched Sample by Subject, Grade and Demographics 1 1 3 2 0 2 2 3 10 50 50 30 21 42 56 39 (3,154) 9 3 0 5 3 0 1 0 2 48 49 13 24 54 18 80 (1,181) Composite 1 4 1 1 0 5 2 8 11 45 45 38 20 35 78 15 (6,419) 2 1 3 2 0 1 2 3 10 48 50 31 20 41 51 44 (4,236) 9 0 4 2 0 1 0 3 10 44 46 19 26 48 19 78 (1,575) Science 9 1 5 1 1 0 4 2 8 45 45 35 20 37 76 17 (7,779) 1 1 3 2 0 1 1 3 10 49 50 29 22 41 53 43 (4,249) 9 3 0 4 2 0 1 0 3 48 49 13 33 47 17 79 (1,484) Reading 1 4 1 1 0 4 2 8 10 45 45 36 21 36 76 18 (7,594) 1 0 4 2 0 1 2 3 10 49 50 29 21 42 52 43 (4,545) 9 3 0 6 2 0 0 0 5 49 48 12 33 46 17 78 (1,641) Mathematics 1 5 1 1 0 4 2 8 10 45 45 36 21 36 74 20 (7,803) 2 0 3 2 0 1 2 2 10 49 49 26 21 45 57 39 (4,643) 9 4 0 5 2 0 1 0 4 48 48 13 34 45 18 78 (1,563) English 8 1 4 1 1 0 5 2 45 45 38 19 35 75 18 10 (7,574) (N) Grade

Zero indicates a percentage fewer than 1%. Missing values indicate no students. Percentages within a demographic grouping may not sum up to 100% due to within a demographic grouping may not sum up to 100% than 1%. Missing values indicate no students. Percentages Zero indicates a percentage fewer Female Male Not Specified Black/African American American Indian/Alaskan Native White Hispanic/Latino Asian Native Hawaiian/Other Pacific Islander Two or More Races East Midwest Southwest West Not Specified Race/Ethnicity Region Gender 17. 3 4 Demographic Grouping % Note. rounding. ACT Aspire Validity

Table 17.13 lists the mean, standard deviation, minimum, and maximum of scale scores for ACT Explore/ACT Plan and ACT Aspire. The mean scale scores increase for ACT Explore/Plan and ACT Aspire across grades 8 to 10. This pattern is consistent with the argument that students at higher grades have higher achievement and therefore should have higher scores.

Table 17.13. Descriptive Statistics for ACT Explore, ACT Plan and ACT Aspire Scale Scores

Subject Grade N Assessment Mean SD Minimum Maximum

English ACT Explore 14.62 4.02 1 25 8 7,574 ACT Aspire 426.17 8.72 400 452

ACT Explore 15.73 4.38 2 25 9 1,563 ACT Aspire 426.87 10.75 400 454

ACT Plan 17.37 4.65 1 32 10 4,643 ACT Aspire 429.58 11.19 400 456

Mathematics ACT Explore 14.88 3.29 1 25 8 7,803 ACT Aspire 421.22 7.54 401 449

ACT Explore 16.56 3.76 4 25 9 1,641 ACT Aspire 423.85 8.47 403 451

ACT Plan 17.82 4.70 1 32 10 4,545 ACT Aspire 425.10 9.14 403 455

Reading ACT Explore 14.60 3.82 1 25 8 7,594 ACT Aspire 420.15 7.29 401 440

ACT Explore 15.30 4.20 1 25 9 1,484 ACT Aspire 420.18 7.74 403 442

ACT Plan 17.10 4.54 1 30 10 4,249 ACT Aspire 420.95 8.03 403 442

Science ACT Explore 16.63 3.21 3 25 8 7,779 ACT Aspire 421.82 7.75 402 446

ACT Explore 17.69 3.54 5 25 9 1,575 ACT Aspire 423.90 8.17 402 446

ACT Plan 18.59 3.89 1 32 10 4,236 ACT Aspire 423.97 8.80 401 449

Composite ACT Explore 15.35 3.16 3 25 8 6,419 ACT Aspire 422.62 6.95 406 443

ACT Explore 16.88 3.44 8 25 9 1,181 ACT Aspire 424.75 7.80 407 445

ACT Plan 17.89 3.97 6 31 10 3,154 ACT Aspire 425.22 8.38 408 447

17.35 ACT Aspire Validity

Table 17.14 lists the correlations between same-subject ACT Explore or ACT Plan scale scores and ACT Aspire scale scores for grades 8 to 10 (sample sizes are listed in Table 17.13). Correlations ranged from .65 (grade 10 reading) to .85 (grade 8 Composite), which are indicative of moderate to strong linear relationships between ACT Explore/ACT Plan and ACT Aspire. From a linear regression perspective, we can say that 42% to 72% of the variance in scale scores is shared between ACT Explore/ ACT Plan and ACT Aspire.

Table 17.14. Correlations between ACT Explore/ACT Plan Scale Scores and ACT Aspire Scale Scores

Sample/Grade English Mathematics Reading Science Composite

ACT Explore-ACT Aspire/8 .75 .72 .70 .69 .85 ACT Explore-ACT Aspire/9 .76 .75 .66 .70 .82 ACT Plan-ACT Aspire/10 .78 .77 .65 .69 .84

Table 17.15 lists the disattenuated correlations between ACT Explore or ACT Plan scale scores and ACT Aspire scale scores (sample sizes are listed in Table 17.13). Disattenuated correlations are estimates of the linear relationships between scores after the reliability of each test was taken into account.6 In classical test theory, disattenuated correlations are referred to as estimates of the relationship between true scores; they provide an estimate of the relationship between ACT Explore/ACT Plan and ACT Aspire as if each contributing score were perfectly reliable. Published or available reliability coefficients for ACT Explore (ACT, 2013a), ACT Plan (ACT, 2013b), and ACT Aspire (see Table 17.16) were used to calculate disattenuated correlations, which ranged from .75 to .91 across subjects, indicative of moderate to strong correlations. From a linear regression perspective, we can say that 56% to 82% of the variance in true scores is shared between ACT Explore/ACT Plan and ACT Aspire.

Table 17.15. Disattenuated Correlations between ACT Explore/ACT Plan Scale Scores and ACT Aspire Scale Scores

Sample/Grade English Mathematics Reading Science Composite

ACT Explore-ACT Aspire/8 .89 .88 .82 .83 .89 ACT Explore-ACT Aspire/9 .86 .89 .76 .82 .85 ACT Plan-ACT Aspire/10 .88 .91 .75 .81 .88

6 Disattenuated correlations are calculated as the correlation divided by the square root of the product of the reliabilities or 17. 3 6 ACT Aspire Validity

Table 17.16. Scale Score Reliabilities

Test and Grade Level English Mathematics Reading Science Composite

ACT Aspire 8* .85 .88 .85 .87 .96 ACT Aspire Early High School* .91 .89 .88 .88 .97 ACT Explore 8† .84 .76 .86 .79 .94 ACT Explore 9† .86 .80 .86 .82 .95 ACT Plan 10‡ .87 .80 .85 .82 .95

* Obtained from spring 2013 reliabilities reported above in the Reliability section † Obtained from the ACT Explore Technical Manual (ACT, 2013a) ‡ Obtained from the ACT Plan Technical Manual (ACT, 2013b)

Disattenuated correlations also provide us with an estimated upper limit of the observed correlations in Table 17.14. With this in mind, many of the moderate observed correlations in Table 17.14 are not that far from their disattenuated values, suggesting that they couldn’t be much higher than those observed due to the reliability of the contributing tests. For example, the smallest observed correlations in Table 17.14 were between reading tests (.70, .66, and .65), but the disattenuated correlations were roughly .10 higher (.82, .76, and .75).

Figures 17.2–17.6 display the relationships between ACT Explore or ACT Plan scale scores and ACT Aspire scale scores using box plots. Each box represents the distribution of ACT Explore or ACT Plan scale scores for a particular ACT Aspire scale score, with the box itself representing the the 25th percentile to the 75th percentiles of ACT Explore or ACT Plan scores, the line in the middle of the box representing the 50th percentile (or median), and the diamond representing the mean. The upper and lower whiskers of the box plots represent plus or minus 1.5 times the interquartile range (i.e., the range represented by the difference between 25th and 75th percentiles), and the circles represent individual scores outside of the range represented by the whiskers. As ACT Aspire scale scores increase (horizontal axis), the boxes representing the ACT Explore or ACT Plan scales score distribution also generally enlarge (vertical axis), illustrating the positive relationship between scale scores.

17.37 ACT Aspire Validity

Figure 17.2. Box plots of ACT Explore or ACT Plan English scale scores conditional on ACT Aspire English scale scores

Grade 8

Grade 9

17. 3 8 ACT Aspire Validity

Figure 17.2. (continued)

Grade 10

Figure 17.3. Box plots of ACT Explore or ACT Plan mathematics scale scores conditional on ACT Aspire mathematics scale scores

Grade 8

17. 3 9 ACT Aspire Validity

Figure 17.3. (continued)

Grade 9

Grade 10

17. 4 0 ACT Aspire Validity

Figure 17.4. Box plots of ACT Explore or ACT Plan reading scale scores conditional on ACT Aspire reading scale scores

Grade 8

Grade 9

17.41 ACT Aspire Validity

Figure 17.4. (continued)

Grade 10

Figure 17.5. Box plots of ACT Explore or ACT Plan science scale scores conditional on ACT Aspire science scale scores

Grade 8

17.42 ACT Aspire Validity

Figure 17.5. (continued)

Grade 9

Grade 10

17.43 ACT Aspire Validity

Figure 17.6. Box plots of ACT Explore or ACT Plan Composite scale scores conditional on ACT Aspire Composite scale scores

Grade 8

Grade 9

17. 4 4 ACT Aspire Validity

Figure 17.6. (continued)

Grade 10

Together, the correlations and box plots show that ACT Explore/ACT Plan and ACT Aspire scores are moderately to strongly related. Because ACT Explore/ACT Plan and ACT Aspire are designed to measure student achievement and have similar purposes, we would expect to observe relatively strong, but not perfect, positive relationships between scale scores across assessments, similar to those observed in Tables 17.14 and 17.15. The observed correlations were lower than anticipated, and this appeared to be explained by the reliabilities of the tests tempering the relationships between scores.

In addition to considering the relationships between scale scores across ACT Explore or ACT Plan and ACT Aspire for the same subject, we can consider the pattern of relationships across subjects (in this case, English, mathematics, reading, and science) and assessments (ACT Explore/ACT Plan and ACT Aspire). This analysis is commonly referred to as investigating a multitrait-multimethod matrix (Campbell & Fiske, 1959). This matrix is intended to provide convergent and divergent evidence regarding the subjects (traits) being measured by different assessments (methods).

Table 17.17 contains a multitrait-multimethod matrix that includes ACT Explore/ACT Plan and ACT Aspire English, mathematics, reading, and science scores for grades 8 to 10. Test reliabilities are reported in parentheses when a row and column contains the same subject and same test (for example, ACT Explore reading with ACT Explore

17.45 ACT Aspire Validity

reading)7 and the remaining cells contain correlations among scores. Ideally, we would include a variety of assessments measuring performance in different subjects to study effects due to methods and those due to traits. However, availability of traits and methods was limited to ACT Explore/ACT Plan scores and ACT Aspire English, reading, mathematics, and science scores for this analysis. Table 17.17. Multitrait-Multimethod Matrices for ACT Explore/Plan and ACT Aspire Scale Scores by Grade Level

ACT Explore/ACT Plan ACT Aspire

Grade (valid N) Scale Score English Mathematics Reading Science English Mathematics Reading Science

Grade 8 ACT Explore English (.84) (6,419) ACT Explore Mathematics .68 (.76)

ACT Explore Reading .76 .63 (.86)

ACT Explore Science .70 .67 .71 (.79)

ACT Aspire English .76 .62 .69 .63 (.85)

ACT Aspire Mathematics .69 .72 .63 .66 .71 (.88)

ACT Aspire Reading .68 .57 .70 .63 .73 .67 (.85)

ACT Aspire Science .69 .64 .68 .69 .72 .77 .76 (.87)

Grade 9 ACT Explore English (.86) (1,181) ACT Explore Mathematics .70 (.80)

ACT Explore Reading .75 .66 (.86)

ACT Explore Science .69 .70 .73 (.82)

ACT Aspire English .75 .62 .70 .64 (.91)

ACT Aspire Mathematics .64 .73 .60 .67 .70 (.89)

ACT Aspire Reading .62 .55 .65 .60 .75 .67 (.88)

ACT Aspire Science .64 .64 .63 .68 .73 .78 .75 (.88)

Grade ACT Plan English (.87) 10 (3,154) ACT Plan Mathematics .73 (.80) ACT Plan Reading .78 .66 (.85)

ACT Plan Science .74 .76 .71 (.82)

ACT Aspire English .77 .63 .68 .68 (.91)

ACT Aspire Mathematics .69 .77 .62 .73 .74 (.89)

ACT Aspire Reading .67 .57 .65 .64 .76 .69 (.88)

ACT Aspire Science .66 .67 .63 .71 .73 .78 .75 (.88)

Note. Cells in parentheses are coefficient alpha reliabilities. Others are correlations.

17. 4 6 7 These reliabilities are coefficient alpha. ACT Aspire Validity

For convergent evidence that ACT Aspire and ACT Explore/ACT Plan are measuring English, mathematics, reading, and science achievement, we want to see large positive monotrait-heteromethod correlations, which are represented by the correlation between ACT Explore/ACT Plan and ACT Aspire in Table 17.17 (see bold text). For discriminant evidence, we want to see smaller positive heterotrait-monomethod correlations, which are represented by all of the correlations between subjects within a given test (ACT Explore, ACT Plan, or ACT Aspire) in Table 17.17. Finally, we want to see the smallest correlations for heterotrait-heteromethod correlations, which are represented by the correlations between different subjects across different tests. To support interpreting ACT Explore/ACT Plan and ACT Aspire test scores as measuring similar academic achievement in a subject and distinct academic achievements in different subjects, we want to see stronger convergent evidence (monotrait-heteromethod correlations) and weaker discriminant evidence (heterotrait- heteromethod and heterotrait-monomethod correlations).

For English, the monotrait-heteromethod correlations (ACT Explore/ACT Plan English and ACT Aspire English) were all relatively large (above .75) and were among the largest in Table 17.17. Within ACT Aspire, the heterotrait-monomethod correlations for English were also relatively large although none were larger than the monotrait- heteromethod correlations. Within ACT Explore and ACT Plan, the heterotrait- monomethod correlations for English were relatively large, and for grades 8 to 10 reading the correlations were the same as or larger than the monotrait-heteromethod correlations. Most of the heterotrait-heteromethod correlations were the lowest but were not that much lower than the heterotrait-monomethod correlations. Higher alphas for ACT Aspire than for ACT Explore/ACT Plan indicated slightly stronger convergent evidence for ACT Aspire than for ACT Explore/ACT Plan. Finally, the large heterotrait-monomethod correlations indicate that the English test was not strongly differentiated from the other subjects.

For mathematics, the monotrait-heteromethod correlations ranged from .72 to .77. Within ACT Aspire, the heterotrait-monomethod correlations for mathematics were moderate to large, ranging from .67 to .78. In particular, the correlations between ACT Aspire science and ACT Aspire mathematics were slightly larger (.77 to .78) than the correlations between ACT Aspire mathematics and reading or English. Within ACT Explore/ACT Plan the heterotrait-monomethod correlations for mathematics were moderate to large but all were smaller than the monotrait-heteromethod correlations. The heterotrait-heteromethod correlations were smaller than the other correlations. Higher alphas for the ACT Aspire suggested that ACT Aspire showed slightly stronger convergent evidence than ACT Explore/ACT Plan. Notably, ACT Aspire mathematics and science were most closely related.

17.47 ACT Aspire Validity

For reading, the monotrait-heteromethod correlations ranged from .65 to .70. For ACT Explore/ACT Plan and ACT Aspire, the heterotrait-monomethod correlations between reading and English and between reading and science were larger than the monotrait- heteromethod correlations. In addition, for ACT Aspire, the correlation between reading and mathematics in grade 10 was larger than the correlation between reading tests for ACT Plan and ACT Aspire. The heterotrait-heteromethod correlations for grade 10 English and reading were larger than the monotrait-heteromethod correlation for grade 10 reading. The ACT Aspire and ACT Explore/ACT Plan reading tests showed similar patterns of relatively weak convergent evidence compared to discriminant evidence; the reading tests showed particularly strong method effects.

For science, the monotrait-heteromethod correlations ranged from .68 to .71. Nearly all of the heterotrait-monomethod correlations were larger than the monotrait- heteromethod correlations. The one exception was the grade 8 ACT Explore mathematics and science correlation, which was .67. The heterotrait-heteromethod correlations were smaller than the heterotrait-monomethod correlations although for grade 8 the ACT Aspire science and ACT Explore English correlation was the same as the monotrait-heteromethod correlation for science (.69), and for grade 10 the ACT Plan science and ACT Aspire mathematics correlation (.73) was larger than the monotrait-heteromethod correlation for science (.71).

The multitrait-multimethod matrix for ACT Explore/ACT Plan and ACT Aspire provided evidence regarding how well a subject (trait) was measured by different assessments (methods) and how well the subjects were differentiated from one another. This evidence was mixed. English showed the strongest convergent evidence, particularly for ACT Aspire, followed by mathematics, reading, and science. ACT Aspire mathematics and science showed particularly strong relationships. While ACT Explore/ACT Plan and ACT Aspire are intended to measure the same traits, the multitrait-multimethod results are consistent with the argument that the assessments may be systematically different in ways that lead to stronger relationships among scores within each assessment compared to between the assessments, particularly for reading and science. One explanation is that ACT Aspire forms contain a variety of item types, including technology-enhanced and fill-in-the-blank items (online) and constructed-response items. The fact that both ACT Explore/ACT Plan and ACT Aspire English tests do not have constructed-response items explains its strongest convergent evidence among all the subjects.

While there may be some degree of method effects, the results of this study support the argument that ACT Aspire is measuring achievement in English, mathematics, reading, and science in grades 8, 9, and 10. The correlations of ACT Aspire scores with ACT Explore/ACT Plan tests, which are intended to measure achievement and have been separately validated, are moderate to large. The magnitudes of these correlations are similar to those observed between ACT Explore and ACT Plan and between ACT Explore and the ACT in other technical documentation (e.g., ACT 2014b, p. 87). 17. 4 8 ACT Aspire Validity

17.2.4.2 Study 2: Comparison of State Assessment Scores to ACT Aspire Scores In this study, the relationships between Alabama reading and mathematics test scores (plus science; ARMT+) and ACT Aspire scale scores were compared using a sample of Alabama students taking both assessments in spring of 2013 in grades 3–8.8 This sample included students participating in 2013 ACT Aspire special studies described in other chapters. The sample sizes by grade and subject are listed in Table 17.18 ; this sample included roughly 5% to 15% of the tested population in Alabama.

ARMT+ and ACT Aspire are intended to assess student achievement. ARMT+ was designed to assess students’ mastery of state content standards in mathematics, reading, and science. ACT Aspire is designed to measure students’ academic achievement in English, mathematics, reading, science, and writing. While each test is built using different blueprints and different specifications, the traits intended to be measured by both include mathematics, reading, and science.

Table 17.19 lists the means, standard deviations, minimums, and maximums for ARMT+ and ACT Aspire scale scores by grade and subject. In all but three cases, mean scale scores increase across grades. For grades 6 and 7, the ACT Aspire reading mean scale score decreases slightly from 418.46 to 418.08.9 For grades 5 and 6 mathematics, the ARMT+ mean scale score decreases from 676.56 to 667.36 and for grades 5 and 7 science, ARMT+ mean scale scores decrease from 596.87 to 563.94. However, scores on ARMT+ are not strictly interpreted assuming vertical scale properties. For this reason, grade-to-grade comparisons of ARMT+ and ACT Aspire scores with data from 2013 administrations were not entirely appropriate. In this study, we compared ARMT+ scores and ACT Aspire scores within grades although many tables and figures include multiple grade levels to save space.

8 Alabama science scores were only available for grades 5 and 7. 9 This decrease in mean scale scores is consistent with results from other samples of students, including those included in the scaling study for ACT Aspire. 17. 4 9 ACT Aspire Validity

Table 17.18. Sample Sizes by Grade and Subject for the Sample of Students with Scores on ARMT+ and ACT Aspire in Spring 2013

Grade Mathematics Reading Science Total

3 9,020 9,198 — 9,688 4 9,185 9,249 — 9,849 5 8,188 8,231 7,209 9,011 6 6,257 6,242 — 6,695 7 5,060 5,058 4,702 5,423 8 4,045 4,306 — 4,475 Total 41,755 42,284 11,911 45,141

Table 17.19. Descriptive Statistics for ARMT+ and ACT Aspire Scale Scores

Subject Grade N Assessment Mean SD Minimum Maximum

Mathematics Alabama 635.27 39.77 503 781 3 9,020 ACT Aspire 412.18 3.91 400 429 Alabama 649.85 44.12 471 865 4 9,185 ACT Aspire 414.91 3.87 402 434 Alabama 676.56 38.18 560 846 5 8,188 ACT Aspire 416.85 4.78 402 437 Alabama 667.36 34.28 563 823 6 6,257 ACT Aspire 418.12 5.53 402 445 Alabama 678.25 42.23 573 873 7 5,060 ACT Aspire 419.10 6.59 401 445 Alabama 696.32 33.05 606 835 8 4,045 ACT Aspire 420.21 7.58 401 449 Reading Alabama 634.60 37.59 471 785 3 9,198 ACT Aspire 412.28 5.24 401 429 Alabama 651.80 37.70 513 789 4 9,249 ACT Aspire 414.60 5.64 401 431 Alabama 663.56 35.77 549 849 5 8,231 ACT Aspire 416.45 6.19 401 434 Alabama 671.91 35.70 538 802 6 6,242 ACT Aspire 418.46 6.66 401 436 Alabama 681.49 33.33 554 787 7 5,058 ACT Aspire 418.08 6.69 402 438 Alabama 681.71 31.50 577 818 8 4,306 ACT Aspire 420.32 7.19 401 440 Science Alabama 596.87 103.50 136 999 5 7,209 ACT Aspire 418.31 6.43 401 438 Alabama 563.94 106.92 42 999 7 4,702 ACT Aspire 418.58 7.42 401 440

17. 5 0 ACT Aspire Validity

Table 17.20 lists the correlations between same-subject ARMT+ scale scores and ACT Aspire scale scores across grades. Correlations ranged from .60 to .81, with higher correlations observed as grade increased for mathematics and science. These correlations are indicative of moderate to strong linear relationships between ARMT+ and ACT Aspire scores across subjects. From a linear regression perspective, we can say that 36% to 66% of the variance in scale scores is shared between ARMT+ and ACT Aspire.10

Table 17.21 lists the disattenuated correlations between ARMT+ and ACT Aspire. Published or available reliability coefficients for ARMT+ and ACT Aspire were used to calculate disattenuated correlations, which ranged from .71 to .90 and are indicative of moderate to strong correlations. From a linear regression perspective, we can say that 50% to 81% of the variance in true scores is shared between ARMT+ and ACT Aspire.

Table 17.20. Correlations between ARMT+ Scale Scores and ACT Aspire Scale Scores

Grade Mathematics Reading Science

3 .60 .73 — 4 .62 .77 — 5 .64 .76 .65 6 .64 .74 — 7 .72 .72 .69 8 .81 .75 —

Table 17.21. Disattenuated Correlations between ARMT+ Scale Scores and ACT Aspire Scale Scores

Grade Mathematics Reading Science

3 .71 .84 — 4 .77 .88 — 5 .79 .87 .73 6 .77 .85 — 7 .84 .84 .76 8 .90 .85 —

10 Percentage of variance shared is calculated by squaring the correlation between scores. 17.51 ACT Aspire Validity

Figures 17.7–17.9 display the relationships between ARMT+ scale scores and ACT Aspire scale scores using box plots. Each box represents the distribution of ARMT+ scale scores for a particular ACT Aspire scale score. The important pattern to observe in Figures 17.7 to 17.9 is that as ACT Aspire scale scores increase (horizontal axis), the boxes representing the ARMT+ scales score distribution also generally mount (vertical axis).

Together, the correlations and box plots show that ARMT+ and ACT Aspire scores are moderately to strongly related. Because ARMT+ and ACT Aspire are both designed to measure student achievement, but each are distinct assessments built to different test specifications with their own unique scale, we would expect to observe moderately to strongly, but not perfectly, positive relationships between ARMT+ scale scores and ACT Aspire scale scores.

Table 17.22 contains a multitrait-multimethod matrix which includes scores in mathematics, reading, and science for ARMT+ and ACT Aspire. Test reliabilities are reported in parentheses when a row and a column contain the same subject and same test (e.g., ARMT+ reading with ARMT+ reading) and the remaining cells contain correlations between scores. Ideally, we would include a variety of assessments measuring performance in different subjects to study effects due to methods and due to traits. However, availability of methods and traits was limited to scores in mathematics, reading, and science for ARMT+ and ACT Aspire in this analysis.

Figure 17.7. Box plots of ARMT+ mathematics scale scores conditional on ACT Aspire mathematics scale scores

Grade 3

17.52 ACT Aspire Validity

Figure 17.7. (continued)

Grade 4

Grade 5

17.53 ACT Aspire Validity

Figure 17.7. (continued)

Grade 6

Grade 7

17. 5 4 ACT Aspire Validity

Figure 17.7. (continued)

Grade 8

Figure 17.8. Box plots of ARMT+ reading scale scores conditional on ACT Aspire reading scale scores

Grade 3

17.55 ACT Aspire Validity

Figure 17.8. (continued)

Grade 4

Grade 5

17. 5 6 ACT Aspire Validity

Figure 17.8. (continued)

Grade 6

Grade 7

17.57 ACT Aspire Validity

Figure 17.8. (continued)

Grade 8

Figure 17.9. Box plots of ARMT+ science scale scores conditional on ACT Aspire science scale scores

Grade 5

17. 5 8 ACT Aspire Validity

Figure 17.9. (continued)

Grade 7

17. 5 9 ACT Aspire Validity

Table 17.22. Multitrait-Multimethod Matrices for ARMT+ and ACT Aspire Scale Scores by Grade Level

Alabama ACT Aspire

Grade (valid N) Scale Score Mathematics Reading Science Mathematics Reading Science

Grade 3 Alabama Mathematics (.92) (8,530) Alabama Reading .71 (.90)

ACT Aspire Mathematics .60 .63 (.77)

ACT Aspire Reading .56 .73 .65 (.84)

Grade 4 Alabama Mathematics (.93) (8,585) Alabama Reading .71 (.92)

ACT Aspire Mathematics .62 .57 (.69)

ACT Aspire Reading .62 .77 .57 (.84)

Grade 5 Alabama Mathematics (.92) (6,543) Alabama Reading .73 (.91)

Alabama Science .70 .73 (.92)

ACT Aspire Mathematics .63 .57 .51 (.71)

ACT Aspire Reading .65 .76 .63 .58 (.83)

ACT Aspire Science .70 .73 .65 .64 .75 (.86)

Grade 6 Alabama Mathematics (.92) (5,804) Alabama Reading .73 (.91)

ACT Aspire Mathematics .63 .53 (.76)

ACT Aspire Reading .64 .74 .54 (.84)

Grade 7 Alabama Mathematics (.94) (4,345) Alabama Reading .73 (.90)

Alabama Science .75 .74 (.94)

ACT Aspire Mathematics .71 .59 .60 (.78)

ACT Aspire Reading .68 .72 .69 .63 (.82)

ACT Aspire Science .73 .70 .69 .69 .76 (.88)

Grade 8 Alabama Mathematics (.93) (3,876) Alabama Reading .75 (.92)

ACT Aspire Mathematics .82 .70 (.88)

ACT Aspire Reading .64 .75 .67 (.85)

Note. Cells in parentheses are coefficient alpha reliabilities. Others are correlations.

17. 6 0 ACT Aspire Validity

For convergent evidence that ACT Aspire and ARMT+ are measuring mathematics, reading, and science achievement, we want to see large positive monotrait- heteromethod correlations, which are represented by the correlation between ARMT+ and ACT Aspire scores in the same subject (e.g., ARMT+ reading and ACT Aspire reading; see bold text in Table 17.22). For discriminant evidence, we want to see smaller positive heterotrait-monomethod correlations, which are represented by all of the correlations among subjects within a given test (ARMT+ or ACT Aspire) in Table 17.22. Finally, we want to see the smallest correlations for heterotrait-heteromethod correlations, which are represented by the correlations between different subjects across different tests. To support interpreting ARMT+ and ACT Aspire test scores as measuring similar academic achievement in a subject and distinct academic achievements in different subjects, we want to see stronger convergent evidence (monotrait-heteromethod correlations) and weaker discriminant evidence (heterotrait- heteromethod and heterotrait-monomethod correlations).

In mathematics, we see some divergent evidence for ARMT+ in all grades except grade 8. For example, in each of grades 3–7, ARMT+ mathematics correlated more highly with ARMT+ reading than with ACT Aspire mathematics (heterotrait- monomethod). For grades 5 and 7, ARMT+ mathematics correlated more highly with ARMT+ science than with ACT Aspire mathematics. In addition, in some cases ARMT+ mathematics correlated more highly with ACT Aspire reading and science than with ACT Aspire mathematics (heterotrait-heteromethod). ACT Aspire mathematics scores also showed some divergent evidence. For example, ACT Aspire grade 3 mathematics scores were correlated more with ACT Aspire reading scores than with ARMT+ mathematics scores.

In reading, we see that the monotrait-heteromethod correlations between ARMT+ and ACT Aspire were relatively large (all are above .7) and for all but grades 7 and 8 these were the largest correlations in the matrices. Moreover, almost all of the heterotrait-monomethod correlations were smaller than the monotrait-heteromethod correlations. The heterotrait-heteromethod correlations were the smallest most of time. One anomaly in reading was that for grade 7, the correlation between reading test scores on ARMT+ and ACT Aspire was not as large as the correlation between (a) ARMT+ reading and ARMT+ science and (b) ACT Aspire reading and ACT Aspire science. In other words, the method effect is slightly stronger for grade 7 reading than the trait effect, at least when reading and science were considered. In general, though, these patterns of correlations indicate stronger convergent evidence than discriminant evidence across reading tests for ARMT+ and ACT Aspire.

In science, we see that the monotrait-heteromethod correlations between ARMT+ and ACT Aspire were not as large as many of the heterotrait-monomethod correlations. For example, for grade 5, the science correlation between ARMT+ and ACT Aspire was .65, all of the heterotrait-monomethod correlations for ARMT+ (i.e., those in the first three rows for grade 5) were .7 or higher, and the correlation

17.61 ACT Aspire Validity

between ACT Aspire reading and science was .75. In science, we see weaker convergent evidence than discriminant evidence, particularly for ARMT+. However, these results may not be surprising given that science is likely to draw on both reading and mathematics.

One possible explanation for the divergent evidence observed in science and mathematics is the effects of the test reliabilities on the correlations among variables. Table 17.23 lists the disattenuated correlations among ARMT+ and ACT Aspire scale scores. As mentioned earlier, these correlations are an estimate of the true score relationships after the reliabilities of the two scores are taken into account. If we consider the disattenuated correlations as an upper limit on the observed correlations between scores (i.e., those in Table 17.22), then the correlations shown in Table 17.23 are quite strong. For example, the correlation between ACT Aspire grade 4 mathematics and reading was .57 with a disattenuated correlation of .75, which was one of the largest differences between observed and disattenuated correlation. In addition, scanning through Table 17.23 reveals that the pattern of convergent evidence is improved and the divergent evidence somewhat weaker using disattenuated correlations, which is indicative that test reliability may explain part of the stronger divergent evidence for mathematics and science. For example, grade 7 ARMT+ mathematics was most strongly correlated with ARMT+ science and reading in Table 17.22 (divergent evidence), but the largest disattenuated correlation for grade 7 ARMT+ mathematics was between mathematics scores for ARMT+ and ACT Aspire.

To summarize, we see relatively strong positive relationships between ACT Aspire and ARMT+ scores in the same subject, particularly if we account for the reliabilities of scores. In addition, we see stronger convergent evidence that ARMT+ and ACT Aspire are measuring similar traits in reading. In mathematics and science, we see divergent evidence in grades 3–7 indicative of method effects. The results are consistent with an interpretation of mathematics and science subjects as being particularly distinct from one another across the ARMT+ and ACT Aspire tests, but the reliabilities of some tests likely exaggerated such divergence. Test design difference could also explain the divergent evidence observed in science. ARMT+ science only included selected-response items, whereas ACT Aspire also included constructed-response items. However, mathematics and reading tests for ARMT+ and ACT Aspire contained selected-response items and constructed-response items, so this explanation may not apply to what was observed for mathematics tests.

17.62 ACT Aspire Validity

Table 17.23. Disattenuated Correlations between ARMT+ and ACT Aspire Scale Scores by Grade Level

Alabama ACT Aspire

Grade (valid N) Scale Score Mathematics Reading Science Mathematics Reading Science

Grade 3 Alabama Mathematics (8,530) Alabama Reading .78

ACT Aspire Mathematics .71 .76

ACT Aspire Reading .64 .84 .81

Grade 4 Alabama Mathematics (8,585) Alabama Reading .77

ACT Aspire Mathematics .77 .72

ACT Aspire Reading .70 .88 .75

Grade 5 Alabama Mathematics (6,543) Alabama Reading .80

Alabama Science .76 .76

ACT Aspire Mathematics .78 .78 .63

ACT Aspire Reading .74 .87 .72 .76

ACT Aspire Science .79 .83 .73 .82 .89

Grade 6 Alabama Mathematics (5,804) Alabama Reading .80

ACT Aspire Mathematics .75 .64

ACT Aspire Reading .73 .85 .68

Grade 7 Alabama Mathematics (4,345) Alabama Reading .79

Alabama Science .80 .80

ACT Aspire Mathematics .83 .70 .70

ACT Aspire Reading .77 .84 .79 .79

ACT Aspire Science .80 .79 .76 .83 .89

Grade 8 Alabama Mathematics (3,876) Alabama Reading .81

ACT Aspire Mathematics .91 .78

ACT Aspire Reading .72 .85 .77

17.63 ACT Aspire Validity

From a validity perspective, these results support the argument that ACT Aspire and ARMT+ measure academic achievement in mathematics, reading, and science. However, these results also support the argument that ARMT+ and ACT Aspire are not the same and likely to measure different aspects within the broad domain of academic achievement in each subject area.

17.2.5 Evidence Regarding Relationships with Criteria Evidence regarding relationships with criteria refers to evaluating the relationship between scores on test items compared to one or more criterion variables. One of the primary interpretations of ACT Aspire scores is to identify students’ readiness on (a) a college readiness trajectory and (b) a career readiness trajectory. This implies that ACT Aspire scores can successfully predict college readiness and career readiness. Some evidence regarding relationships with college and career readiness are available in Chapters 14 (growth) and 15 (career readiness). Additional evidence using results from growth analysis that includes predictions between ACT Aspire and the ACT scores is included in Allen (2014). However, evidence from longitudinal data connecting performance on ACT Aspire to eventual performance in college and career are not yet available for ACT Aspire.

17.2.6 Evidence Based on Consequences of Tests Evidence based on consequences of tests refers to evaluating whether unintended consequences are influenced by or attributable to unintended characteristics or underrepresentation of the intended construct for particular students (or groups of students).

For example, cognitive characteristics like academic achievement, as measured by ACT Aspire, are not the only characteristic indicative of readiness on a college and career readiness trajectory. Other characteristics, such as noncognitive variables (e.g., behaviors, attitudes, and education and career planning skills) have also been shown to contribute to student success (Mattern, Burrus, Camara, O’Connor, Hansen, Gambrell, Casillas, & Bobek, 2014). An unintended consequence of using ACT Aspire performance as indicative of readiness for college and career may be to perpetuate an over emphasis on the cognitive contribution and an underemphasis on important, but frequently difficult to measure, noncognitive contributions.

Providing instructionally actionable information to educators may have unintended consequences. If used in an extreme way, such information could lead to what is often referred to as “teaching to the test”. ACT Aspire is a summative assessment of academic achievement that is built to particular specifications that are based on standards and ultimately sample from broad domains (English, mathematics, reading, science, and writing). However, the collection of items that appear on the ACT Aspire test forms are a sampling of content, as defined by the test specifications, from the broad content domain. There is a risk that if the instructionally actionable information is too specific, it may lead educators to pay less attention to or ignore those aspects of the domain that are less heavily emphasized by the ACT Aspire test specifications. 17. 6 4 ACT Aspire Validity

Another aspect of unintended consequences is potential performance differences across particular groupings or characteristics of students. Where such differences occur, they would need to be carefully considered to ensure that they are not explainable by some unintended characteristic. This will be an ongoing area of inquiry for ACT Aspire.

In each of these examples of unintended consequences, we need to evaluate the extent to which construct underrepresentation or unintended characteristics of the construct contribute to these consequences. At this early point in the usage of ACT Aspire assessments, it is difficult to adequately evaluate consequences because data for evaluating consequences are not always immediately available. However, the entire testing cycle, from determining the content domains to evaluating the results from test administrations has been established to reduce or eliminate problematic sources of construct underrepresentation or unintended characteristics. Descriptions of these principles, processes and procedures are included in various chapters of this manual, including test development, test administration, reporting, and scaling.

In addition, unintended consequences of ACT Aspire scores cannot always be anticipated, so the examples and descriptions above should be viewed as a sampling of unintended consequences rather than a comprehensive list. ACT will continue to review and monitor consequences and uses of ACT Aspire performance and collect data that can be used to evaluate unintended consequences.

17.3 Summary This chapter provided evidence regarding the validity of ACT Aspire scores. The interpretation of ACT Aspire scale scores as measures of academic achievement is foundational to the primary and secondary interpretations of ACT Aspire performance. However, much work on the validation of ACT Aspire score interpretations for proposed uses remains. This is especially true for a relatively new testing program like ACT Aspire where the body of evidence must be established.

This chapter presented and summarized two studies that provided evidence regarding the relationships of ACT Aspire scores with scores on other assessments. However, several types of evidence need to be addressed directly to support further validation of ACT Aspire scores, including evidence based on test content, evidence based on response processes, evidence based on internal test structure, evidence based on relations to other variables (specifically, test-criterion relationships), and evidence considering the consequences of testing. Evidence will continue to be collected to support the interpretations of ACT Aspire scores.

17.65

CHAPTER 18 Fairness

The chapter contains evidence to address the fairness of ACT Aspire assessments. Three aspects of fairness are considered: fairness during the testing process, fairness in test accessibility, and fairness in score interpretations for intended uses.

18.1 Overview Fairness is a fundamental consideration in the interpretation of test scores for particular uses. However, the term fairness is used in many ways and does not have a single technical definition. In addition, a comprehensive evaluation of fairness would consider the full range of goals implied by a testing program which could extend beyond the interpretations of scores for particular uses. Such goals could include social goals and legal requirements (AERA et al., 2014; Camilli, 2006).

This chapter adopts the perspective on fairness included in The Standards for Educational and Psychological Testing (AERA et al., 2014), which focuses on tests, testing, and test use when evaluating fairness. Three aspects of fairness are considered: fairness during the testing process, fairness in test accessibility, and fairness in score interpretations for intended uses. Each aspect of fairness is described in more detail in separate sections below.

Similar to validity, evaluation of fairness is ongoing, with evidence being collected and reported as it becomes available. Evidence regarding fairness with ACT Aspire assessments is not limited to this chapter and can be drawn from other chapters of this manual and other ACT Aspire documentation although we have attempted to present or reference relevant available evidence for evaluating fairness here.

18.1 Fairness

18.2 Fairness During the Testing Process Fairness during the testing process refers to students being tested in a way that optimizes their capability of showing what they know and can do. In other words, the entire testing process, from test design to scoring, facilitates students being able to perform their best and does not adversely affect the performance of particular students (or groups of students). ACT Aspire design, development, and scoring are conducted in a way that attempts to be sensitive to issues of fairness. Earlier chapters regarding test development, scoring, accessibility, scores, and reporting illustrate this commitment to fairness during the testing process.

18.3 Fairness in Test Accessibility Accessibility in the context of fairness refers to the extent to which test takers can access the knowledge, skills, and/or abilities intended to be measured by the test without being unduly burdened by aspects of the test or test administration that may affect or limit access. For example, a student with a visual impairment may not be able to appropriately answer questions on a mathematics achievement test if the test material cannot be seen. ACT Aspire includes a variety of accessibility options for students, additional details can be found in Chapter 5.

18.4 Fairness in Score Interpretations for Intended Uses The validity of score interpretations for the intended population served by an assessment is another consideration for fairness. It involves studying the validity of test score interpretations for different groupings of students but also implies considering individuals, especially when scores are interpreted for individual students.

When evaluating validity in the context of fairness, it is particularly important to consider the extent to which characteristics of the test may be under-represented or irrelevant to score interpretations for particular uses and may differentially affect groups taking the test. These characteristics are often referred to as construct underrepresentation or construct-irrelevant components (e.g., AERA et al., 2014) and may lead to scores having different meaning, and therefore differential validity, across groups. Evaluation of validity across groups can involve evidence from such analysis as differential item functioning, differential test functioning, and predictive bias (e.g., AERA et al., pp. 51-52). Empirical evidence across groupings of test takers is not yet available for inclusion in this technical manual. However, the development of ACT Aspire tests includes processes and procedures that attempt to reduce characteristics of the tests that could adversely affect the validity of test score interpretations across groups (e.g., see Chapter 2).

Evaluating fairness at the individual test score level is particularly important for tests like ACT Aspire, where score interpretations may be applied at the individual level and uses may have implications for individual test takers. There are two aspects of validity

18.2 Fairness

at the individual student level. One is whether the test can support interpretations of scores for intended uses that involve the individual student. For example, ACT Aspire scores are intended to provide instructionally actionable information, which could imply making inferences for individual students. Collection of data to evaluate such evidence is ongoing for ACT Aspire and relevant validity evidence can be found in Chapter 17.

A second aspect of validity at the individual student level is that the interpretation of a score for a particular student is valid for its intended use. For example, test users should consider scores and score interpretations from ACT Aspire in the context of what they know about their students and consider how valid they judge particular interpretations of scores for intended uses to be for each student. It may be that a teacher, parent, student or other test user knows that scores are not appropriately reflective of a student’s achievement or that a student needs additional supports (e.g., test accommodations) for the student’s scores to be appropriately interpreted. Hopefully, consideration of student needs during testing can be addressed so that interpretations of scores for the student are valid for their intended uses, but test users should keep validity in mind when interpreting individual student performance. Best practices suggest using multiple sources of information when making decisions that affect individuals.

18.3

CHAPTER 19 ACT Aspire Mode Comparability Study

This chapter describes a mode comparability study comparing ACT Aspire paper and online forms. The study included data from a special administration of ACT Aspire that involved a randomly equivalent group design. Because online and paper test forms are not 100% the same, test forms are statistically linked prior to obtaining scale scores to adjust for differences across mode. This chapter examines potential mode effects for the same items across mode and examines the comparability of scale scores after statistical linking. The results indicate that it is not reasonable to ignore possible mode effects, but after linking, scales scores across mode appeared comparable.

19.1 Introduction A mode comparability study was undertaken to investigate ACT Aspire performance across online and paper modes. Most test forms included in this study were not identical across modes. This led us to assume that scores from paper and online forms were not interchangeable without first statistically linking test forms across modes. Two primary purposes of the comparability study included (1) determining whether mode of administration affected student performance on common (i.e., identical) items across modes prior to scaling and (2) investigating the comparability of ACT Aspire scale scores, which were obtained after linking paper and online forms. The overarching question to address with the comparability study was whether the validity of the interpretations of test scores is similar across paper and online test forms. In this chapter we describe comparisons of raw number-of-points scores (on identical items), item-level performance, and scale scores across modes.

19.1 ACT Aspire Mode Comparability Study

Studying the effects of mode on performance of collections of identical items gives us an idea of direction and degree of differences due to mode. This, in turn, can help us determine whether performance on collections of identical items across modes could be considered interchangeable without statistical linking across modes to moderate potential mode effects. Investigating the comparability of scale scores after linking forms across modes, where items on forms are not 100% the same but may have most of the same items, provides us with evidence regarding (a) the effectiveness of the linking and (b) the apparent interchangeability of scale scores across forms administered in different modes. When a testing program maintains test forms administered in different modes and scores are to be used interchangeably across forms, it is incumbent on those maintaining the test to show that scale scores are indeed comparable across forms (e.g., see standard 5.17 of the Standards for Educational and Psychological Testing; AERA et al., 2014, p. 106).

19.2 Method and Results Test materials included one online and one paper ACT Aspire English, mathematics, reading, science, and writing test form for grades 3–10. At least 80% of test items were the same1 across modes and consisted of multiple-choice and constructed- response item types.2 However, most online test forms also contained a small number of technology-enhanced and fill-in-the-blank items that were not amenable to paper- based presentation. Paper forms contained analogous multiple-choice items covering the same content, but these items were not considered identical across forms due to item-type differences.3 Table 19.1 shows the percentages of items considered different across modes. We handled these nonidentical items differently depending on the purpose of our analysis. For studying mode effects on raw number-of-points scores, we excluded such items and focused on the common (identical) items to gauge mode effects. For studying the comparability of scale scores across forms (by mode) we included the nonidentical items because scale scores are obtained using all operational items on a test form.

Participants in this study included students in grades 3 through 10 from a sample representing 13 states, 72 districts, and 108 schools. Table 19.2 identifies sample sizes for each subject by grade and mode. Sample sizes ranged from 645 (online grade 8 writing) to 1,539 (paper grade 3 mathematics). Additional details regarding the samples are provided in Tables 19.3–19.7, specifically percentages of demographic groupings by grade and mode.

1 Items contained identical content and required a consistent response process (e.g., selecting an option) from examinees. 2 English forms did not contain constructed-response items. 3 A separate study will explore nonoverlapping items across paper and online modes. 19.2 ACT Aspire Mode Comparability Study

Table 19.1. Percentage of Items Different Across Online and Paper Forms

Grade

Subject 3 4 5 6 7 8 EHS

English 16% 20% 16% 9% 6% — —

Mathematics 20% 16% 12% 18% 15% 8% 5%

Reading 8% 8% 13% 8% 13% — —

Science 11% 14% 14% 13% 13% 9% 13%

Writing — — — — — — —

Note. Technology-enhanced items and fill-in-the blank items were included in online forms, with selected-response analogs on paper forms. — = all items identical across modes (writing was based on a single prompt).

Table 19.2. Sample Sizes by Grade and Subject Grade

3 4 5 6 7 8 EHS

Subject O P O P O P O P O P O P O P

English 1,359 1,405 1,480 1,489 1,349 1,346 1,129 1,166 1,019 1,033 866 887 1,176 1,209

Mathematics 1,488 1,539 1,534 1,532 1,450 1,427 1,156 1,194 974 1,010 896 927 1,075 1,098

Reading 1,441 1,513 1,399 1,401 1,299 1,294 1,004 1,038 1,010 1,037 800 831 857 919

Science 1,376 1,405 1,435 1,444 1,378 1,379 1,047 1,040 1,028 1,042 913 934 959 1,023

Writing 1,189 1,268 1,260 1,280 1,101 1,102 840 885 825 836 645 658 698 767

Note. O = online form, P = paper form.

19.3 ACT Aspire Mode Comparability Study

19.3 Spring 2013 Comparability Study Demographic Summary Tables Table 19.3. Percentages of Students Taking English Tests in the Comparability Study by Grade, Mode and Demographics 6 2 6 5 5 1 2 7 P 18 36 45 28 47 58 33 (1,209) EHS 7 1 5 5 5 1 2 7 O 17 36 47 26 49 60 31 (1,176) 8 3 6 7 6 1 P 20 41 40 24 44 54 13 22 11 (887) 8 7 3 6 6 6 2 O 19 42 39 24 48 54 13 22 11 (866) 7 2 3 3 7 1 6 P 17 43 41 25 52 56 21 17 (1,033) 7 6 2 4 4 6 1 5 O 16 42 42 25 52 55 22 18 (1,019) 5 3 6 3 7 1 9 P 14 41 45 27 47 54 22 16

(1,166) 6 (N) Mode Grade 6 2 5 4 8 1 9 O 14 42 43 28 46 51 23 17 (1,129) 4 1 3 2 8 1 8 7 P 14 43 43 35 45 64 20 (1,346) 5 3 2 2 3 8 1 8 7 O 14 40 45 35 46 65 21 (1,349) 3 1 3 2 2 1 7 P 15 42 44 42 45 69 13 12 (1,489) 4 4 2 3 2 3 1 6 O 15 40 45 41 46 70 13 11 (1,480) 3 3 3 1 2 0 8 6 P 19 41 40 43 44 74 12 (1,405) 3 3 2 3 1 2 0 9 6 O 19 40 41 42 47 73 12 (1,359)

O = online, P = paper. Zero indicates fewer than 0.5%. Missing values indicate no students for a cell. Percentages within a demographic grouping may not add up to 100% within a demographic grouping may not add up to 100% than 0.5%. Missing values indicate no students for a cell. Percentages Zero indicates fewer O = online, P paper. Not Specified Female Male Not Specified Black/African American American Indian/ Alaskan Native White Hispanic/Latino Asian Native Hawaiian/Other Pacific Islander Two or More Races East Midwest Southwest West

19.4 Demographic Grouping % Gender Race/Ethnicity Geographic Region Note. due to rounding. ACT Aspire Mode Comparability Study

Table 19.4. Percentages of Students Taking Mathematics Tests in the Comparability Study by Grade, Mode and Demographics 6 2 6 6 6 1 2 7 P 15 38 48 22 51 59 31 (1,098) EHS 5 2 6 6 5 1 2 8 O 14 37 49 21 54 60 30 (1,075) 8 3 6 7 5 1 P 19 41 40 24 46 56 12 21 11 (927) 8 6 3 5 5 6 1 O 19 42 39 23 49 56 13 19 12 (896) 8 2 3 3 7 1 6 P 17 42 41 26 50 55 21 18 (1,010) 7 6 2 3 4 6 1 6 O 17 40 42 26 51 56 20 17 (974) 5 3 3 3 7 1 5 P 14 42 45 36 43 50 30 15

(1,194) 6 (N) Mode Grade 5 2 3 3 6 1 5 O 14 43 43 37 42 50 29 16 (1,156) 8 7 1 2 2 7 1 6 P 14 43 43 33 45 67 19 (1,427) 5 7 6 2 2 3 7 1 7 O 14 40 46 33 46 67 20 (1,450) 5 2 3 2 2 1 6 P 16 41 43 42 43 72 11 11 (1,532) 4 6 2 3 2 2 0 6 O 16 40 44 41 43 71 12 10 (1,534) 5 2 2 3 3 1 4 P 17 42 42 42 42 75 11 10 (1,539) 3 5 2 2 2 3 0 3 O 17 40 42 42 44 74 11 11 (1,488)

O = online, P = paper. Zero indicates fewer than 0.5%. Missing values indicate no students for a cell. Percentages within a demographic grouping may not add up to 100% due to within a demographic grouping may not add up to 100% than 0.5%. Missing values indicate no students for a cell. Percentages Zero indicates fewer O = online, P paper. Not Specified Female Male Not Specified Black/African American American Indian/Alaskan Native White Hispanic/Latino Asian Native Hawaiian/Other Pacific Islander Two or More Races East Midwest Southwest West

Demographic Grouping % Gender Race/Ethnicity Geographic Region Note. rounding. 19.5 ACT Aspire Mode Comparability Study

Table 19.5. Percentages of Students Taking Reading Tests in the Comparability Study by Grade, Mode and Demographics 7 3 7 6 6 1 3 9 P 18 33 49 27 43 52 36 (919) EHS 7 3 7 7 6 2 3 9 O 18 35 47 27 41 53 35 (857) 9 3 6 6 6 1 P 21 39 39 25 43 50 15 24 12 (831) 8 8 3 6 5 7 1 O 20 41 39 24 46 49 15 23 13 (800) 7 2 6 4 4 1 7 P 14 44 42 25 50 54 22 17 (1,037) 7 6 3 5 5 3 1 7 O 13 43 44 25 52 54 23 16 (1,010) 6 3 4 3 6 1 P 13 41 46 32 45 58 25 17

(1,038) 6 (N) Mode Grade 6 2 3 3 5 2 O 13 44 43 32 47 58 24 18 (1,004) 7 1 1 2 6 1 4 P 13 43 44 33 48 74 21 (1,294) 5 6 2 1 3 7 1 4 O 13 40 47 32 48 74 22 (1,299) 4 2 1 2 1 1 7 P 16 41 43 43 47 79 14 (1,401) 4 6 2 1 1 1 1 8 O 16 40 44 42 46 78 14 (1,399) 6 2 2 3 1 1 P 15 42 42 44 42 77 11 11 (1,513) 3 5 2 1 2 2 0 O 15 42 43 42 45 77 12 11 (1,441)

Table 19.6. Percentages of Students Taking Science Tests in the Comparability Study by Grade, Mode and Demographics 4 3 6 5 6 2 2 8 P 21 33 46 34 41 45 45 (1,023) EHS 5 2 6 7 5 1 3 8 O 21 34 45 33 41 45 45 (959) 8 3 6 6 5 1 P 19 41 40 23 47 55 13 21 11 (934) 8 7 4 5 6 6 1 O 18 43 39 23 49 55 13 21 11 (913) 7 2 3 3 7 2 6 P 17 43 40 27 50 53 24 17 (1,042) 7 6 3 3 4 6 1 5 O 16 42 43 27 50 54 25 16 (1,028) 6 3 5 3 5 1 5 P 16 40 45 31 46 57 20 18

(1,040) 6 (N) Mode Grade 6 2 5 4 4 2 7 O 15 42 43 31 46 55 21 18 (1,047) 8 3 1 1 2 8 1 2 P 14 42 45 36 47 68 21 (1,379) 5 8 2 2 2 3 8 1 2 O 14 40 46 35 48 68 22 (1,378) 1 2 2 2 2 1 2 P 15 41 44 43 48 73 13 12 (1,444) 4 2 2 2 2 3 1 2 O 15 40 45 42 47 74 14 11 (1,435) 1 3 2 3 3 1 2 P 18 41 41 47 40 74 12 12 (1,405) 3 1 2 2 2 3 0 2 O 19 40 42 48 42 73 13 12 (1,376)

Table 19.7. Percentages of Students Taking Writing Tests in the Comparability Study by Grade, Mode and Demographics 6 3 5 0 0 1 3 P 27 28 45 38 47 44 53 (767) EHS 5 3 4 1 1 3 O 26 29 45 39 47 44 53 (698) 9 4 3 3 0 1 P 23 39 38 31 48 55 17 28 (658) 8 7 4 3 2 2 O 21 42 37 30 51 53 18 30 (645) 7 2 3 1 1 1 P 17 43 40 31 53 53 26 21 (836) 7 6 3 4 1 1 1 O 16 42 42 29 55 51 26 23 (825) 5 3 4 1 4 1 P 15 41 44 36 45 51 28 20

(885) 6 (N) Mode Grade 6 3 3 1 3 1 O 16 41 43 37 46 52 27 22 (840) 3 2 1 0 5 1 P 12 44 44 40 48 65 25 10 (1,102) 5 3 2 1 0 6 1 O 13 41 46 40 47 66 25 10 (1,101) 2 2 2 1 0 1 P 13 42 45 47 46 73 15 12 (1,280) 4 3 2 2 0 0 0 O 14 41 45 46 46 73 16 11 (1,260) 3 2 2 1 0 0 P 16 42 41 49 42 75 13 12 (1,268) 3 3 2 1 1 0 0 O 17 41 41 47 45 74 14 12 (1,189)

Students were recruited to complete an ACT Aspire test in one or more subjects. Students at each grade within a school were randomly assigned to take either the paper or the online version of the test. This design is called random equivalent groups and ensured that recruited students had an equal chance of being assigned the online or paper test within a school. If students testing in each mode are equivalent, we can attribute observed differences in performance to differences in testing mode, not to differences in groups of students (or a combination of group and performance differences). To help ensure our analysis included adequately balanced samples of students testing in each mode, schools were included in the analysis sample only if the ratio of students testing in one mode compared to the other within the school was less than two, implying that fewer than twice as many students tested in one mode versus the other. This data cleaning rule led to excluding fewer than 10% of students from analysis for most grades and subjects.4 Additional details regarding the samples, including demographic characteristics, are included in Tables 19.3–19.7.

19.4 Mode Effects for Raw Scores on Common Items We investigated student raw score performance by summing scores across items that were the same across paper and online forms (i.e., excluding those items mentioned earlier that differed in item type across modes). English and writing showed more consistent evidence of mode effects across grades compared to mathematics, reading, and science. But for some subjects and grades, raw scores appeared to differ across modes, and for others they did not. It is apparent from these results that it was not safe to assume raw scores and raw score distributions on identical items across modes were comparable. Tables 19.8–19.12 summarize the aggregated raw score statistics for the common items across modes, including score moments (mean, standard deviation, skewness, and kurtosis), effect sizes (standardized differences in mean scores), and statistical significance tests from tests of equivalence (for details on this test, see Rogers, Howard & Vessey, 1993; Serlin & Lapsley, 1985; see also the Kolmogorov-Smirnov test of differences in cumulative distributions by subject).

4 Up to 22% of students (five schools) were excluded from analysis. Grade 7 science and Grade 8 writing had more than 20% of students excluded. Most excluded schools only tested in one mode. 19.9 ACT Aspire Mode Comparability Study

Table 19.8. English Common-Item Raw Score Summary for Online and Paper Modes 3 P 50 50 .90 9.81 0.16 -0.75 1,209 25.54 0.02 0.56 EHS (2,383) 4 2.99/-1.96* O 50 49 .91 9.96 0.20 -0.82 1,176 25.75 3 P 35 34 .86 887 6.52 -statistic for the difference in means greater -statistic for the difference -0.43 -0.54 22.06 t 8 1.12 -0.02 (1,751) is the 5 O 2.84/-3.85* 35 34 .83 866 5.97 -0.32 -0.51 21.91 t-lower 2 P 33 33 .81 -tests. 5.85 0.24 t -0.49 1,033 16.41 7 0.18 1.90* (2,050) 7.84/0.14 4 O 33 32 .81 5.92 0.02 -0.62 1,019 17.44 0 P 32 30 .80 5.32 0.30 -0.38 1,166 14.84 6 0.32 Grade 3.47* (2,293) 3 O 12.08/3.08 32 32 .80 5.32 0.14 -0.34 1,129 16.52 0 P 21 20 .78 4.04 0.09 -0.67 1,346 10.61 5 0.15 1.66* (2,693) 0 O 21 21 .79 10.14/-2.54* 4.15 0.11 -0.71 1,349 11.21 1 P 20 20 .78 3.77 in means across modes. This test is comprised of two one sided in means across modes. This -0.11 -0.57 1,489 11.06 4 0.11 1.45* (2,967) 1 O 20 20 .75 10.35/-4.62* 3.50 -0.09 -0.65 1,480 11.44 no difference 1 P 21 20 .78 4.05 0.36 -0.64 1,405 10.07 3 0.39 4.68* -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or greater these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference (2,762) t 1 O 16.58/3.73 21 21 .79 4.14 0.00 -0.68 1,359 11.65 is the -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference t † t-upper -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Reliability Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t Number of common items Sample size Raw score Stat. test (df) Note that statistical significance indicates than -1 and than 1. If both Note. * Statistically significant at .05. † 19.10 ACT Aspire Mode Comparability Study

Table 19.9. Mathematics Common-Item Raw Score Summary for Online and Paper Modes 1 P 36 45 .87 8.15 0.45 -0.46 1,098 18.14 0.73 EHS -0.07 (2,171) 1.32/-4.38 3 O 36 44 .88 8.21 0.48 -0.47 1,075 17.60 1 P 35 39 .87 927 7.18 0.03 -statistic for the difference in means greater -statistic for the difference -0.55 19.75 t 8 -0.11 1.47* (1,821) is the 1 0.63/-5.32 O 35 42 .87 896 7.16 0.25 -0.37 18.96 t-lower 1 P 29 32 .83 -tests. 5.57 0.55 t -0.03 1,010 13.91 7 1.25 -0.11 (1,982) 2 1.62/-6.41 O 29 34 .82 974 5.51 0.61 0.20 13.31 1 P 28 29 .73 4.11 0.36 0.25 1,194 12.01 6 0.06 0.84 Grade (2,348) 2 O 7.42/-4.34* 28 31 .74 4.13 0.53 0.51 1,156 12.27 0 P 22 23 .63 9.99 3.34 0.32 0.24 1,427 5 0.00 0.96 (2,875) 1 O 7.92/-7.78* 22 26 .67 3.49 0.58 0.69 1,450 10.00 0 P 21 27 .62 9.48 3.39 0.62 1.12 in means across modes. This test is comprised of two one sided in means across modes. This 1,532 4 0.02 0.43 (3,064) 1 O 8.54/-7.63* 21 25 .64 9.53 3.46 0.60 1.00 1,534 no difference 1 P 20 27 .75 4.77 0.16 -0.41 1,539 12.61 3 1.05 -0.04 -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or greater these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference (3,025) t 0 O 4.74/-6.91* 20 26 .76 4.68 0.17 -0.29 1,488 12.42 is the -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference t † t-upper -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Reliability Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Number of common items Sample size Raw score Stat. test Note that statistical significance indicates than -1 and than 1. If both Note. * Statistically significant at .05. † 19.11 ACT Aspire Mode Comparability Study

Table 19.10. Reading Common-Item Raw Score Summary for Online and Paper Modes 1 P 31 24 .88 919 6.99 -0.02 -1.10 15.20 0.00 0.47 EHS (1,774) 1 2.94/-3.14* O 30 24 .87 857 6.87 0.01 -1.07 15.17 1 P 30 24 .84 831 6.41 -statistic for the difference in means -statistic for the difference -0.16 -0.86 15.96 t 8 0.07 0.73 (1,629) is the 1 O 4.57/-1.81* 29 24 .83 800 6.25 -0.04 -0.88 16.39 t-lower 0 P 24 21 .81 -tests. 5.31 0.04 t -0.94 1,037 12.31 7 0.96 -0.07 (2,045) 1 O 2.79/-5.81* 25 21 .80 5.20 0.15 -0.81 1,010 11.96 2 P 26 22 .81 5.15 -0.21 -0.82 1,038 14.69 6 0.68 Grade -0.03 (2,040) 2 O 3.68/-4.85* 26 22 .83 5.44 -0.11 -0.91 1,004 14.56 1 P 24 21 .83 5.03 -0.08 -0.81 1,294 12.82 5 0.02 0.52 (2,591) 1 O 5.59/-4.44* 25 21 .83 5.13 0.05 -0.82 1,299 12.94 1 P 22 26 .83 5.69 in means across modes. This test is comprised of two one sided in means across modes. This -0.87 -0.14 1,401 14.32 4 -0.18 2.48* (2,798) 1 O -0.09/-9.42 22 26 .83 5.66 0.10 -0.91 1,399 13.30 no difference 1 P -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference 22 26 .84 t 5.68 -0.88 -0.15 1,513 13.99 3 -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference t -0.19 2.76* is the (2,952) 1 O -0.31/-9.81 22 26 .85 5.75 0.09 -1.01 1,441 12.93 t-upper † -upper t O = online form, P paper form. -lower/ SD Minimum Maximum Skewness Kurtosis Reliability Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Mean Stat. test Number of common items Sample size Raw score Note that statistical significance indicates greater than -1 and -1 or greater than 1. If both Note. * Statistically significant at .05. †

19.12 ACT Aspire Mode Comparability Study

Table 19.11. Science Common-Item Raw Score Summary for Online and Paper Modes 1 P 36 28 .87 7.20 0.61 -0.33 1,023 14.04 EHS -0.12 1.69* (1,980) 0.46/-5.89 2 O 35 28 .87 959 6.80 0.76 -0.21 13.18 2 P 36 29 .87 934 7.28 0.15 -statistic for the difference in means -statistic for the difference -0.71 16.93 t 8 0.59 -0.02 (1,845) is the 2 O 2.60/-3.37* 36 29 .87 913 7.11 0.19 -0.78 16.80 t-lower 2 P 33 28 .89 -tests. 7.34 0.35 t -0.88 1,042 15.18 7 0.89 -0.03 (2,068) 1 O 2.34/-3.75* 35 28 .89 7.59 0.42 -0.80 1,028 14.95 1 P 35 28 .88 7.30 0.09 -0.86 1,040 17.04 6 0.03 0.59 Grade (2,085) 2 O 3.77/-2.48* 35 28 .88 7.30 0.21 -0.79 1,047 17.25 1 P 29 24 .85 6.25 0.02 -0.84 1,379 14.97 5 0.03 0.54 (2,755) 1 O 5.02/-3.36* 32 24 .85 6.28 0.02 -0.80 1,378 15.17 1 P 24 30 .84 5.74 0.23 in means across modes. This test is comprised of two one sided in means across modes. This -0.57 1,444 14.15 4 0.71 -0.04 (2,877) 1 O 3.73/-5.71* 24 30 .84 5.63 0.27 -0.54 1,435 13.94 no difference 1 P -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference 25 32 .88 t 6.77 0.52 -0.51 1,405 12.69 3 1.22 -0.04 is the (2,779) 0 O 2.81/-4.81* 25 32 .88 -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference 7.07 0.52 t -0.60 1,376 12.43 t-upper † -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Reliability Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Raw score Stat. test Number of common items Sample size Note that statistical significance indicates greater than -1 and greater than 1. If both Note. * Statistically significant at .05. †

19.13 ACT Aspire Mode Comparability Study

Table 19.12. Writing Common-Item Raw Score Summary for Online and Paper Modes 1 4 P 24 767 3.57 0.04 0.10 12.27 0.03 0.86 EHS (1,463) 1 4 5.87/-4.72* O 24 698 3.65 0.02 0.12 12.38 1 4 P 24 658 3.18 0.31 -statistic for the difference in means -statistic for the difference -0.01 11.94 t 8 0.07 1.59* (1,301) is the 1 4 O 6.67/-4.14* 23 3.5 645 -0.20 -0.20 12.18 t-lower 1 4 P 24 -tests. 836 3.33 0.17 0.21 t 13.29 7 -0.25 2.64* (1,659) 1 4 0.49/-10.8 O 24 825 3.87 0.27 0.09 12.38 1 4 P 24 885 3.44 -0.37 -0.03 14.48 6 Grade -0.25 3.12* (1,723) 1 4 O 23 0.39/-10.84 840 3.95 -0.14 -0.32 13.55 1 4 P 20 3.27 -0.13 -0.16 1,102 12.21 5 -0.22 2.65* (2,201) 4 1 O 20 1.49/-11.87 3.74 0.08 -0.28 1,101 11.44 4 1 P 20 3.13 0.23 in means across modes. This test is comprised of two one sided in means across modes. This -0.33 1,280 10.72 4 0.14 2.40* (2,538) 4 1 O 20 10.92/-4.11* 3.56 -0.07 -0.52 1,260 11.18 no difference 4 1 P -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference 20 t 3.01 0.03 11.8 -0.20 1,268 3 -0.21 3.27* is the (2,455) 4 1 O 20 2.29/-12.85* -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference 3.53 0.21 11.1 t -0.37 1,189 t-upper † -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t Number of common items Sample size Raw score Stat. test (df) Note that statistical significance indicates greater than -1 and greater than 1. If both Note. * Statistically significant at .05. † 19.14 ACT Aspire Mode Comparability Study

Figure 19.1 displays the cumulative percentage of students at each raw score by mode for each subject and grade. The solid curve represents the online form and the dashed curve represents the paper form. When one of the plotted cumulative percentage curves is to the left of the other, it indicates that this group scored lower relative to the group to the right (or, conversely, the group to the right scored higher). If curves cross or if the relative position of curves varies across raw scores, it indicates that the cumulative percentage of students between groups varies across scores. The distance between the two curves indicates the magnitude of difference between modes (difference in cumulative percentage in the vertical direction, difference in raw score in the horizontal direction). For example, for grade 3 English, the online cumulative percentage curve is to the right of the paper curve, which implies that online students scored higher. Reading up from a raw score point of 10, approximately 40% of online students scored 10 or lower, whereas approximately 60% of paper students scored 10 or lower. By subtracting these percentages from 100, one could say that approximately 60% of online students scored above a 10, but about 40% of students testing on paper scored above a 10. A statistical test of the differences in cumulative distributions is provided by the Kolmogorov-Smirnov test listed in Table 19.8.

Figure 19.1. Cumulative percentage of students for common item raw scores across modes

English Grade 3 Grade 4

19.15 ACT Aspire Mode Comparability Study

Figure 19.1. (continued) Grade 5 Grade 6

Grade 7 Grade 8

Early High School

19.16 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Mathematics Grade 3 Grade 4

Grade 5 Grade 6

19.17 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Grade 7 Grade 8

Early High School

19.18 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Reading Grade 3 Grade 4

Grade 5 Grade 6

19.19 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Grade 7 Grade 8

Early High School

19.20 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Science Grade 3 Grade 4

Grade 5 Grade 6

19.21 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Grade 7 Grade 8

Early High School

19.22 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Writing Grade 3 Grade 4

Grade 5 Grade 6

19.23 ACT Aspire Mode Comparability Study

Figure 19.1. (continued)

Grade 7 Grade 8

Early High School

19.24 ACT Aspire Mode Comparability Study

English raw scores and distributions showed evidence of differences across modes in grades 3, 5, 6, and 7, with means differing by more than one point, effect sizes between .15 and .39,5 statistically significant differences in score distributions, and nonstatistically significant similarities in mean scores. Although not consistent across statistics, grade 4 did show some evidence of mode effects with a statistically significant Kolmogorov-Smirnov test indicating that score distributions differed (see also Figure 19.1 for a plot of the score distributions). Online tests scored higher than paper tests for grades where differences were evident.

Mathematics raw scores and distributions showed evidence of differences across modes in grade 7, 8, and EHS, with the paper group scoring higher than the online group. The test of equivalence indicated that mean scores were not statistically similar. The score distributions differed statistically at grade 8 (see Figure 19.1, which plots the difference). However, the magnitudes of differences across modes were relatively small; means differed by less than one point and effect sizes were within ±.15.

Reading raw scores and distributions showed evidence of differences across modes in grades 3 and 4, with the paper group scoring higher than the online group. Means differed by more than one point, effect sizes were −.19 and −.18, differences in score distributions were statistically significant, and similarities in mean scores were not statistically significant.

Science raw scores and distributions did not appear to differ across modes for all but the EHS forms, where distributions were statistically different and means were not statistically similar. However, the difference was not large; means favored paper by less than one point and the effect size was −.12.

For writing raw scores and distributions, results showed differences across modes for all forms except EHS. Forms showed statistically significant differences in distributions in grades 3–8. Tests of equivalence did not show evidence of similarity of means for grades 3, 4, 5, 6, or 7. However, means differed by less than one point for all but grade 7. Effect sizes were within ±.15 for grades 4, 8 and EHS and were between −.20 and −.30 for grades 3, 4, 5, and 7. Paper tests scored higher than online tests in grades 3, 5, 6, and 7 and online tests scored higher than paper tests in grades 4 and 8.

In many cases, the different methods of checking mode effects for raw number- of-points scores on common items led to slightly different grades in each subject showing statistically significant mode effects. While typically not large, there were statistically significant differences across modes observed in each subject area.

5 We judged effect sizes within ±.15 to be small to negligible. 19.25 ACT Aspire Mode Comparability Study

Our interpretation of the comparisons of raw number-of-points scores based on identical items across modes is that mode effects observed for these ACT Aspire forms should not be ignored. Score differences were not always observed, and when they were observed they were generally not large, but we would argue that it is not reasonable to assume no mode effects across collections of identical items. If two ACT Aspire forms did contain 100% overlapping items, we would recommend additional statistical linking to adjust for mode effects. However, as mentioned earlier, most ACT Aspire online and paper forms are not 100% identical, so regardless of mode effects, best practices would involve linking to place forms on the same scale (Kingston 2009).

19.5 Item-level Performance across Modes To examine the comparability of item-level performance across modes, two sets of results are presented. Plots of item statistics are presented to provide an overview of the patterns of item-level performance across modes. Also, item statistics across modes are compared to gauge the degree to which individual item statistics were unusually high or low compared to all items.

Item statistics. Figures 19.2–19.5 display the mean number of points correct6 for common items across online (horizontal axis) and paper (vertical axis) forms. For multiple choice items, these statistics are often referred to as p-values. Each letter in the plot represents an item (M = multiple choice, X = constructed response). The plots illustrate patterns in the item difficulty for items across modes. For example, if the mean item performance did not differ across modes (i.e., online item mean = paper item mean), items would fall along the diagonal of the plot from lower left to upper right. Or, if one mode was harder than the other, we would see all of the items cluster above or below an imaginary line drawn through the diagonal (online = paper). We are unlikely to observe either pattern exactly across modes, but these plots can provide a better picture of the relationship between items across modes to identify (a) the strength of the relationship in item difficulty across modes and (b) whether there are particular items that we might consider unusually difficult or easy in one mode compared to the other.

Looking over the plots for English, mathematics, reading, and science, even though the scales on the plots for English have a smaller range, it is clear that the item mean differences across modes tended to be larger in English than in each of the other subjects. This is consistent with the raw number-of-points results described above, where English showed more evidence of mode effects. In addition, there were some items that appeared unusually difficult and the most pervasive differences appeared in English, particularly grades 3 and 6. To look more closely at differential effects across modes for items we used delta plots, which are described next.

6 For multiple choice items, these statistics are commonly referred to as p-values, which correspond to the proportion of students answering an item correct. 19.26 ACT Aspire Mode Comparability Study

Delta plots. Figure 19.6 displays delta plots of items across online and paper forms. Delta is a transformed item p-value7 that can be used to compare item difficulty statistics across examinee groupings (Angoff & Ford, 1973) and identify items that function differently across groups. In the case of comparability across modes, delta plots can be used to identify items that show unusually high or low performance in one mode compared to other items common across modes.

Deltas are calculated by first transforming the p-value (item difficulty) to a normal deviate (z) and then scaling z to a mean of 13 and a standard deviation of 4. Delta values can be plotted by group (in this case, online on one axis, paper on the other axis) and prediction line (or delta hat) estimated from the observed delta values (see Angoff & Ford, 1973 for details). In addition, confidence bands that indicate one- and two-standard errors above and below the predicted delta value can be estimated and used to indicate items that perform statistically differently across modes.

For our purposes here, we flagged items as performing differently when they fell outside of the two standard error band, which corresponds to a roughly 95% confidence interval. In addition, we were primarily interested in the pattern of delta values across items and the number of items flagged as performing differently across modes. Given that the number of items per form ranged from 20 to 50 (see Table 19.8), we would expect to flag 1 to 2 items per plot due to chance.

In general, all but one or two items fall within plus-or-minus two standard errors of the predicted delta value. Forms with two items flagged were those with relatively larger numbers of items (the number of items increases as grade level increases). Therefore, almost all grades and subjects showed little evidence that items were performing differently across modes.

Delta plots for English early high school forms had four items that were flagged as performing differently across modes. For three of the items, the online version appeared easier (or paper version appeared harder) and for one of the items the paper version appeared easier (or online version appeared harder). To put these differences in perspective, Table 19.13 lists the item p-values for English EHS items for the flagged items8. The item that appeared easier on paper (or harder online) was earlier in the test and the items that appeared easier online (or harder on paper) were the last three items on the test. A review of the items indicated that these items did indeed appear to be the same, except for differences due to mode of administration. For the last three items on the test, it is possible that the EHS English test on paper ended up being slightly more speeded than the online version, which resulted in the last three items being flagged as unusually difficult for the paper version.

7 See previous footnote for p-value explanation. For extended/constructed response items, item means were converted to mean proportion of points before converting to delta. 8 The p-value values and p-value differences for non-flagged items are not included in Table 19.13 but the differences between online and paper p-values for non-flagged English EHS items ranged from -.10 to .10. 19.27 ACT Aspire Mode Comparability Study

Considering the delta plot results along with the results from the item statistic plots illustrates two things. First, observed differences across modes at the raw-number- of-points correct score level were not explainable by specific items performing differentially well (or poorly) in one mode. In fact, with the exception of English EHS test, few items were observed that appeared to perform differentially across mode using delta plots. Second, while it wasn’t necessarily easy to see in the plots of item- level statistics, there did appear to be small but consistent mode effect across items for some grades and subjects, which are summarized in earlier tables comparing raw number-of-points correct score across modes.

Figure 19.2. English common item mean scores for paper and online forms by item type*

19.28 ACT Aspire Mode Comparability Study

Figure 19.2. (continued)

* Note that English only has multiple choice (MC) item type, which is represented by an “M” in the figure.

19.29 ACT Aspire Mode Comparability Study

Figure 19.3. Mathematics common item mean scores for paper and online forms by item type*

19.30 ACT Aspire Mode Comparability Study

Figure 19.3. (continued)

* Note that “M” = multiple choice (MC) and “X” = constructed response (XR) item type in the figure.

19.31 ACT Aspire Mode Comparability Study

Figure 19.4. Reading common item mean scores for paper and online forms by item type*

19.32 ACT Aspire Mode Comparability Study

Figure 19.4. (continued)

* Note that “M” = multiple choice (MC) and “X” = constructed response (XR) item type in the figure.

19.33 ACT Aspire Mode Comparability Study

Figure 19.5. Science common item mean scores for paper and online forms by item type*

19.34 ACT Aspire Mode Comparability Study

Figure 19.5. (continued)

* Note that “M” = multiple choice (MC) and “X” = constructed response (XR) item type in the figure.

19.35 ACT Aspire Mode Comparability Study

Figure 19.6. Delta plots for online and paper common items with predicted delta and one- and two-standard error bands.

English Grade 3 English Grade 4

English Grade 5 English Grade 6

19.36 ACT Aspire Mode Comparability Study

Figure 19.6. (continued)

English Grade 7 English Grade 8

English Early High School

19.37 ACT Aspire Mode Comparability Study

Figure 19.6. (continued)

Mathematics Grade 3 Mathematics Grade 4

Mathematics Grade 5 Mathematics Grade 6

19.38 ACT Aspire Mode Comparability Study

Figure 19.6. (continued)

Mathematics Grade 7 Mathematics Grade 8

Mathematics Early High School

19.39 ACT Aspire Mode Comparability Study

Figure 19.6. (continued)

Reading Grade 3 Reading Grade 4

Reading Grade 5 Reading Grade 6

19.40 ACT Aspire Mode Comparability Study

Figure 19.6. (continued)

Reading Grade 7 Reading Grade 8

Reading Early High School

19.41 ACT Aspire Mode Comparability Study

Figure 19.6. (continued)

Science Grade 3 Science Grade 4

Science Grade 5 Science Grade 6

19.42 ACT Aspire Mode Comparability Study

Figure 19.6. (continued)

Science Grade 7 Science Grade 8

Science Early High School

Table 19.13. ACT Aspire Spring 2013 Comparability Study P-values for English Early High School Items Flagged in Delta Plots

Flagged Online Paper Difference Item p-value p-value (Online – Paper)

1 .29 .46 –.17 2 .70 .52 .18 3 .41 .19 .22 4 .50 .35 .15

19.43 ACT Aspire Mode Comparability Study

19.6 Comparisons of Scale Scores across Modes We investigated the comparability of ACT Aspire scale scores, which were obtained after linking paper and online forms. Paper forms were linked to online forms using random equivalent groups equipercentile linking (Kolen & Brennan, 2014). Equipercentile methodology has been used extensively with other ACT testing programs, including the operational equating of ACT Aspire (see Chapter 20). The primary purpose of this particular linking was to statistically adjust for differences across forms due to item type and mode.

Tables 19.14–19.18 summarize scale score statistics, including score moments (mean, standard deviation, skewness, and kurtosis), effect sizes, statistical significance tests from tests of equivalence, and the Kolmogorov-Smirnov test of difference in cumulative distributions for each subject. Figure 19.7 displays the cumulative percentage of students at each scale score by mode for each subject and grade. Figure 19.8 displays box plots of scale scores by mode for each subject and grade.

19.44 ACT Aspire Mode Comparability Study

Table 19.14. English Scale Score Summary for Online and Paper Modes P 50 400 456 0.01 0.90 -0.60 1,209 10.66 428.05 0.02 0.64 EHS (2,383) -1.87/2.69* O 50 400 454 0.01 0.90 -0.69 1,176 10.75 428.23 P 35 887 401 447 8.79 0.86 -0.17 -0.39 427.64 -statistic for the difference in means -statistic for the difference t 8 0.00 0.94 (1,751) O -2.46/2.32* is the 35 866 403 448 8.74 0.84 -0.17 -0.40 427.61 t-lower P 35 400 450 8.95 0.82 -0.07 -0.34 1,033 -tests. 424.63 t 7 0.01 1.02 (2,050) O -2.27/2.80* 35 403 448 8.93 0.83 -0.08 -0.39 1,019 424.74 P 35 400 446 7.80 0.12 0.81 -0.24 1,166 424.22 6 0.00 1.11 Grade (2,293) O -3.07/3.10* 35 402 447 7.73 0.12 0.82 -0.21 1,129 424.23 P 25 404 441 7.05 0.25 0.79 -0.35 1,346 421.73 5 0.01 1.42 (2,693) O -3.44/3.87* 25 403 442 7.15 0.23 0.80 -0.44 1,349 421.79 P 25 405 438 6.13 0.11 0.79 -0.58 1,489 419.52 in means across modes. This test is comprised of two one sided in means across modes. This 4 0.02 1.26 (2,967) O -3.91/4.97* 25 402 438 6.14 0.08 0.76 -0.50 1,480 419.64 no difference P 25 401 434 5.86 0.49 0.79 -0.05 1,405 -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference t 416.38 3 1.41 -0.01 (2,762) is the O -4.79/4.17* 25 401 435 5.87 0.51 0.79 -0.09 1,359 -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference 416.32 t † t-upper -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Reliability Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Number of items Sample size Scale score Effect size Stat. test Note that statistical significance indicates greater than -1 and greater than 1. If both Note. * Statistically significant at .05. †

19.45 ACT Aspire Mode Comparability Study

Table 19.15. Mathematics Scale Score Summary for Online and Paper Modes P 38 405 451 8.73 0.35 0.88 -0.56 1,098 425.11 1.03 EHS -0.03 (2,171) -3.29/2.07* O 38 408 451 8.65 0.42 0.89 -0.50 1,075 424.88 P 38 927 403 444 7.90 0.32 0.88 -0.41 422.65 -statistic for the difference in means -statistic for the difference t 8 0.03 0.92 (1,821) O -1.97/3.43* is the 38 896 401 445 7.92 0.24 0.89 -0.37 422.92 t-lower P 34 403 442 6.67 0.39 0.02 0.83 1,010 -tests. 420.03 t 7 0.98 -0.05 (1,982) O -4.39/2.35* 34 974 404 443 6.53 0.46 0.08 0.84 419.73 P 34 404 442 5.68 0.40 0.20 0.77 1,194 419.87 6 0.00 Grade 1.50* (2,348) O -4.33/4.20* 34 404 445 5.69 0.45 0.27 0.78 1,156 419.85 P 25 403 435 4.75 0.39 0.36 0.69 1,427 417.37 5 0.02 1.95* (2,875) O -5.17/6.03* 25 404 438 4.83 0.53 0.58 0.70 1,450 417.44 P 25 402 433 4.18 0.19 0.58 0.66 1,532 414.99 in means across modes. This test is comprised of two one sided in means across modes. This 4 0.06 2.49* (3,064) O -4.94/8.36* 25 403 430 4.15 0.09 0.26 0.68 1,534 415.25 no difference P 25 400 427 4.04 0.77 -0.04 -0.03 1,539 -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference t 412.19 3 0.00 1.59* (3,025) is the O -6.75/6.99* 25 400 425 3.96 0.06 0.79 -0.02 1,488 -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference 412.21 t † t-upper -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Reliability Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Number of items Sample size Scale score Stat. test Note that statistical significance indicates greater than -1 and greater than 1. If both Note. * Statistically significant at .05. †

19.46 ACT Aspire Mode Comparability Study

Table 19.16. Reading Scale Score Summary for Online and Paper Modes P 24 919 403 442 7.89 0.02 0.88 -0.89 421.51 0.00 0.64 EHS (1,774) -2.74/2.60* O 24 857 403 440 7.87 0.88 -0.01 -0.90 421.48 P 24 831 402 440 7.55 0.85 -0.05 -0.78 421.07 -statistic for the difference in means -statistic for the difference t 8 0.01 1.03 (1,629) O -2.40/2.97* is the 24 800 401 437 7.49 0.85 -0.05 -0.80 421.18 t-lower P 24 402 436 6.86 0.03 0.83 -0.71 1,037 -tests. 419.07 t 7 0.00 0.90 (2,045) O -3.21/3.35* 24 402 438 6.93 0.04 0.83 -0.68 1,010 419.09 P 24 402 436 6.65 0.04 0.82 -0.67 1,038 418.19 6 1.15 Grade -0.01 (2,040) O -3.53/3.24* 24 403 436 6.68 0.05 0.84 -0.68 1,004 418.15 P 24 401 433 6.10 0.28 0.83 -0.52 1,294 416.38 5 0.76 -0.02 (2,591) O -4.57/3.65* 24 401 434 6.29 0.23 0.84 -0.57 1,299 416.27 P 24 401 431 5.62 0.27 0.84 -0.56 1,401 414.37 in means across modes. This test is comprised of two one sided in means across modes. This 4 0.00 1.03 (2,798) O -4.84/4.63* 24 402 431 5.56 0.26 0.84 -0.59 1,399 414.35 no difference P 24 401 429 5.22 0.39 0.84 -0.50 1,513 -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference t 411.78 3 0.01 1.07 (2,952) is the O -4.90/5.45* 24 401 427 5.28 0.42 0.85 -0.49 1,441 -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference 411.84 t † t-upper -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Reliability Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Number of items Sample size Scale score Stat. test Note that statistical significance indicates greater than -1 and greater than 1. If both Note. * Statistically significant at .05. †

19.47 ACT Aspire Mode Comparability Study

Table 19.17. Science Scale Score Summary for Online and Paper Modes P 32 404 449 8.09 0.17 0.87 -0.58 1,023 424.64 0.94 EHS -0.02 (1,980) -3.15/2.33* O 32 959 405 447 8.15 0.17 0.86 -0.69 424.49 P 32 934 403 444 7.87 0.88 -0.02 -0.70 422.80 -statistic for the difference in means -statistic for the difference t 8 0.02 0.86 (1,845) O -2.27/3.18* is the 32 913 403 444 7.90 0.88 -0.06 -0.67 422.97 t-lower P 32 402 438 8.03 0.16 0.89 -0.93 1,042 -tests. 419.74 t 7 0.00 0.55 (2,068) O -2.94/2.72* 32 401 441 8.04 0.17 0.90 -0.89 1,028 419.70 P 32 401 439 7.04 0.10 0.88 -0.55 1,040 419.71 6 0.00 0.81 Grade (2,085) O -3.29/3.15* 32 403 438 7.15 0.07 0.89 -0.65 1,047 419.69 P 28 401 435 6.66 0.87 -0.10 -0.68 1,379 418.83 5 0.02 0.79 (2,755) O -3.36/4.58* 28 402 438 6.55 0.86 -0.10 -0.48 1,378 418.98 P 28 400 435 6.69 0.02 0.85 -0.77 1,444 417.09 in means across modes. This test is comprised of two one sided in means across modes. This 4 1.19 -0.01 (2,877) O -4.32/3.77* 28 401 434 6.59 0.07 0.85 -0.55 1,435 417.02 no difference P 28 401 432 5.93 0.40 0.88 -0.50 1,405 -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference t 413.59 3 0.00 1.16 (2,779) is the O -4.41/4.48* 28 401 431 5.94 0.32 0.89 -0.60 1,376 -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference 413.60 t † t-upper -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Reliability Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Number of items Sample size Scale score Stat. test Note that statistical significance indicates greater than -1 and greater than 1. If both Note. * Statistically significant at .05. †

19.48 ACT Aspire Mode Comparability Study

Table 19.18. Writing Scale Score Summary for Online and Paper Modes 1 P 777 408 448 7.29 0.06 0.13 424.75 0.00 1.22 EHS (1,463) 1 -2.62/2.62* O 708 408 448 7.31 0.02 0.12 424.75 1 P 665 408 448 7.03 -0.14 -0.17 424.37 -statistic for the difference in means -statistic for the difference t 8 0.00 1.57* (1,301) 1 O -2.62/2.53* is the 646 408 446 6.99 -0.20 -0.20 424.36 t-lower 1 P 839 408 448 7.73 0.28 -0.01 -tests. 424.76 t 7 0.00 1.88* (1,659) 1 O -2.65/2.62* 834 408 448 7.74 0.27 0.09 424.76 1 P 888 408 448 7.61 -0.06 -0.32 427.04 6 0.01 Grade 3.76* (1,723) 1 O -2.51/2.84* 847 408 446 7.90 -0.14 -0.32 427.10 1 P 408 440 7.45 0.06 -0.33 1,106 422.88 5 0.00 1.83* (2,201) 1 O -3.17/3.12* 408 440 7.48 0.08 -0.28 1,106 422.87 1 P 408 440 6.87 -0.05 -0.40 1,289 422.47 in means across modes. This test is comprised of two one sided in means across modes. This 4 -0.02 3.77* (2,538) 1 O -4.02/3.18* 408 440 7.12 -0.07 -0.52 1,272 422.35 no difference 1 P 408 440 6.91 0.23 -0.34 1,295 -statistic for the difference in means less than 1. Together, these two statistics test the null hypothesis that the difference in means is less than -1 or these two statistics test the null hypothesis that difference in means less than 1. Together, -statistic for the difference t 422.19 3 0.00 1.70* (2,455) is the 1 O -3.50/3.60* 408 440 7.06 0.21 -0.37 1,236 -tests are statistically significant, then the difference between online and paper means is -1 1. -tests are statistically significant, then the difference 422.20 t † t-upper -upper t O = online form, P paper form. -lower/ Mean SD Minimum Maximum Skewness Kurtosis Effect size Kolmogorov- Smirnov test Test of equivalence for mean difference t (df) Number of items Sample size Scale score Stat. test Note that statistical significance indicates greater than -1 and greater than 1. If both Note. * Statistically significant at .05. †

19.49 ACT Aspire Mode Comparability Study

Compared to Tables 19.8–19.12, which contained raw scores on common items across modes, we see that differences in scale scores are small and generally not statistically significant. For example, if we compare the grade 3 English raw score cumulative distributions in Figure 19.1 to the grade 3 English scale score cumulative distributions in Figure 19.7, we see that the cumulative score differences are small or negligible for the scale scores. While the score differences across modes were eliminated for most of the statistics in Tables 19.14–19.18, there were some cases where the Kolmogorov-Smirnov test indicated statistical differences in the cumulative distributions between modes in mathematics and writing. However, these differences appeared to be small, as illustrated by Figure 19.7, and effect sizes were near zero. The box plots in Figure 19.8 further illustrate scale score comparability across modes. The box plots are not identical across modes for each grade, but they do appear similar in most cases, which is indicative of similar score distributions across modes. Figure 19.7. Cumulative percentage of students for scale scores across modes

English Grade 3 Grade 4

Grade 5 Grade 6

19.50 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Grade 7 Grade 8

Early High School

19.51 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Mathematics Grade 3 Grade 4

Grade 5 Grade 6

19.52 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Grade 7 Grade 8

Early High School

19.53 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Reading Grade 3 Grade 4

Grade 5 Grade 6

19.54 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Grade 7 Grade 8

Early High School

19.55 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Science Grade 3 Grade 4

Grade 5 Grade 6

19.56 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Grade 7 Grade 8

Early High School

19.57 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Writing Grade 3 Grade 4

Grade 5 Grade 6

19.58 ACT Aspire Mode Comparability Study

Figure 19.7. (continued)

Grade 7 Grade 8

Early High School

19.59 ACT Aspire Mode Comparability Study

Figure 19.8. Box plots of scale scores by grade and mode for each subject area

English

Mathematics

19.60 ACT Aspire Mode Comparability Study

Figure 19.8. (continued)

Reading

Science

19.61 ACT Aspire Mode Comparability Study

Figure 19.8. (continued)

Writing

Based in part on these results, we argue that the statistical linking was successful in assuring that scale scores across paper and online modes were comparable. In other words, mode effects across online and paper forms were effectively eliminated; scale scores appeared to function similarly across modes.

19.7 Summary Based on the results comparing raw number-of-points-correct scores on identical items across modes, it is preferable to assume mode effects for ACT Aspire summative assessments. Not every grade and subject showed evidence of mode effects, but it occurred often enough that assuming no mode effects was not a reasonable assumption. When mode effects were observed, they generally appeared relatively small and in some cases (e.g., writing) were inconsistent. If some degree of score differences due to mode were not a strong concern, it may be acceptable or even preferable to ignore mode effects on identical items across modes, but not under current interpretations of ACT Aspire scores.

As described above, some form of statistical linkage was required across modes due to item-type differences across modes. Statistical linkages appeared to successfully adjust for differences in items and mode across paper and online forms under the random groups design. Scale scores appeared comparable across modes.

19.62 CHAPTER 20 ACT Aspire Equating

This chapter contains two sections. The first section presents the general equating plan used for ACT Aspire. The second section presents the equating process and equating results of the 2014 spring operational administration.

20.1 Methods and Procedures Multiple ACT Aspire test forms are developed each year. Despite being constructed to follow the same content and statistical specifications, test forms may differ slightly in difficulty. Equating is used to control for these differences across forms so that scale scores reported to students have the same meaning regardless of the specific form administered. Equating is the process of making statistical adjustments to maintain score interchangeability across test forms (see Holland & Dorans, 2006; Kolen & Brennan, 2014).

ACT Aspire equating typically uses a random equivalent groups design, which involves spiraling the administration of test forms. In this case, test forms are interspersed within a classroom so that forms are distributed equally and randomly equivalent groups of students take each form. Under this design, if groups are indeed randomly equivalent, differences observed in performance across forms can be attributed to differences in form difficulty and equating methods are applied to adjust for these differences.1

Each year, a sample of students from an operational administration of ACT Aspire is used as an equating sample. Students in this sample are administered a spiraled set of ACT Aspire test forms that includes new test forms and an anchor test form that has already been equated to previous forms. Spiraling occurs separately for paper and online test forms, but a large sample of students takes each form.

1 This methodology is also used to evaluate the comparability of forms when items are scrambled and when forms contain different pretest items. 20.1 ACT Aspire Equating

Scores on alternate test forms are equated to the ACT Aspire score scale using equipercentile equating methodology (Kolen & Brennan, 2014). In equipercentile equating, scores on different test forms are considered equivalent if they have the same percentile rank in a given group of students. Equipercentile equating is applied to the raw number-of-points scores for each subject test separately. The equipercentile equating results are subsequently smoothed using an analytic method described by Kolen (1984) to establish a smooth curve, and the equivalents are rounded to integers. The conversion tables that result from this process are used to transform raw scores on the new forms to scale scores.

In special cases where slight changes are made to the anchor form in the current year’s equating study compared to its administration in the previous equating study, the revised anchor form is first equated to its original version using a common-item- nonequivalent group design, and then new forms are equated to the revised anchor using a random equivalent groups design.

Composite scores, including STEM and ELA scores, are not directly equated across forms but are calculated based on the separate subject test scale scores. Other scores, such as reporting category scores, are not equated across forms and are calculated based on raw number of points earned.

20.2 Summary from the Spring 2014 Administration Equating conducted in 2014 spring involved slight changes to some items on the anchor forms administered in spring of 2013, upon which the ACT Aspire scales were constructed. Therefore, the 2014 forms were equated to the 2013 anchor forms through common items using a common-item-nonequivalent-group design. The majority of items remained unchanged from the 2013 to 2014 administration. A small proportion of items were revised or replaced. For example, some constructed- response items were scored using revised rubrics or machine instead of human. Items with any content or scoring change from 2013 to 2014 were excluded from the common item set.

Once common items were identified after evaluating content and scoring modifications, statistical analyses were conducted to examine that the statistical characteristics of common items were similar between the 2013 and 2014 samples. Results included delta plots (Angoff & Ford, 1973) and analysis of differences in mean proportion-of-points earned2 on common items. Items that fell outside of the two standard error band of delta plots or had absolute mean proportion-of-points earned difference larger than 0.15 were flagged as outliers for further evaluation. The final common item sets had full coverage with respect to both content and statistical specifications, such as mean proportion-of-points earned and point-biserial

2 Note that for multiple-choice items, where possible scores include 0 or 1, this statistic is often referred to as the p-value. But, since most ACT Aspire forms include items with maximum possible scores greater than 1, this statistic is referred to as mean proportion of points earned. 20.2 ACT Aspire Equating

distributions on the intact forms, and had sufficient number of items from each reporting category. The proportions of common items ranged from 52% to 97% across forms.

Classical linear equating methods (e.g., Tucker method, Levine observed score method) or frequency estimation equipercentile equating methods were used to conduct equating, after evaluating the characteristics of group and form differences between 2013 and 2014 administrations. Equating was conducted separately for the online and paper forms using 2014 operational data. The paper forms were linked to the online forms in the 2013 comparability study. Pretest items included in test forms were not scored and were not used in the equating process. There were scrambled versions of each test form in 2014. Operational items in the scrambled forms were identical to the 2014 anchor forms but were presented in slightly different orders. Examinees’ performance on the scrambled forms was examined and evidence indicated that scrambled forms functioned similarly to the anchor forms.

20.2.1 Results Tables 20.1 to 20.8 present the demographic information for the samples used in the equating study. All demographic information was provided by schools. The non- response rates were higher for online forms than for paper forms. More examinees took mathematics and reading than English and science. Genders were evenly distributed. Depending on specific grade and mode, distributions of ethnicity were slightly different. Approximately half of the sample was white and a quarter was African American.

To evaluate the equating results, plots of students’ average scale scores conditional on common item raw scores were reviewed. Students taking the 2013 form and those taking the 2014 form with the same common item raw score should have similar scale scores after equating. Figures 20.1-20.4 present plots for each subject and certain grade. The red circle points represent the group of examinees taking the 2014 forms and the blue cross points represent the group of examinees taking the 2013 forms. These plots indicate that for most score points, students with the same common item raw score have the same scale scores regardless of which form they take. Scale score differences tend to be larger at both ends than in the middle range of the score points.

20.2.2 Standard Error of Equating Standard error of equating (SEE) is an index capturing random error in estimating equating relationships using scores of examinees from samples instead of from a population. According to Kolen and Brennan (2014), SEE is “conceived of as the standard deviation of equated scores over hypothetical replications of an equating procedure in samples from a population or populations of examinees.” To compute SEE for ACT Aspire forms, the bootstrap method was adopted. Two hundred bootstrap samples of the same size were drawn with replacement from the original 20.3 ACT Aspire Equating

sample. Based on each bootstrap sample, a new conversion table was generated. SEE was computed as the standard deviation of the unrounded scale scores corresponding to each raw score point on the new form across all replications.

Figure 20.5 presents the plots of SEE against raw score point for selected subjects and grades. The magnitudes of the SEEs are affected by the equating sample size— the larger the sample size, the smaller the SEE. The shape of the SEE curves in Figure 20.5 is related to the equating method that was selected. Generally speaking, when a classical linear method is selected, the SEE curve tends to be smooth and follows a U shape. When the equipercentile method is selected, SEEs are low and the curve is flat in the middle section of the raw score scale; however, SEEs tend to get larger as raw score points move from the middle to either end; and around the two end points, SEEs become small again due to a linear interpolation in the equating process.

20.4 ACT Aspire Equating

Table 20.1. Percentages of Students Taking English Online Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (3,342) (3,243) (3,346) (3,743) (5,463) (8,979) (18,116) Gender Not Specified 13 13 11 13 15 12 16 Female 43 43 44 42 41 45 42 Male 44 44 44 44 43 43 42 Race/Ethnicity Not Specified 37 37 41 51 51 46 55 Black/African American 24 23 21 14 9 6 2 American Indian/Alaskan 1 1 0 1 0 1 1 Native White 29 31 29 27 32 42 36 Hispanic/Latino 6 6 6 5 4 3 3 Asian 1 1 1 1 1 1 1 Native Hawaiian/Other 0 0 0 0 0 0 0 Pacific Islander Two or More Races 1 1 0 0 0 0 0 Prefer Not to Respond 0 0 0 0 1 1 1 Geographic Region No Response 0 0 0 0 0 0 1 East 56 56 57 46 34 18 7 Midwest 15 12 13 26 45 66 79 Southwest 6 8 6 9 6 5 7 West 23 24 24 20 16 10 6

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.5 ACT Aspire Equating

Table 20.2. Percentages of Students Taking English Paper Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (5,049) (5,215) (5,038) (5,658) (5,692) (6,027) (4,247) Gender Not Specified 1 1 1 2 2 2 16 Female 48 47 49 49 49 49 46 Male 51 52 50 49 50 49 38 Race/Ethnicity Not Specified 8 9 9 11 11 12 63 Black/African American 24 25 24 25 25 25 2 American Indian/Alaskan 1 1 2 1 1 1 0 Native White 60 58 60 57 58 56 28 Hispanic/Latino 4 4 4 4 4 4 5 Asian 2 2 2 1 2 2 2 Native Hawaiian/Other 0 0 0 0 0 0 0 Pacific Islander Two or More Races 0 0 0 0 0 0 0 Prefer Not to Respond Geographic Region No Response 0 0 0 0 0 1 0 East 95 96 96 93 92 90 26 Midwest 0 0 0 2 4 4 29 Southwest 5 4 4 4 3 5 18 West 0 0 0 0 0 1 28

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.6 ACT Aspire Equating

Table 20.3. Percentages of Students Taking Mathematics Online Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (4,415) (4,511) (5,512) (5,364) (6,735) (10,614) (16,694) Gender Not Specified 9 8 6 7 10 8 16 Female 47 46 47 45 44 47 42 Male 44 46 46 48 46 45 41 Race/Ethnicity Not Specified 25 24 27 31 35 36 56 Black/African American 23 23 22 20 15 11 2 American Indian/Alaskan 0 0 0 0 0 1 1 Native White 42 44 42 42 43 46 36 Hispanic/Latino 7 6 6 5 5 4 3 Asian 1 2 2 2 1 1 1 Native Hawaiian/Other 0 0 0 0 0 0 0 Pacific Islander Two or More Races 0 0 0 0 0 0 0 Prefer Not to Respond 0 0 0 0 1 1 1 Geographic Region No Response 0 0 0 0 0 0 1 East 72 75 66 68 53 34 7 Midwest 10 8 10 16 31 53 79 Southwest 5 5 12 5 6 5 8 West 14 12 13 11 10 8 5

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.7 ACT Aspire Equating

Table 20.4. Percentages of Students Taking Mathematics Paper Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (19,375) (18,635) (17668) (19,106) (20,340) (19,923) (4,121) Gender Not Specified 0 0 0 1 0 1 15 Female 49 48 49 49 49 49 47 Male 50 52 51 51 51 50 38 Race/Ethnicity Not Specified 5 4 4 5 4 5 62 Black/African American 30 31 31 33 33 33 3 American Indian/Alaskan 1 1 1 1 1 1 0 Native White 56 56 57 55 55 55 29 Hispanic/Latino 6 6 6 5 5 5 5 Asian 1 1 1 1 1 1 2 Native Hawaiian/Other 0 0 0 0 0 0 0 Pacific Islander Two or More Races 0 0 0 0 0 0 0 Prefer Not to Respond Geographic Region No Response 0 0 0 0 0 0 0 East 99 99 99 98 98 97 26 Midwest 0 0 0 1 1 1 29 Southwest 1 1 1 1 1 2 18 West 0 0 0 0 0 0 26

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.8 ACT Aspire Equating

Table 20.5. Percentages of Students Taking Reading Online Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (5,343) (5,579) (3,523) (6,002) (7,697) (11,608) (17,486) Gender Not Specified 8 9 5 8 10 9 17 Female 45 44 46 45 43 46 42 Male 46 47 49 47 47 45 42 Race/Ethnicity Not Specified 24 23 22 31 34 35 57 Black/African American 24 23 24 20 16 12 2 American Indian/Alaskan 1 1 0 0 0 1 1 Native White 42 44 46 42 43 46 35 Hispanic/Latino 7 7 5 6 5 4 3 Asian 1 2 1 1 1 1 1 Native Hawaiian/Other 0 0 0 0 0 0 0 Pacific Islander Two or More Races 0 0 0 0 0 0 0 Prefer Not to Respond 0 0 0 0 1 1 1 Geographic Region No Response 0 0 0 0 0 0 1 East 73 75 75 69 56 37 7 Midwest 10 9 10 15 29 51 79 Southwest 4 5 1 5 4 4 7 West 13 11 14 11 11 9 6

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.9 ACT Aspire Equating

Table 20.6. Percentages of Students Taking Reading Paper Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (10,462) (13,965) (6,258) (11,263) (10,641) (12,276) (4,101) Gender Not Specified 1 0 1 1 1 1 16 Female 48 48 48 48 48 49 47 Male 51 51 51 51 51 50 38 Race/Ethnicity Not Specified 6 5 7 6 6 6 62 Black/African American 26 28 26 29 31 30 2 American Indian/Alaskan 1 1 1 1 1 1 0 Native White 59 58 59 57 56 57 29 Hispanic/Latino 7 6 6 5 5 5 5 Asian 1 1 2 1 1 1 2 Native Hawaiian/Other 0 0 0 0 0 0 0 Pacific Islander Two or More Races 0 0 0 0 0 0 0 Prefer Not to Respond Geographic Region No Response 0 0 0 0 0 0 0 East 97 98 97 97 96 95 27 Midwest 0 0 0 1 2 2 28 Southwest 3 2 3 2 1 2 18 West 0 0 0 0 0 0 27

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.10 ACT Aspire Equating

Table 20.7. Percentages of Students Taking Science Online Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (2,158) (2,305) (2,440) (2,593) (4,048) (7,654) (16,460) Gender Not Specified 11 12 13 14 17 12 17 Female 45 44 43 41 41 45 42 Male 44 44 44 44 43 43 42 Race/Ethnicity Not Specified 37 35 49 56 57 47 57 Black/African American 24 23 19 11 6 5 2 American Indian/Alaskan 1 0 0 0 0 1 1 Native White 29 33 25 26 30 42 36 Hispanic/Latino 8 7 5 5 4 3 3 Asian 1 1 1 1 1 1 1 Native Hawaiian/Other 1 0 1 0 0 0 0 Pacific Islander Two or More Races 1 0 0 0 1 0 0 Prefer Not to Respond 0 0 0 0 2 1 1 Geographic Region No Response 0 0 0 0 0 0 1 East 53 60 50 39 24 15 7 Midwest 14 11 22 29 53 68 79 Southwest 9 10 7 10 7 7 8 West 23 19 21 22 16 11 5

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.11 ACT Aspire Equating

Table 20.8. Percentages of Students Taking Science Paper Forms in the Spring 2014 Equating Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 EHS Demographic Grouping (4,208) (4,371) (4,012) (4,963) (5,217) (5,394) (4,159) Gender Not Specified 1 1 1 2 2 2 15 Female 49 47 48 47 48 49 47 Male 50 53 51 50 50 48 38 Race/Ethnicity Not Specified 10 10 10 12 11 13 63 Black/African American 27 26 32 29 35 27 2 American Indian/Alaskan 1 1 0 1 0 1 0 Native White 56 56 52 53 49 54 28 Hispanic/Latino 5 5 5 4 5 4 5 Asian 1 1 1 1 1 1 2 Native Hawaiian/Other 0 0 0 0 0 0 0 Pacific Islander Two or More Races 0 0 0 0 0 0 0 Prefer Not to Respond Geographic Region No Response 0 0 0 0 0 1 0 East 94 95 95 92 92 88 26 Midwest 0 0 0 3 4 4 30 Southwest 6 5 5 5 3 6 18 West 0 0 0 0 0 1 26

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

20.12 ACT Aspire Equating

Figure 20.1. Average English scale scores against common item raw scores

English grade 4 online

English grade 6 paper

20.13 ACT Aspire Equating

Figure 20.2. Average mathematics scale scores against common item raw scores

Mathematics grade 5 online

Mathematics grade 7 paper

20.14 ACT Aspire Equating

Figure 20.3. Average reading scale score against common item raw scores

Reading grade 4 online

Reading grade 8 paper

20.15 ACT Aspire Equating

Figure 20.4. Average science scale scores against common item raw scores

Science grade 6 online

Science grade 8 paper

20.16 ACT Aspire Equating

Figure 20.5. Unrounded scale score standard error of equating for selected subjects and grades English, Grade 3, Online Form English, Grade EHS, Paper Form 8 8 . . 0 0 6 6 . . 0 0 SE E SE E

e e r r c o c o S S

4 4 e . . e l l 0 0 c a c a S S ed ed 2 2 oun d . . oun d r r 0 0 n n U U 0 0 . . 0 0

0 5 10 15 20 25 0 10 20 30 40 50 New Form Raw Score New Form Raw Score

Mathematics, Grade 7, Online Form Mathematics, Grade EHS, Paper Form 8 8 . . 0 0 6 6 . . 0 0 SE E SE E

e e r r c o c o S S

e e l l 4 4 . . 0 0 c a c a S S ed ed oun d oun d r r 2 2 . . n n 0 0 U U 0 0 . . 0 0

0 10 20 30 40 0 10 20 30 40 50 New Form Raw Score New Form Raw Score

20.17 ACT Aspire Equating

Figure 20.5. (Continued) Reading, Grade 8, Online Form Reading, Grade EHS, Paper Form 8 8 . . 0 0 6 6 . . 0 0 SE E SE E

e e r r c o c o S S

e e l l 4 4 . . 0 0 c a c a S S ed ed oun d oun d r r 2 2 . . n n 0 0 U U 0 0 . . 0 0

0 5 10 15 20 25 30 0 5 10 15 20 25 30 New Form Raw Score New Form Raw Score

Science, Grade 3, Online Form Science, Grade 5, Paper Form 8 8 . . 0 0 6 6 . . 0 0 SE E SE E

e e r r c o c o S S

e e l l 4 4 . . 0 0 c a c a S S ed ed oun d oun d r r 2 2 . . n n 0 0 U U 0 0 . . 0 0

0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35 New Form Raw Score New Form Raw Score

20.18 Appendices

Appendices 41 Appendix A: Illustration of the ACT Aspire Mathematics

Bulletin Assessments ACT Aspire Technical ACTTechnical Aspire

Figure A.1. Mathematics test content specifications and flow A.1

Appendices

Appendix B: The ACT with Constructed-Response Items While ACT Aspire covers grades 3 through 10, the scaling study also included grade 11. The primary reason for including grade 11 was so that the ACT Aspire vertical scale could be extended to a version of the ACT and provide scale continuity from grades 3 through 11. The implications of this are twofold. First, the scaling test (ST4) included ACT items so that the vertical scale could include grade 11 and be extended to a version of the ACT. Second, a grade 11 on-grade test was included in the scaling study.

The on-grade test covering grade 11 included in the scaling study had two parts: an operationally administered form of the ACT and a separately administered set of constructed-response-only items. These two components were combined and scaled to obtain scores on the ACT Aspire scale. This grade 11 test is referred to as the ACT with constructed-response tests. In some places, such as tables and figures with limited space, this test may be referred to as “ACT” or “the ACT.”

The constructed-response component was included for mathematics, reading and science Tests. Table B.1 lists the number of constructed-response items included in each component. The ACT writing test was not included in the scaling study.

By including the ACT with constructed-response tests in the scaling study, the ACT Aspire vertical scale was extended to grade 11 and included the ACT combined with constructed-response items, consistent with tests for grades 3–10 that also included constructed-response items at these grades.

Table B.1. Number of Constructed-Response Items and Number of Score Points Included in the On-Grade ACT with Constructed-Response Test Forms in the Scaling Study

Number of Constructed Raw Number-of-Points Subject Response Items Score Range

Mathematics 6 0–24 Reading 3 0–10 Science 11 0–30

B.1

Appendices

Appendix C: Spring 2013 Scaling Study Demographic Summary Tables Table C.1. Percentages of Students Taking English Tests in the Scaling Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 9 10 11 Demographic Grouping (4,277) (3,825) (4,301) (4,683) (4,762) (3,609) (2,415) (2,258) (2,240) Gender Not Specified 5 6 5 4 4 5 2 1 89 Female 48 47 48 49 49 49 52 54 6 Male 47 47 46 47 47 47 47 45 5 Race/Ethnicity Not Specified 33 34 31 26 30 30 39 35 93 Black/African American 27 27 29 27 24 22 29 28 3 American Indian/Alaskan 0 0 0 0 0 0 0 0 0 Native White 29 28 29 26 24 28 17 22 1 Hispanic/Latino 8 8 8 18 17 16 12 11 2 Asian 2 1 2 1 2 2 2 3 1 Native Hawaiian/Other 1 1 0 0 0 0 0 0 Pacific Islander Two or More Races 1 0 1 1 2 1 0 2 Geographic Region East 35 40 36 31 30 40 25 24 24 Midwest 40 40 39 44 45 39 50 50 41 Southwest 19 13 18 10 13 12 14 20 27 West 6 7 7 16 13 8 11 6 8

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

C.1 Appendices

Table C.2. Percentages of Students Taking Mathematics Tests in the Scaling Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 9 10 11 Demographic Grouping (4,300) (3,830) (4,260) (4,577) (4,497) (3,746) (2,479) (2,528) (1,796) Gender Not Specified 5 6 5 3 5 5 1 1 92 Female 48 46 49 49 48 48 52 53 4 Male 47 47 46 48 47 47 47 46 4 Race/Ethnicity Not Specified 33 34 31 26 31 29 38 38 94 Black/African American 27 28 29 27 23 24 29 25 3 American Indian/Alaskan 0 0 0 0 0 0 0 0 Native White 29 28 29 27 23 28 17 20 1 Hispanic/Latino 8 8 8 17 17 16 12 12 1 Asian 2 1 2 2 2 2 3 3 1 Native Hawaiian/Other 1 1 0 0 0 0 0 0 Pacific Islander Two or More Races 1 0 1 1 3 1 0 2 Geographic Region East 34 40 36 30 31 39 25 21 27 Midwest 41 40 39 46 44 42 48 53 40 Southwest 18 13 19 10 14 12 13 18 27 West 7 7 7 13 11 7 14 8 6

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

C.2 Appendices

Table C.3. Percentages of Students Taking Reading Tests in the Scaling Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 9 10 11 Demographic Grouping (4,307) (3,661) (4,129) (4,520) (4,475) (3,585) (2,257) (2,260) (1,789) Gender Not Specified 4 6 6 4 4 5 1 0 90 Female 48 46 48 49 49 48 52 54 5 Male 48 48 46 47 47 47 46 46 4 Race/Ethnicity Not Specified 32 35 32 27 31 32 41 40 95 Black/African American 28 27 27 26 23 23 28 24 3 American Indian/Alaskan 0 0 0 0 0 0 0 0 0 Native White 28 28 30 27 24 27 17 21 1 Hispanic/Latino 9 8 8 17 16 16 10 10 1 Asian 2 1 2 2 2 2 2 3 1 Native Hawaiian/Other 1 1 0 0 0 0 0 0 Pacific Islander Two or More Races 1 1 1 1 3 0 0 2 Geographic Region East 36 41 37 32 30 40 26 23 26 Midwest 40 39 39 44 45 40 51 51 40 Southwest 19 14 18 10 14 13 14 19 29 West 6 6 6 14 11 8 9 7 5

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

C.3 Appendices

Table C.4. Percentages of Students Taking Science Tests in the Scaling Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 9 10 11 Demographic Grouping (4,214) (3,571) (3,903) (4,642) (4,756) (3,544) (2,167) (2,314) (1,717) Gender Not Specified 5 7 5 4 4 5 2 1 93 Female 48 46 48 49 49 49 51 54 4 Male 47 47 47 47 47 46 47 46 4 Race/Ethnicity Not Specified 33 37 33 26 29 29 43 41 96 Black/African American 26 25 26 25 24 22 25 20 3 American Indian/Alaskan 0 0 0 0 0 0 0 0 Native White 29 28 30 28 24 28 19 22 0 Hispanic/Latino 9 8 8 19 17 17 10 11 1 Asian 2 1 2 1 2 2 2 3 1 Native Hawaiian/Other 1 1 0 0 0 0 0 0 Pacific Islander Two or More Races 1 1 1 1 3 1 1 2 Geographic Region East 37 40 37 31 28 39 28 24 28 Midwest 38 39 36 44 46 40 50 51 39 Southwest 19 14 20 10 13 13 14 19 29 West 7 7 8 16 13 8 9 6 4

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

C.4 Appendices

Table C.5. Percentages of Students Taking Writing Tests in the Scaling Study by Grade and Demographics

Grade (N) 3 4 5 6 7 8 9 10 Demographic Grouping (4,109) (3,439) (3,935) (4,512) (4,666) (3,522) (2,117) (2,258) Gender Not Specified 4 5 4 2 3 3 1 1 Female 49 48 49 50 49 50 53 55 Male 46 47 47 48 47 47 47 45 Race/Ethnicity Not Specified 33 36 32 25 29 29 43 36 Black/African American 25 24 25 25 23 22 27 24 American Indian/Alaskan 0 0 0 0 0 0 0 0 Native White 30 29 30 28 25 28 16 22 Hispanic/Latino 9 8 9 19 18 17 11 14 Asian 2 1 2 2 2 2 2 3 Native Hawaiian/Other 1 1 0 0 0 0 0 0 Pacific Islander Two or More Races 1 1 1 1 3 1 1 2 Geographic Region East 37 43 34 31 29 39 28 21 Midwest 37 35 37 42 44 41 50 52 Southwest 19 15 20 10 13 12 15 19 West 7 8 9 17 14 8 7 8

Note. Zero indicates fewer than 0.5%. Missing values indicate no students. Percentages within a demographic grouping may not add up to 100% due to rounding.

C.5

Appendices

Appendix D: Descriptive Statistics and Reliabilities of Reporting Category Raw Scores Table D.1. English Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration

Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM COE 3315 15 8.98 3.32 0.74 1.68 Online POW 3315 10 4.33 2.08 0.52 1.44 3 COE 5139 15 9.45 3.44 0.78 1.63 Paper POW 5139 10 4.79 2.18 0.57 1.43 COE 3217 15 9.05 2.86 0.66 1.66 Online KLA 3217 3 1.45 0.96 0.32 0.79 POW 3217 7 5.01 1.53 0.53 1.05 4 COE 5356 15 9.66 2.91 0.70 1.60 Paper KLA 5356 3 1.64 0.94 0.31 0.78 POW 5356 7 5.14 1.50 0.53 1.03 COE 3303 15 8.37 2.92 0.67 1.68 Online KLA 3303 3 1.88 0.79 0.10 0.75 POW 3303 7 4.07 1.64 0.55 1.10 5 COE 5380 15 8.95 2.88 0.68 1.62 Paper KLA 5380 3 1.98 0.81 0.15 0.75 POW 5380 7 4.35 1.62 0.56 1.08 COE 3706 20 11.44 3.56 0.72 1.89 Online KLA 3706 3 1.58 0.93 0.26 0.80 POW 3706 12 5.90 2.59 0.65 1.53 6 COE 6055 20 11.69 3.56 0.71 1.90 Paper KLA 6055 3 1.71 0.93 0.25 0.80 POW 6055 12 5.76 2.46 0.62 1.53 COE 5413 20 12.29 3.82 0.75 1.93 Online KLA 5413 5 2.74 1.21 0.31 1.01 POW 5413 10 5.22 2.01 0.48 1.44 7 COE 5892 20 12.75 3.84 0.75 1.91 Paper KLA 5892 5 2.94 1.23 0.35 0.99 POW 5892 10 5.20 2.03 0.50 1.43

D.1 Appendices

Table D.1. (continued) Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM COE 8921 20 12.14 3.44 0.72 1.81 Online KLA 8921 5 3.68 1.29 0.53 0.88 POW 8921 10 6.09 2.29 0.66 1.33 8 COE 6213 20 12.35 3.61 0.75 1.79 Paper KLA 6213 5 3.87 1.22 0.53 0.83 POW 6213 10 6.15 2.25 0.65 1.33 COE 18046 31 17.22 5.81 0.83 2.42 EHSb Online KLA 18046 5 2.74 1.36 0.53 0.93 POW 18046 14 7.99 2.99 0.71 1.60 COE 4335 31 18.19 6.03 0.84 2.38 EHSb Paper KLA 4335 5 2.95 1.43 0.57 0.93 POW 4335 14 8.09 3.12 0.75 1.57

a Reporting category: POW=Production of Writing, KLA=Knowledge of Language, and COE=Conventions of Standard English b Early high school form taken by grades 9 and 10 students.

D.2 Appendices

Table D.2. Mathematics Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration

Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

FOUNDATION 5467 14 5.74 2.53 0.61 1.57 GLP 5467 23 9.25 3.65 0.68 2.08 GLP_G 5467 3 1.35 0.70 0.17 0.64 GLP_MD 5467 3 1.11 0.82 0.06 0.79 Online GLP_NBT 5467 4 1.31 1.05 0.35 0.85 GLP_NF 5467 3 1.07 0.91 0.28 0.77 GLP_OA 5467 2 1.19 0.70 0.28 0.60 JE 5467 16 6.26 2.57 0.63 1.56 MODELING 5467 24 9.53 3.77 0.69 2.10 3 FOUNDATION 20114 14 6.32 2.54 0.63 1.55 GLP 20114 23 9.67 3.58 0.65 2.14 GLP_G 20114 3 1.49 0.75 0.12 0.70 GLP_MD 20114 3 1.12 0.81 0.05 0.79 Paper GLP_NBT 20114 4 1.42 1.09 0.33 0.90 GLP_NF 20114 3 0.98 0.89 0.29 0.75 GLP_OA 20114 2 1.30 0.70 0.27 0.60 JE 20114 16 6.84 2.57 0.64 1.55 MODELING 20114 24 9.90 3.71 0.68 2.09 FOUNDATION 5740 14 4.37 1.79 0.38 1.40 GLP 5740 23 6.93 2.88 0.59 1.85 GLP_G 5740 3 0.77 0.84 0.34 0.68 GLP_MD 5740 3 0.88 0.87 0.28 0.74 Online GLP_NBT 5740 3 0.86 0.77 0.12 0.72 GLP_NF 5740 3 0.99 0.85 0.25 0.74 GLP_OA 5740 3 0.93 0.79 0.08 0.76 JE 5740 16 5.00 1.71 0.52 1.18 MODELING 5740 20 5.72 2.31 0.47 1.69 4 FOUNDATION 19366 14 4.45 1.74 0.37 1.38 GLP 19366 23 6.86 2.95 0.59 1.90 GLP_G 19366 3 0.85 0.84 0.23 0.74 GLP_MD 19366 3 0.86 0.85 0.23 0.74 Paper GLP_NBT 19366 3 0.83 0.78 0.15 0.72 GLP_NF 19366 3 1.02 0.87 0.28 0.74 GLP_OA 19366 3 0.92 0.79 0.07 0.76 JE 19366 16 4.95 1.67 0.47 1.22 MODELING 19366 20 5.62 2.37 0.47 1.73

D.3 Appendices

Table D.2. (continued) Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

FOUNDATION 6536 14 4.64 1.99 0.47 1.46 GLP 6536 23 7.26 3.03 0.61 1.90 GLP_G 6536 3 0.81 0.79 0.13 0.74 GLP_MD 6536 3 0.91 0.74 0.07 0.71 Online GLP_NBT 6536 3 1.00 0.87 0.23 0.76 GLP_NF 6536 3 1.28 0.92 0.19 0.82 GLP_OA 6536 3 1.04 0.75 0.25 0.65 JE 6536 16 4.61 1.81 0.49 1.30 MODELING 6536 24 7.39 2.97 0.60 1.87 5 FOUNDATION 19064 14 4.29 1.84 0.43 1.39 GLP 19064 23 7.44 3.07 0.59 1.96 GLP_G 19064 3 0.98 0.79 0.08 0.76 GLP_MD 19064 3 0.87 0.75 0.07 0.73 Paper GLP_NBT 19064 3 0.93 0.85 0.20 0.76 GLP_NF 19064 3 1.12 0.90 0.19 0.81 GLP_OA 19064 3 1.36 0.88 0.22 0.77 JE 19064 16 4.50 1.77 0.49 1.26 MODELING 19064 24 7.57 2.87 0.56 1.91 FOUNDATION 6288 18 5.40 2.57 0.61 1.60 GLP 6288 28 10.01 3.84 0.68 2.18 GLP_EE 6288 4 1.09 0.98 0.27 0.84 GLP_G 6288 4 1.50 1.03 0.22 0.91 Online GLP_NS 6288 4 1.75 0.97 0.26 0.84 GLP_RP 6288 4 1.80 1.00 0.27 0.85 GLP_S 6288 4 1.50 0.93 0.28 0.79 JE 6288 16 4.22 1.78 0.48 1.29 MODELING 6288 26 9.47 3.82 0.69 2.13 6 FOUNDATION 20077 18 4.91 2.48 0.59 1.60 GLP 20077 28 10.17 3.85 0.67 2.22 GLP_EE 20077 4 1.09 0.95 0.17 0.86 GLP_G 20077 4 1.36 0.99 0.20 0.89 Paper GLP_NS 20077 4 1.83 1.04 0.28 0.89 GLP_RP 20077 4 1.73 0.97 0.25 0.84 GLP_S 20077 4 1.66 0.97 0.27 0.83 JE 20077 16 4.13 1.86 0.49 1.33 MODELING 20077 26 9.08 3.70 0.68 2.10

D.4 Appendices

Table D.2. (continued) Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

FOUNDATION 8195 18 6.41 2.69 0.60 1.70 GLP 8195 28 10.36 4.47 0.78 2.09 GLP_EE 8195 4 1.14 1.05 0.38 0.83 GLP_G 8195 4 1.64 1.13 0.39 0.89 Online GLP_NS 8195 4 1.64 1.00 0.38 0.79 GLP_RP 8195 4 1.42 1.11 0.38 0.88 GLP_S 8195 4 1.93 1.04 0.44 0.78 JE 8195 16 5.46 1.98 0.57 1.29 MODELING 8195 30 10.73 4.30 0.75 2.14 7 FOUNDATION 20965 18 5.97 2.57 0.57 1.68 GLP 20965 28 9.73 4.13 0.73 2.15 GLP_EE 20965 4 1.13 1.01 0.31 0.83 GLP_G 20965 4 1.39 1.07 0.33 0.88 Paper GLP_NS 20965 4 1.48 1.00 0.29 0.84 GLP_RP 20965 4 1.39 1.09 0.36 0.88 GLP_S 20965 4 1.80 1.07 0.30 0.89 JE 20965 16 5.20 1.84 0.52 1.27 MODELING 20965 30 9.95 3.96 0.71 2.13 FOUNDATION 12154 20 7.20 2.85 0.65 1.69 GLP 12154 33 14.01 5.66 0.83 2.36 GLP_EE 12154 6 2.95 1.53 0.56 1.01 GLP_F 12154 4 2.26 1.19 0.44 0.89 Online GLP_G 12154 6 2.91 1.43 0.44 1.07 GLP_NS 12154 2 1.00 0.79 0.41 0.61 GLP_S 12154 3 1.69 1.00 0.47 0.73 JE 12154 20 5.47 2.41 0.63 1.47 MODELING 12154 40 16.04 5.73 0.82 2.45 8 FOUNDATION 20856 20 7.42 2.91 0.65 1.72 GLP 20856 33 13.14 5.64 0.82 2.42 GLP_EE 20856 6 2.64 1.45 0.51 1.02 GLP_F 20856 4 1.97 1.18 0.42 0.90 Paper GLP_G 20856 6 2.71 1.43 0.46 1.05 GLP_NS 20856 2 0.92 0.79 0.40 0.61 GLP_S 20856 3 1.49 1.01 0.45 0.75 JE 20856 20 5.79 2.48 0.61 1.55 MODELING 20856 40 15.21 5.51 0.80 2.47

D.5 Appendices

Table D.2. (continued) Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

FOUNDATION 17767 20 8.63 3.78 0.75 1.87 GLP 17767 33 12.21 5.08 0.79 2.32 GLP_A 17767 5 2.13 1.22 0.41 0.94 GLP_F 17767 4 1.32 1.09 0.41 0.84 Online GLP_G 17767 5 2.24 1.16 0.30 0.97 GLP_N 17767 3 1.34 0.96 0.36 0.77 GLP_S 17767 4 1.84 1.14 0.38 0.90 JE 17767 20 5.45 2.73 0.70 1.49 MODELING 17767 33 12.36 4.97 0.79 2.29 EHSb FOUNDATION 4320 20 8.87 4.02 0.77 1.93 GLP 4320 33 12.62 5.42 0.81 2.38 GLP_A 4320 5 2.31 1.32 0.44 0.98 GLP_F 4320 4 1.42 1.14 0.45 0.84 Paper GLP_G 4320 5 2.21 1.17 0.32 0.96 GLP_N 4320 3 1.42 0.98 0.39 0.76 GLP_S 4320 4 1.93 1.17 0.40 0.91 JE 4320 20 5.63 3.00 0.72 1.58 MODELING 4320 33 12.57 5.41 0.81 2.35

a Reporting category: GLP=Grade Level Progress, JE=Justification and Explanation, GLP_NF=Number and Operations—Fractions, GLP_NBT=Number and Operations in Base 10, GLP_NS=The Number System, GLP_ N=Number and Quantify, GLP_OA=Operations and Algebraic Thinking, GLP_EE=Expressions and Equations, GLP_RP=Ratios and Proportional Relationships, GLP_A=Algebra, GLP_F=Functions, GLP_G=Geometry, GLP_ MD=Measurement and Data, and GLP_S=Statistics and Probability b Early high school form taken by grades 9 and 10 students.

D.6 Appendices

Table D.3. Reading Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration

Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

CAS 6027 8 4.25 1.97 0.58 1.28 Online IOK 6027 5 1.77 1.38 0.41 1.06 KID 6027 16 8.03 3.69 0.77 1.76 3 CAS 20058 8 4.65 1.90 0.55 1.28 Paper IOK 20058 5 1.92 1.41 0.40 1.09 KID 20058 16 9.05 3.52 0.74 1.80 CAS 6281 9 5.26 2.32 0.62 1.42 Online IOK 6281 5 1.46 1.28 0.48 0.92 KID 6281 15 8.31 3.18 0.71 1.70 4 CAS 19344 9 5.58 2.19 0.59 1.41 Paper IOK 19344 5 1.63 1.23 0.39 0.97 KID 19344 15 8.78 3.11 0.69 1.72 CAS 6345 8 5.45 1.88 0.55 1.26 Online IOK 6345 5 2.24 1.39 0.41 1.07 KID 6345 16 9.26 3.61 0.75 1.79 5 CAS 19405 8 5.41 1.86 0.55 1.24 Paper IOK 19405 5 2.23 1.34 0.39 1.04 KID 19405 16 9.43 3.45 0.73 1.80 CAS 6622 8 4.43 1.94 0.56 1.29 Online IOK 6622 5 2.37 1.58 0.40 1.22 KID 6622 16 10.13 3.51 0.75 1.77 6 CAS 20070 8 4.65 1.90 0.54 1.29 Paper IOK 20070 5 2.39 1.61 0.41 1.24 KID 20070 16 10.42 3.41 0.73 1.76 CAS 8396 9 4.83 1.91 0.50 1.35 Online IOK 8396 5 2.07 1.48 0.38 1.17 KID 8396 15 7.69 3.20 0.70 1.76 7 CAS 21043 9 4.90 1.92 0.49 1.38 Paper IOK 21043 5 2.38 1.48 0.37 1.18 KID 21043 15 7.66 3.18 0.69 1.76

D.7 Appendices

Table D.3. (continued)

Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

CAS 12567 6 4.01 1.54 0.53 1.06 Online IOK 12567 4 1.49 1.25 — c — c KID 12567 21 12.21 4.21 0.77 2.03 8 CAS 20915 6 4.02 1.58 0.52 1.10 Paper IOK 20915 4 1.72 1.24 — c — c KID 20915 21 12.15 4.08 0.75 2.04 Online CAS 18132 7 4.01 1.80 0.57 1.18 Online IOK 18132 5 1.75 1.09 0.28 0.92 Online KID 18132 19 10.62 4.45 0.82 1.89 EHSb Paper CAS 4326 7 4.28 1.82 0.60 1.16 Paper IOK 4326 5 1.91 1.11 0.21 0.98 Paper KID 4326 19 11.48 4.36 0.82 1.87 a Reporting category: KID=Key Ideas and Details, CAS=Craft and Structure, and IOK=Integration of Knowledge and Ideas b Early high school form taken by grades 9 and 10 students. c Only one constructed response item is included in this reporting category, thus reliability and SEM cannot be computed.

D.8 Appendices

Table D.4. Science Reporting Category Raw Score Descriptive Statistics and Reliabilities from Spring 2014 Test Administration

Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

EMI 2715 12 4.08 2.77 0.65 1.64

Online IOD 2715 14 6.95 3.23 0.76 1.58

SIN 2715 10 3.91 2.31 0.64 1.38 3 EMI 4286 12 5.18 2.96 0.68 1.67

Paper IOD 4286 14 7.41 3.44 0.80 1.55

SIN 4286 10 5.01 2.53 0.65 1.50

EMI 2915 11 4.10 2.18 0.56 1.44

Online IOD 2915 15 9.13 3.14 0.75 1.57

SIN 2915 10 3.56 1.85 0.49 1.33 4 EMI 4468 11 4.54 2.42 0.60 1.53

Paper IOD 4468 15 9.60 3.26 0.75 1.62

SIN 4468 10 4.13 1.93 0.51 1.35

EMI 3006 8 3.78 2.17 0.52 1.50

Online IOD 3006 14 9.03 2.99 0.69 1.66

SIN 3006 14 5.41 2.64 0.67 1.52 5 EMI 4357 8 4.09 1.99 0.45 1.48

Paper IOD 4357 14 9.20 2.91 0.68 1.65

SIN 4357 14 5.48 2.71 0.69 1.52

EMI 3034 14 7.57 2.93 0.63 1.78

Online IOD 3034 14 8.54 3.06 0.72 1.61

SIN 3034 12 5.85 2.72 0.72 1.44 6 EMI 5333 14 8.01 3.03 0.66 1.76

Paper IOD 5333 14 8.94 3.04 0.73 1.57

SIN 5333 12 5.70 2.77 0.72 1.47

EMI 4688 12 5.83 2.94 0.67 1.69

Online IOD 4688 20 10.19 4.24 0.81 1.85

SIN 4688 8 4.20 2.10 0.63 1.27 7 EMI 5329 12 5.68 2.84 0.63 1.72

Paper IOD 5329 20 10.09 4.03 0.78 1.90

SIN 5329 8 3.87 2.03 0.64 1.22

D.9 Appendices

Table D.4. (continued)

Tested Reporting Max Grade Mode Categorya N Points Mean SD Reliability SEM

EMI 8323 13 4.93 2.92 0.68 1.64

Online IOD 8323 17 10.32 3.36 0.75 1.69

SIN 8323 10 4.61 2.28 0.59 1.45 8 EMI 5476 13 5.13 3.08 0.69 1.73

Paper IOD 5476 17 10.44 3.32 0.75 1.65

SIN 5476 10 4.84 2.39 0.63 1.45

EMI 17199 13 5.15 3.13 0.73 1.64

Online IOD 17199 18 7.83 3.63 0.75 1.82

SIN 17199 9 3.82 2.18 0.59 1.40 EHSb EMI 4199 13 5.68 3.41 0.76 1.68

Paper IOD 4199 18 8.92 3.79 0.76 1.87

SIN 4199 9 4.10 2.34 0.65 1.39

a Reporting category: IOD=Interpretation of Data, SIN=Scientific Investigation, and EMI=Evaluation of Models, Inferences, and Experimental Results b Early high school form taken by grades 9 and 10 students.

D.10 Appendices

Appendix E: ACT Aspire ELA Content Framework and Alignment: English Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 EHS (9-10)

3 essays 3 essays 3 essays 3 essays 4 essays 4 essays 4 essays Avg. 150 Avg. 150 Avg. 250 Avg. 300 Avg. 300 Avg. 300 Avg. 300 words each words each words each words each words each words each words each • Increasingly complex text • Increasingly sophisticated language use • Balance of Expressive Narrative, Informational, and Persuasive essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. [Knowledge of Knowledge of Language Language not assessed with Clarity, Style, and Tone Grade 3 items] Demonstrate effective language use through ensuring precision and concision in word choice, maintaining consistency in style and tone, and using language appropriate for the context. Conventions of Standard English Sentence Structure and Formation Recognize common problems with sentence structure and formation in a text and make revisions to improve the writing. Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing.

E.1 Appendices

Grade 3 • 3 essays, average 150 words each • Balance of Expressive Narrative and Informational Essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Identify the purpose of a word or phrase. • Use a word or phrase to accomplish a straightforward rhetorical purpose or effect. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. • Organize sentences in logical sequence within a short paragraph. • Determine the most logical placement for a sentence in a short paragraph. • Use logical ordering for phrases within a sentence. • Use an effective introductory or concluding sentence in a short paragraph with a straightforward topic. • Use effective transition words and phrases to link ideas between sentences (e.g., “For example”). Conventions of Standard English Sentence Structure and Formation Recognize common problems with sentence structure and formation in a text and make revisions to improve the writing. Forming Sentences and Relating Clauses • Correct faulty coordination in compound sentences and issues with subordination in complex sentences appropriate for 3rd-grade students. • Use a coordinating conjunction to join two short, simple sentences. Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Forming and Using Nouns and Pronouns • Use the correct pronoun in a sentence. • Form regular and irregular plural nouns. Forming and Using Verbs • Ensure subject-verb agreement in simple sentences. • Use the correct verb form in a simple sentence (e.g., runs, ran, running) so that it makes sense in context. • Form regular and irregular verbs in various tenses. Forming and Using Modifiers • Form adjectives and adverbs; ensuring the correct degree (comparative, superlative) is used. • Determine whether to use the adjective or adverb form (ending in –ly) in a sentence. Forming Possessives • Correctly form possessive pronouns and adjectives when the possessive relationship is clear. Observing Usage Conventions • Correctly use idiomatic language appropriate for 3rd-grade school contexts.

E.2 Appendices

Grade 3 (continued) Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing. Punctuating Sentences and Sentence Elements • Correct unnecessary punctuation in a short sentence or paragraph. • Eliminate unnecessary punctuation that causes ambiguity from a sentence (e.g., a comma that separates subject and verb in a simple sentence). • Use the correct punctuation at the end of a sentence. Punctuating Possessives • Use an apostrophe to form a possessive noun. Capitalization conventions • Capitalize proper names.

Grade 4 • 3 essays, average 150 words each • Balance of Expressive Narrative and Informational Essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Describe how deleting a sentence changes the development of a short paragraph. • Use a word or phrase to achieve a specific rhetorical purpose or effect (e.g., provide better support for a claim). • Determine if content is relevant to the topic and focus of a paragraph. Provide a reason to support the decision. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. • Organize sentences in logical sequence within a short paragraph. • Determine the most logical placement for a sentence in a paragraph. • Use logical ordering for phrases within a sentence. • Use an effective introductory or concluding sentence in a short paragraph. • Use effective transition words and phrases to emphasize relationships between sentences. • Determine where to divide a short paragraph with multiple topics in order to accomplish a specific rhetorical purpose. Knowledge of Language Clarity, Style, and Tone Demonstrate effective language use through ensuring precision and concision in word choice, maintaining consistency in style and tone, and using language appropriate for the context. • Use words that provide an appropriate level of precision, and express simple relationships within sentences (e.g., conjunctions). • Use words and phrases appropriate to level of text formality (“effective president” vs. “really cool president”).

E.3 Appendices

Grade 4 (continued)

Conventions of Standard English Sentence Structure and Formation Sentence Structure and Formation • Correct faulty coordination, subordination, and parallelism in sentences at the appropriate level of complexity for 4th-grade students. • Use coordinating conjunctions to combine 2-3 short sentences. Fixing Fragments and Run-ons • Identify and correct obvious sentence fragments. • Correct run-on sentences, including comma splices. Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Forming and Using Nouns and Pronouns • Use the correct pronoun in a sentence. • Form regular and irregular plural nouns. Forming and Using Verbs • Correct common mistakes with subject-verb agreement. • Form regular and irregular common verbs in various tenses (rode, became, watches). • Use modal auxiliary verb appropriate to context. Forming and Using Modifiers • Form adjectives and adverbs; ensuring the correct degree (comparative, superlative) is used. • Determine whether to use the adjective or adverb form (ending in –ly) in a sentence. Forming Possessives • Correctly form possessive pronouns and adjectives when the possessive relationship is clear. Observing Usage Conventions • Correctly use idiomatic language appropriate for 4th-grade school contexts. • Distinguishing between and among words frequently confused by students at the 4th-grade level. • Use prepositions appropriate to context. Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing. Punctuating Sentences and Sentence Elements • Correct unnecessary punctuation in a short sentence or paragraph • Eliminate unnecessary punctuation that causes ambiguity from a sentence (e.g., a comma that separates subject and verb in a simple sentence). • Use the correct punctuation at the end of a sentence. • Use a comma before a coordinating conjunction in compound sentences. Punctuating Speech and Textual Quotations • Use quotation marks to punctuate dialogue correctly. Punctuating Possessives • Use context to decide between plural and possessive nouns. Capitalization conventions • Observe capitalization conventions for multi word proper nouns (e.g., Apex Soccer League).

E.4 Appendices

Grade 5 • 3 essays, average 250 words each • Balance of Expressive Narrative and Informational Essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Describe how deleting a sentence changes the development of a paragraph. • Determine whether an essay has met a specified rhetorical purpose and provide examples to explain why. • Use a word or phrase to accomplish a specific rhetorical purpose (e.g., provide better support for a claim). • Determine if content is relevant to the topic and focus of a paragraph. Provide a reason to support the decision. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. • Organize sentences in a logical sequence within a short paragraph; ensure a cohesive flow of ideas. • Determine the most logical placement for a sentence in a paragraph which has several ideas. • Use logical ordering for phrases within a sentence. • Use an effective introductory or concluding sentence that maintains content focus without redundancy in a short paragraph. • Use effective transition words or phrases appropriate to content and genre (e.g., later v second for a narrative). • Determine where to divide a short paragraph with multiple topics in order to accomplish a specific rhetorical purpose. • Order paragraphs in a logical sequence. Knowledge of Language Clarity, Style, and Tone Demonstrate effective language use through ensuring precision and concision in word choice, maintaining consistency in style and tone, and using language appropriate for the context. • Use words that provide an appropriate level of precision, and express simple relationships within sentences (e.g., conjunctions). • Use words and phrases appropriate to level of text formality. Conventions of Standard English Sentence Structure and Formation Recognize common problems with sentence structure and formation in a text and make revisions to improve the writing. Forming Sentences and Relating Clauses • Correct faulty coordination, subordination, and parallelism in sentences at the appropriate level of complexity for 5th-grade students. Fixing Fragments and Run-ons • Identify and correct longer sentence fragments. • Correct run-on sentences, including comma splices. Maintaining Consistency and Coherence • Correct inappropriate shifts in verb tense and aspect.

E.5 Appendices

Grade 5 (continued) Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Forming and Using Nouns and Pronouns • Use the correct pronoun in a sentence. Forming and Using Verbs • Correct common mistakes with subject-verb agreement. • Form regular and irregular common verbs in various tenses (rode, became, watches). • Use modal auxiliary verb appropriate to context. Forming and Using Modifiers • Form adjectives and adverbs; ensuring the correct degree (comparative, superlative) is used. • Determine whether to use the adjective or adverb form (ending in –ly) in a sentence. Forming Possessives • Correctly form possessive adjectives when the possessive relationship is somewhat clear. Observing Usage Conventions • Correctly use idiomatic language appropriate for 5th-grade school contexts. • Distinguish between and among words frequently confused by students at the 5th-grade level. • Use prepositions appropriate to context. Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing. Punctuating Sentences and Sentence Elements • Correct unnecessary punctuation in a short sentence or paragraph. • Eliminate unnecessary punctuation that causes ambiguity from a sentence (e.g., a comma that separates subject and verb in a simple sentence). • Use a comma before a coordinating conjunction in compound sentences. • Use a comma to correctly punctuate a phrase at the beginning of a sentence. • Use a comma to set off tag questions and yes and no. • Use commas in a sentence to correctly separate items in a series. • Distinguish between restrictive/nonrestrictive or essential/nonessential sentence elements. Punctuating Speech and Textual Quotations • Punctuate dialogue, including exclamations and questions. • Punctuating Possessives • Form possessive singular and plural nouns and use them in appropriate contexts. Capitalization conventions • Apply conventions of capitalization.

E.6 Appendices

Grade 6 • 3 essays, average 300 words each • Balance of Expressive Narrative and Informational Essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Determine the writer’s purpose for including a given sentence. • Describe how deleting a phrase or sentence changes the development of a paragraph. • Determine whether an essay has met a specified rhetorical purpose and provide examples to explain why. • Use a word or phrase to accomplish a specific rhetorical purpose (e.g., provide better support for a claim). • Determine if content is relevant to the topic and focus of a paragraph. Provide a reason to support the decision. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. • Organize sentences in a logical sequence within a paragraph; ensure a cohesive flow of ideas. • Add or move sentences in order to improve local cohesion and clarify the paragraph as a whole. • Use logical ordering for phrases within a sentence. • Use an effective introductory, concluding or transition sentence to emphasize relationships within and between shorter paragraphs. • Use effective transition words or phrases to connect ideas in two adjacent sentences and to connect a new paragraph with ideas in the previous (e.g., on the other hand). • Determine where to divide a short paragraph with multiple topics in order to accomplish a specific rhetorical purpose. • Order paragraphs in a logical sequence to improve cohesion. Knowledge of Language Clarity, Style, and Tone Demonstrate effective language use through ensuring precision and concision in word choice, maintaining consistency in style and tone, and using language appropriate for the context. • Revise a redundant phrase to make the sentence more concise. • Use words and phrases to precisely express ideas and relationships (e.g., subordinating conjunctions, conjunctive adverb). • Revise a word or phrase so that the sentence matches the tone of the essay (either too informal or too formal for a given purpose). Conventions of Standard English Sentence Structure and Formation Recognize common problems with sentence structure and formation in a text and make revisions to improve the writing. Forming Sentences and Relating Clauses • Correct faulty coordination, subordination, and parallelism in sentences at the appropriate level of complexity for 6th-grade students. Fixing Fragments and Run-ons • Identify and correct longer sentence fragments. • Correct run-on sentences, including comma splices. Maintaining Consistency and Coherence • Drawing on the context of a sentence or paragraph, correct an inappropriate shift in pronoun number or person. • Correct inappropriate shifts in verb tense and aspect, including auxiliary verbs.

E.7 Appendices

Grade 6 (continued) Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Forming and Using Nouns and Pronouns • Correct common mistakes with pronoun-antecedent agreement. • Correct vague or ambiguous pronouns to improve local coherence. • Use appropriate pronoun case.. Forming and Using Verbs • Correct common mistakes with subject-verb agreement. • Form regular and irregular verbs to correctly express tense and aspect of actions (e.g., she reads, yesterday she had read). Forming and Using Modifiers • Form adjectives and adverbs; ensuring the correct degree (comparative, superlative) is used. • Make distinctions between different comparative constructions (e.g., fewer than, less than). • Determine whether to use the adjective or adverb form of modifiers in a sentence with pairs of modifiers (e.g., “she spoke quietly but quickly”). Forming Possessives • Correctly form possessive adjectives when the possessive relationship is somewhat clear. Observing Usage Conventions • Correctly use idiomatic language appropriate for 6th-grade school contexts. • Distinguish among words frequently confused by students at the 6th-grade level. • Form and use prepositional phrases to express specific actions (e.g., “comes from outside,” “look up in the tree”). Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing. Punctuating Sentences and Sentence Elements • Correct unnecessary punctuation from a longer sentence with a relatively complex structure (e.g., comma used before a coordinating conjunction when there are not independent clauses). • Eliminate unnecessary punctuation that causes ambiguity from a sentence when the context is more complex. • Use a comma to correctly punctuate a phrase at the beginning of a sentence. • Use a comma to correctly indicate direct address. • Use a comma to set off tag questions and yes and no. • Use commas in a sentence to correctly separate items in a series. • Determine if a phrase is restrictive/nonrestrictive and use commas to set it off, if necessary Punctuating Possessives • Form possessive singular and plural nouns and use them in appropriate contexts.

E.8 Appendices

Grade 7 • 4 essays, average 300 words each • Balance of Expressive Narrative, Informational, and Persuasive Essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Describe how deleting a phrase or sentence effects the development of a longer paragraph. • Determine whether an essay has met a specified rhetorical purpose and provide examples to explain why. • Use a word or phrase to accomplish a specific rhetorical purpose (e.g., provide better support for a claim). • Determine if content is relevant to the topic and focus of a paragraph and/or to the essay as a whole. Provide a reason to support the decision. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. • Organize sentences in a logical sequence within a longer paragraph to accomplish a specific rhetorical purpose; ensure a cohesive flow of ideas. • Add or move sentences in order to improve cohesion of the essay as a whole. • Use logical ordering for phrases within a sentence. • Use an effective introductory, concluding or transition sentence to emphasize relationships within and between paragraphs. • Use effective transition words or phrases to emphasize relationships among ideas. • Determine where to divide a paragraph with multiple topics in order to accomplish a specific rhetorical purpose. • Order paragraphs in a logical sequence to improve cohesion. Knowledge of Language Clarity, Style, and Tone Demonstrate effective language use through ensuring precision and concision in word choice, maintaining consistency in style and tone, and using language appropriate for the context. • Revise a redundant phrase to make the sentence more concise. • Use words and phrases to precisely express ideas and relationships (e.g., subordinating conjunctions, conjunctive adverb). • Revise a word or phrase so that the sentence matches the tone of the essay (either too informal or too formal for a given purpose). Conventions of Standard English Sentence Structure Recognize common problems with sentence structure and formation in a text and make revisions to improve the writing. Forming Sentences and Relating Clauses • Correct faulty coordination, subordination, and parallelism in sentences at the appropriate level of complexity for 7th-grade students. • Correct common problems with misplaced modifiers. • Correct common problems with squinters and danglers. Fixing Fragments and Run-ons • Identify and correct sentence fragments when the fragments are more complex in structure. • Correct run-on sentences, including comma splices. Maintaining Consistency and Coherence • Drawing on the context of a sentence or paragraph, correct an inappropriate shift in pronoun number or person. • Correct inappropriate shifts in verb tense and aspect, including auxiliary verbs.

E.9 Appendices

Grade 7 (continued) Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Forming and Using Nouns and Pronouns • Correct common mistakes with pronoun-antecedent agreement. • Correct vague or ambiguous pronouns to improve local coherence. • Use appropriate pronoun case. Forming and Using Verbs • Correct common mistakes with subject-verb agreement in compound and complex sentences. Forming and Using Modifiers • Form adjectives and adverbs; ensuring the correct degree (comparative, superlative) is used. • Make distinctions between different comparative constructions (e.g., fewer than, less than). • Determine whether to use the adjective or adverb form of modifiers in a sentence with pairs of modifiers (e.g., “she spoke quietly but quickly”). Forming Possessives • Correctly form possessive adjectives in sentences with somewhat complicated structures. Observing Usage Conventions • Correctly use idiomatic language appropriate for 7th-grade school contexts. • Distinguish among words frequently confused by students at the 7th-grade level. Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing. Punctuating Sentences and Sentence Elements • Correct unnecessary punctuation from longer sentences (e.g., those containing prepositional phrases and compound items in a series). • Eliminate unnecessary punctuation that causes ambiguity from a sentence when the context is more complex. • Use commas to separate multi-word phrases in a series. • Determine if a phrase is restrictive/nonrestrictive and use commas to set it off, if necessary. Punctuating Possessives • Form possessive singular and plural nouns and use them in appropriate contexts

E.10 Appendices

Grade 8 • 4 essays, average 300 words each • Balance of Expressive Narrative and Informational Essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Describe how deleting a phrase or sentence changes the development of a paragraph or the essay as a whole. • Determine whether an essay has met a specified rhetorical purpose and provide examples to explain why. • Use a word or phrase to accomplish a specific rhetorical purpose (e.g., provide better support for a claim). • Determine if content is relevant to the topic and focus of a paragraph and/or to the essay as a whole when the context is more complex. Provide a reason to support the decision. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. • Organize sentences in a logical sequence within a longer paragraph to accomplish a specific rhetorical purpose; ensure a cohesive flow of ideas. • Add or move sentences in order to improve cohesion of the essay as a whole. • Use logical ordering for phrases within a sentence. • Use an effective introductory, concluding or transition sentence to emphasize relationships within and between paragraphs. • Use effective transition words or phrases to emphasize relationships among ideas. • Determine where to divide a more complex paragraph with multiple topics in order to accomplish a specific rhetorical purpose. • Order paragraphs in a logical sequence to improve cohesion and accomplish a specific rhetorical purpose. Knowledge of Language Clarity, Style, and Tone Demonstrate effective language use through ensuring precision and concision in word choice, maintaining consistency in style and tone, and using language appropriate for the context. • Revise a redundant phrase to make the sentence or paragraph more concise. • Use words and phrases to precisely express ideas and relationships (e.g., subordinating conjunctions, conjunctive adverb). • Revise a word or phrase so that the sentence matches the tone of the essay (either too informal or too formal for a given purpose). Conventions of Standard English Sentence Structure and Formation Recognize common problems with sentence structure and formation in a text and make revisions to improve the writing. Forming Sentences and Relating Clauses • Correct faulty coordination, subordination, and parallelism in sentences at the appropriate level of complexity for 8th-grade students. • Correct common problems with misplaced modifiers. • Correct common problems with squinters and danglers. Fixing Fragments and Run-ons • Identify and correct sentence fragments when the fragments are more complex in structure. • Correct run-on sentences, including comma splices. Maintaining Consistency and Coherence • Drawing on the context of the paragraph, correct an inappropriate shift in pronoun number or person. • Correct inappropriate shifts in verb tense and aspect, including auxiliary verbs. • Correct inappropriate shifts in verb voice. E.11 Appendices

Grade 8 (continued) Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Forming and Using Nouns and Pronouns • Correct common mistakes with pronoun-antecedent agreement. • Correct vague or ambiguous pronouns to improve local coherence. • Use appropriate pronoun case. Forming and Using Verbs • Correct more advanced mistakes with subject-verb agreement (e.g., compound subjects) in compound and complex sentences. Forming and Using Modifiers • Form adjectives and adverbs; ensuring the correct degree (comparative, superlative) is used. • Make distinctions between different comparative constructions (e.g., fewer than, less than). • Determine whether to use the adjective or adverb form of modifiers in a sentence with pairs of modifiers (e.g., “she spoke quietly but quickly”). Forming Possessives • Correctly form possessive adjectives in sentences with complicated structures. Observing Usage Conventions • Correctly use idiomatic language appropriate for 8th-grade school contexts. • Distinguish among words frequently confused by students at the 8th-grade level Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing. Punctuating Sentences and Sentence Elements • Correct unnecessary punctuation from sentences with relatively complex structure. • Eliminate unnecessary punctuation that causes ambiguity from a sentence when the context is more complex. • Use commas to separate items in a series in a complex sentence. • Determine if a phrase is restrictive/nonrestrictive and use commas to set it off, if necessary. Punctuating Possessives • Form possessive singular and plural nouns and use them in appropriate contexts.

E.12 Appendices

Early High School (9-10) • 4 essays, average 300 words each • Balance of Expressive Narrative and Informational Essays Production of Writing Text Purposes and Topic Development Demonstrate an understanding of and control over the rhetorical aspects of texts by identifying the purpose of parts of texts, determining whether a text has met its intended goal, and evaluating the relevance of material in terms of a text’s focus. • Describe how deleting a supporting detail changes the meaning of a complex sentence and the overall development of the essay as a whole. • Decide whether information should be deleted from an illustrative sentence and explain the rationale for the decision. • Determine whether an essay has met a complex rhetorical purpose and provide examples to explain why. • Use a word, phrase, or sentence to accomplish a specific rhetorical purpose (e.g., express an idea with nuance or emotional overtones). • Determine if content is relevant to the topic and focus of a paragraph and/or to the essay as a whole when the context is more complex. Provide a reason to support the decision. Organization Use various strategies to ensure that a text is logically organized, flows smoothly, and has an effective introduction and conclusion. • Organize sentences in a logical sequence within a long paragraph to accomplish a specific rhetorical purpose; ensure a cohesive flow of ideas. • Add or move sentences in order to improve cohesion of the essay as a whole when the context is more complex. • Use logical ordering for phrases within a sentence. • Use an effective introductory, concluding or transition sentence to emphasize relationships within and between longer, more complex paragraphs. • Use effective transition words or phrases to emphasize complex/subtle relationships among ideas in longer paragraphs and across paragraphs. • Determine where to divide a complex paragraph with multiple topics in order to accomplish a specific rhetorical purpose. • Order paragraphs in a logical sequence to improve cohesion and accomplish a specific rhetorical purpose. Knowledge of Language Clarity, Style, and Tone Demonstrate effective language use through ensuring precision and concision in word choice, maintaining consistency in style and tone, and using language appropriate for the context. • Revise a redundant phrase to make a sentence or paragraph more concise when the context is more complex. • Use words and phrases to precisely express ideas and relationships (e.g., subordinating conjunctions, conjunctive adverb). • Revise a word or phrase so that the sentence matches the tone of the essay (either too informal or too formal for a given purpose).

E.13 Appendices

Early High School (9-10) (continued) Conventions of Standard English Sentence Structure and Formation Recognize common problems with sentence structure and formation in a text and make revisions to improve the writing. Forming Sentences and Relating Clauses • Correct faulty coordination and subordination and parallelism in sentences at the appropriate level of complexity for early- high-school students. • Correct common problems with misplaced modifiers. • Correct common problems with squinters and danglers. Fixing Fragments and Run-ons • Identify and correct sentence fragments when the fragments are more complex in structure. • Correct run-on sentences, including comma splices. Maintaining Consistency and Coherence • Ensure that verb mood is used consistently. • Drawing on the context of the paragraph or preceding paragraphs, correct an inappropriate shift in pronoun number or person. • Correct inappropriate shifts in verb tense and aspect, including auxiliary verbs. • Correct inappropriate shifts in verb voice. Usage Recognize common problems with standard English usage in a text and make revisions to improve the writing. Forming and Using Nouns and Pronouns • Correct common mistakes with pronoun-antecedent agreement. • Correct vague or ambiguous pronouns to improve coherence within the essay. • Use appropriate pronoun case. Forming and Using Verbs • Correct more advanced mistakes with subject-verb agreement (e.g., sentences with a number of interceding elements) in compound and complex sentences. Forming Possessives • Correctly form possessive adjectives in sentences with complicated structures. Observing Usage Conventions • Correctly use idiomatic language appropriate for early-high-school contexts. • Distinguish among words frequently confused by students at the early-high-school level. Punctuation and Capitalization Recognize common problems with standard English punctuation and capitalization and make revisions to improve the writing. Punctuating Sentences and Sentence Elements • Correct unnecessary punctuation from longer sentences with a complex structures. • Eliminate unnecessary punctuation that causes ambiguity from a sentence when the context is more complex. • Use commas to separate items in a series in a complex sentence. • Determine if a phrase is restrictive/nonrestrictive and use commas to set it off, if necessary. Punctuating Possessives • Form possessive singular and plural nouns and use them in appropriate contexts.

E.14 Appendices

Appendix F: ACT Aspire ELA Content Framework and Alignment: Reading

Somewhat More Basic Straightforward Challenging Challenging Complex Highly Complex ACT 4 passages: 1 Literary + 3 Informational Average Word Count: 500, CCSS FK Range: 10.34-14.2 Early High School (Grades 9-10) 3 passages: 1 Literary + 2 Informational Average Word Count: 500, CCSS FK Range: 8.32-12.12 Grade 8 3 passages: 1 Literary + 2 Informational Average Word Count: 500, CCSS FK Range: 6.51-10.34 Grade 7 3 passages: 1 Literary + 2 Informational Average Word Count: 500, CCSS FK Range: 6.51-10.34 Grade 6 3 passages: 1 Literary + 2 Informational Average Word Count: 500, CCSS FK Range: 6.51-10.34 Grade 5 4 passages: 2 Literary + 2 Informational Average Word Count: 375, CCSS FK Range: 4.51-7.73 Grade 4 4 passages: 2 Literary + 2 Informational Average Word Count: 300, CCSS FK Range: 4.51-7.73 Grade 3 4 passages: 2 Literary + 2 Informational Average Word Count: 250, CCSS FK Range: 1.98-5.34

F.1 Appendices

Grade 3 4 passages: 2 Literary + 2 Informational Average Word Count: 250, CCSS FK Range: 1.98-5.34 Key Ideas and Details Close Reading Locate and interpret details, paraphrase information and ideas accurately, and draw logical conclusions based on textual evidence. • Demonstrate understanding of basic and straightforward texts, using the text as the basis for the answers Central Ideas, Themes & Summaries Identify and infer whole-text-level central (main) ideas and themes, identify and infer central ideas of defined parts of texts, and summarize information and ideas accurately. • Determine the main ideas of basic and straightforward texts and identify key details to support them • Summarize basic and straightforward texts • Summarize stories with plots in chronological order and simple, external conflicts • Determine the central message, lesson, or moral of a story and identify key details to support them Relationships Identify and infer sequences, comparative relationships (comparisons and contrasts), and cause-effect relationships in texts. • Describe the relationship between a series of historical events, scientific ideas or concepts, or steps in a process • Describe and/or compare the traits, motivations, or feelings of a character • Explain how characters’ actions contribute to the sequence of events Craft and Structure Word Meanings & Word Choice Determine the meaning of words and phrases and to analyze word choice rhetorically. • Determine the meaning of general academic and domain-specific words and phrases in basic and straightforward texts relevant to a Grade 3 topic or subject area • Determine the meaning of words and phrases as they are used in basic and straightforward stories • Determine the meaning of simple, non-literal language and literary devices, such as simple similes and metaphors Text Structure Analyze text structure rhetorically at the whole-text and partial-text levels. • Use text features to identify the purpose and function of overall and specific text portions • Describe how parts of a story affect its overall theme, moral, or lesson Purpose & Point of View Understand the purpose or purposes for which texts are written, understand point of view, evaluate the choice of narration in literary texts, and analyze bias, stereotypes, and persuasive techniques in texts. • Distinguish author’s point of view from one’s own point of view • Distinguish a narrator’s point of view from the point of view of an author or other characters’ point of view

F.2 Appendices

Grade 3 (continued) Integration of Knowledge and Ideas Arguments Analyze arguments for their claims, counterclaims, reasons, evidence, and soundness of reasoning. • Distinguish between facts and opinions • Describe and identify logical connections between particular sentences and paragraphs of a text Multiple Texts Make connections between two or more texts in the same medium and format. • Compare and contrast the most important points and key details presented in two texts on the same topic • Compare and contrast the themes, settings, and plots of multiple stories Basic Straightforward BASIC texts will have simple purposes and use primarily STRAIGHTFORWARD texts will have clear purpose and short and basic sentences. Vocabulary will mostly be familiar organization with explicit transitions and include a variety of and contemporary, with some polysyllabic words and domain- sentence structures. Vocabulary may include some general specific language. Most objects and events will be concrete. academic or uncommon words and phrases. These texts Ideas and concepts will be few, clear, and often repeated. may require inferential reading and contain some abstract Informational texts will have clear focus on one subject and concepts and diverse perspectives. use a simple structure with basic transitions. Informational texts will exhibit features of the field of study Literary texts will be primarily literal, with plots in (e.g. science), including some domain-specific language and chronological order and simple, usually external, conflicts. assumption of broad content knowledge. These texts will have a single narrator and generally flat Literary texts will have a single narrator and conflicts that characters. These texts may include a few instances of are straightforward and usually external. Plots may deviate simple non-literal language and literary devices such as from chronological order with simple, heavily signaled flash- simple similes and metaphors. backs or flash-forwards. These texts may include some non- literal language and simple literary devices. Major and minor characters may show some depth and the text will offer basic insights into characters, people, situations, and events (e.g., simple motives).

F.3 Appendices

Grade 4 4 passages: 2 Literary + 2 Informational Average Word Count: 300, CCSS FK Range: 4.51-7.73 Key Ideas and Details Close Reading Locate and interpret details, paraphrase information and ideas accurately, and draw logical conclusions based on textual evidence. • Refer to details and examples in basic and straightforward texts when explaining what the text says explicitly and when drawing inferences from the text Central Ideas, Themes & Summaries Identify and infer whole-text-level central (main) ideas and themes, identify and infer central ideas of defined parts of texts, and summarize information and ideas accurately. • Determine the main idea and themes of a basic or straight forward text and explain how it is supported by key details • Summarize the main ideas and themes of basic and straightforward texts Relationships Identify and infer sequences, comparative relationships (comparisons and contrasts), and cause-effect relationships in texts. • Use specific details from a story, such as a characters’ words, thoughts, and actions, to describe a character, setting, or event • Draw comparisons between ideas, characters, problems in stories • Explain events, procedures, ideas, or concepts in a historical, scientific, or technical text, including what happened and why, based on specific information in the text Craft and Structure Word Meanings & Word Choice Determine the meaning of words and phrases and to analyze word choice rhetorically. • Determine the meaning of general academic and domain-specific words and phrases in texts relevant to a Grade 4 topic or subject area • Analyze how word choice affects meanings in stories Text Structure Analyze text structure rhetorically at the whole-text and partial-text levels. • Describe the purpose or function of structures within a story • Describe the purpose or function of paragraphs in an informational text as they relate to chronology, comparison, and cause/effect Purpose & Point of View Understand the purpose or purposes for which texts are written, understand point of view, evaluate the choice of narration in literary texts, and analyze bias, stereotypes, and persuasive techniques in texts. • Identify the point of view from which different stories are narrated, including the difference between first-and third person narrations • Determine the overall purpose of a text • Identify bias, stereotypes, unstated assumptions, and persuasive techniques

F.4 Appendices

Grade 4 (continued) Integration of Knowledge and Ideas Arguments Analyze arguments for their claims, counterclaims, reasons, evidence, and soundness of reasoning. • Distinguish between facts and opinions and identify how they are used in a text • Explain how an author uses reasons and evidence to support particular points in a text. Multiple Texts (SYN) Make connections between two or more texts in the same medium and format. • Integrate information from two texts on the same topic to provide more knowledge about the topic • Compare and contrast the treatment of similar topics, and patterns of events in stories Basic Straightforward BASIC texts will have simple purposes and use primarily STRAIGHTFORWARD texts will have clear purpose and short and basic sentences. Vocabulary will mostly be familiar organization with explicit transitions and include a variety of and contemporary, with some polysyllabic words and domain- sentence structures. Vocabulary may include some general specific language. Most objects and events will be concrete. academic or uncommon words and phrases. These texts Ideas and concepts will be few, clear, and often repeated. may require inferential reading and contain some abstract Informational texts will have clear focus on one subject and concepts and diverse perspectives. use a simple structure with basic transitions. Informational texts will exhibit features of the field of study Literary texts will be primarily literal, with plots in (e.g. science), including some domain-specific language and chronological order and simple, usually external, conflicts. assumption of broad content knowledge. These texts will have a single narrator and generally flat Literary texts will have a single narrator and conflicts that characters. These texts may include a few instances of are straightforward and usually external. Plots may deviate simple non-literal language and literary devices such as from chronological order with simple, heavily signaled flash- simple similes and metaphors. backs or flash-forwards. These texts may include some non-literal language and simple literary devices. Major and minor characters may show some depth and the text will offer basic insights into characters, people, situations, and events (e.g., simple motives).

F.5 Appendices

Grade 5 4 passages: 2 Literary + 2 Informational Average Word Count: 375, CCSS FK Range: 4.51-7.73 Key Ideas and Details Close Reading Locate and interpret details, paraphrase information and ideas accurately, and draw logical conclusions based on textual evidence. • Use accurately quoted or paraphrased evidence from basic, straightforward, and somewhat challenging texts to draw inferences • Use accurately quoted or paraphrased details from basic, straightforward, and somewhat challenging texts to support conclusions Central Ideas, Themes & Summaries Identify and infer whole-text-level central (main) ideas and themes, identify and infer central ideas of defined parts of texts, and summarize information and ideas accurately. • Explain how the main idea of a basic, straightforward, or somewhat challenging text are supported by key details • Determine a theme of a story from details in the text, including how the characters in a story respond to challenges and/or reflect upon a topic Relationships Identify and infer sequences, comparative relationships (comparisons and contrasts), and cause-effect relationships in texts. • Explain the relationships or interactions between two or more individuals, events, ideas, or concepts in a historical, scientific, or technical text based on specific information in the text • Compare and contrast two or more characters, settings, or events in a story, drawing on specific details in the text Craft and Structure Word Meanings & Word Choice Determine the meaning of words and phrases and to analyze word choice rhetorically. • Determine the meaning or general academic and domain-specific words and phrases in a text relevant to a grade 5 topic or subject area • Determine the meaning of words and phrases as they are used in a text, including figurative language such as metaphors and similes Text Structure Analyze text structure rhetorically at the whole-text and partial-text levels. • Demonstrate an understanding of different text structures, such as chronology, comparison, cause/effect, problem solution • Explain how the purpose and function of parts of a story in relationship to the overall theme, message, or outcome of the story Purpose & Point of View Understand the purpose or purposes for which texts are written, understand point of view, evaluate the choice of narration in literary texts, and analyze bias, stereotypes, and persuasive techniques in texts. • Recognize how point of view influences the information provided about a topic • Describe how a narrator’s point of view influences how events are described

F.6 Appendices

Grade 5 (continued) Integration of Knowledge and Ideas Arguments Analyze arguments for their claims, counterclaims, reasons, evidence, and soundness of reasoning. • Distinguish between facts and opinions and identify how they are used in a text • Explain how an author uses reasons and evidence to support particular points in a text, identifying which reasons and evidence support which points Multiple Texts Make connections between two or more texts in the same medium and format. • Synthesize information from two or more texts on the same topic to offer new ideas about that topic • Compare and contrast stories in the same genre on their approaches to similar themes and topics Basic Straightforward Somewhat Challenging BASIC texts will have simple purposes STRAIGHTFORWARD texts will have SOMEWHAT CHALLENGING and use primarily short and basic clear purpose and organization with texts will have clear purpose and sentences. Vocabulary will mostly be explicit transitions and include a variety organization, with variety in sentence familiar and contemporary, with some of sentence structures. Vocabulary styles and transitions. Vocabulary may polysyllabic words and domain-specific may include some general academic or include some general academic or language. Most objects and events will uncommon words and phrases. These uncommon words and phrases. These be concrete. Ideas and concepts will be texts may require inferential reading texts will require inferential reading few, clear, and often repeated. and contain some abstract concepts and contain abstract concepts as well Informational texts will have clear and diverse perspectives. as experiences and perspectives that focus on one subject and use a simple Informational texts will exhibit features may be different that those of the structure with basic transitions. of the field of study (e.g. science), reader. Literary texts will be primarily literal, with including some domain-specific Informational texts will exhibit features plots in chronological order and simple, language and assumption of broad of the general discipline (e.g. natural usually external, conflicts. These texts content knowledge. science), including domain-specific will have a single narrator and generally Literary texts will have a single narrator language and assumption of everyday flat characters. These texts may include and conflicts that are straightforward and content knowledge appropriate to a few instances of simple non-literal and usually external. Plots may deviate the grade assessed. language and literary devices such as from chronological order with simple, Literary texts may have multiple simple similes and metaphors. heavily signaled flash-backs or flash- narrators and non-linear plots, with forwards. These texts may include shifts in point of view or time clearly some non-literal language and simple signaled. Characters will be dynamic literary devices. Major and minor and experience uncomplicated internal characters may show some depth and or external conflicts. These texts the text will offer basic insights into may include non-literal language and characters, people, situations, and literary devices such as symbolism or events (e.g., simple motives). irony. Themes and subject matter may be challenging, offering insights into people, situations and events.

F.7 Appendices

Grade 6 3 passages: 1 Literary + 2 Informational, Average Word Count: 500, CCSS FK Range: 6.51-10.34 Key Ideas and Details Close Reading Locate and interpret details, paraphrase information and ideas accurately, and draw logical conclusions based on textual evidence. • Use textual evidence from a basic, straightforward, or somewhat challenging text to support analysis of what the text says explicitly as well as inferences drawn from the text Central Ideas, Themes & Summaries Identify and infer whole-text-level central (main) ideas and themes, identify and infer central ideas of defined parts of texts, and summarize information and ideas accurately. • Determine a central idea of a basic, straightforward, or somewhat challenging text and identify how this idea is communicated through strategically selected evidence • Determine a central theme of a basic, straightforward, or somewhat challenging text and identify how this theme is communicated through strategically selected details • Summarize a basic, straightforward, or somewhat challenging text based only on ideas from the text rather than from prior knowledge or personal opinion Relationships Identify and infer sequences, comparative relationships (comparisons and contrasts), and cause-effect relationships in texts. • Analyze how comparison, cause and effect, and sequence are used to introduce, illustrate, or elaborate upon a key event, idea or individual • Describe how a story’s plot unfolds in a series of events • Describe how characters’ respond or change as a plot moves toward a resolution Craft and Structure Word Meanings & Word Choice Determine the meaning of words and phrases and to analyze word choice rhetorically. • Determine figurative and connotative meaning • Analyze how word choice influences meaning and tone Text Structure (TST) Analyze text structure rhetorically at the whole-text and partial-text levels. • Analyze how a particular sentence or paragraph fits into the overall structure of a text and contributes to the development of ideas • Analyze how a particular story event or scene fits into the overall development of a theme, setting, or plot Purpose & Point of View Assess how point of view or purpose shapes the content and style of a text. (CCSS R.6) • Determine how an author’s point of view or purpose is conveyed in a text • Explain how an author develops the point of view of a narrator in a text Integration of Knowledge and Ideas Arguments Analyze arguments for their claims, counterclaims, reasons, evidence, and soundness of reasoning. • Distinguish between facts and opinions and identify how they are used in a text • Trace and evaluate the argument and specific claims in a text, distinguishing claims that are supported by reasons and evidence from claims that are not

F.8 Appendices

Grade 6 (continued) Multiple Texts Make connections between two or more texts in the same medium and format. • Compare and contrast one author’s presentation of information with that of another’s • Compare and contrast texts in different forms or genres in terms of their approaches to similar themes and topics Straightforward Somewhat Challenging STRAIGHTFORWARD texts will have clear purpose and SOMEWHAT CHALLENGING texts will have clear purpose organization with explicit transitions and include a variety of and organization, with variety in sentence styles and sentence structures. Vocabulary may include some general transitions. Vocabulary may include some general academic academic or uncommon words and phrases. These texts or uncommon words and phrases. These texts will require may require inferential reading and contain some abstract inferential reading and contain abstract concepts as well concepts and diverse perspectives. as experiences and perspectives that may be different that Informational texts will exhibit features of the field of study those of the reader. (e.g. science), including some domain-specific language and Informational texts will exhibit features of the general assumption of broad content knowledge. discipline (e.g. natural science), including domain-specific Literary texts will have a single narrator and conflicts that language and assumption of everyday and content are straightforward and usually external. Plots may deviate knowledge appropriate to the grade assessed. from chronological order with simple, heavily signaled flash- Literary texts may have multiple narrators and non-linear backs or flash-forwards. These texts may include some non- plots, with shifts in point of view or time clearly signaled. literal language and simple literary devices. Major and minor Characters will be dynamic and experience uncomplicated characters may show some depth and the text will offer internal or external conflicts. These texts may include non- basic insights into characters, people, situations, and events literal language and literary devices such as symbolism (e.g., simple motives). or irony. Themes and subject matter may be challenging, offering insights into people, situations and events.

Grade 7 3 passages: 1 Literary + 2 Informational Average Word Count: 500, CCSS FK Range: 6.51-10.34 Key Ideas and Details Close Reading Locate and interpret details, paraphrase information and ideas accurately, and draw logical conclusions based on textual evidence. • Use textual evidence from straightforward and somewhat challenging texts to support analysis of what the text says explicitly as well as inferences drawn from the text Central Ideas, Themes & Summaries Identify and infer whole-text-level central (main) ideas and themes, identify and infer central ideas of defined parts of texts, and summarize information and ideas accurately. • Determine a central idea in a straightforward or somewhat challenging text and analyze its development over the course of the text • Determine a central theme in a straightforward or somewhat challenging text and analyze its development over the course of the text • Provide an objective summary of a straightforward or somewhat challenging text Relationships Identify and infer sequences, comparative relationships (comparisons and contrasts), and cause-effect relationships in texts. • Analyze the interactions between individuals, events, and ideas in a text

F.9 Appendices

Grade 7 (continued) Craft and Structure Word Meanings & Word Choice Determine the meaning of words and phrases and to analyze word choice rhetorically. • Determine the meaning of words and phrases as they are used in a text, including figurative, connotative, and technical meanings • Analyze the impact of a specific word choice on meaning and tone Text Structure Analyze text structure rhetorically at the whole-text and partial-text levels. • Analyze the structure an author uses to organize a text, including how the major sections contribute to the whole and to the development of the ideas Purpose & Point of View Understand the purpose or purposes for which texts are written, understand point of view, evaluate the choice of narration in literary texts, and analyze bias, stereotypes, and persuasive techniques in texts. • Determine an author’s point of view and how the author distinguishes his or her position from that of others • Determine an author’s purpose • Analyze how an author develops and contrasts the points of view of different characters or narrators in a text Integration of Knowledge and Ideas Arguments Analyze arguments for their claims, counterclaims, reasons, evidence, and soundness of reasoning. • Trace and evaluate the argument and specific claims in a text • Assess whether evidence is relevant and sufficient to support the claims Multiple Texts Make connections between two or more texts in the same medium and format. • Analyze how two authors writing about the same topic hone their presentations of key information by emphasizing different evidence or advancing different interpretations of facts • Compare and contrast two fictional portrayals of a time, place, or character Straightforward Somewhat Challenging STRAIGHTFORWARD texts will have clear purpose and SOMEWHAT CHALLENGING texts will have clear purpose organization with explicit transitions and include a variety of and organization, with variety in sentence styles and sentence structures. Vocabulary may include some general transitions. Vocabulary may include some general academic academic or uncommon words and phrases. These texts or uncommon words and phrases. These texts will require may require inferential reading and contain some abstract inferential reading and contain abstract concepts as well concepts and diverse perspectives. as experiences and perspectives that may be different that Informational texts will exhibit features of the field of study those of the reader. (e.g. science), including some domain-specific language and Informational texts will exhibit features of the general assumption of broad content knowledge. discipline (e.g. natural science), including domain-specific Literary texts will have a single narrator and conflicts that language and assumption of everyday and content are straightforward and usually external. Plots may deviate knowledge appropriate to the grade assessed. from chronological order with simple, heavily signaled flash- Literary texts may have multiple narrators and non-linear backs or flash-forwards. These texts may include some non- plots, with shifts in point of view or time clearly signaled. literal language and simple literary devices. Major and minor Characters will be dynamic and experience uncomplicated characters may show some depth and the text will offer internal or external conflicts. These texts may include non- basic insights into characters, people, situations, and events literal language and literary devices such as symbolism (e.g., simple motives). or irony. Themes and subject matter may be challenging, offering insights into people, situations and events.

F.10 Appendices

Grade 8 3 passages: 1 Literary + 2 Informational Average Word Count: 500, CCSS FK Range: 6.51-10.34 Key Ideas and Details Close Reading Locate and interpret details, paraphrase information and ideas accurately, and draw logical conclusions based on textual evidence. • Cite the textual evidence from somewhat challenging and more challenging texts that most strongly supports an analysis of what the text says explicitly as well as inferences drawn from the text Central Ideas, Themes & Summaries Identify and infer whole-text-level central (main) ideas and themes, identify and infer central ideas of defined parts of texts, and summarize information and ideas accurately. • Determine a central idea of a somewhat challenging or more challenging text and analyze its development over the course of the text, including its relationship to supporting ideas • Determine a theme of a somewhat challenging or more challenging text and analyze its development over the course of the text, including its relationship to the characters, setting, and plot • Provide an objective summary of a somewhat challenging or more challenging text Relationships Identify and infer sequences, comparative relationships (comparisons and contrasts), and cause-effect relationships in texts. • Analyze how a text makes connections among and distinctions between individuals, ideas, or events • Analyze how particular lines of dialogue or incidents in a story cause other story events, reveal traits of a character, or influence a decision Craft and Structure Word Meanings & Word Choice Determine the meaning of words and phrases and to analyze word choice rhetorically. • Determine the meaning of words and phrases as they are used in a text, including figurative, connotative, and technical meanings • Analyze the impact of specific word choices on meaning and tone, including analogies or allusions to other texts Text Structure Analyze text structure rhetorically at the whole-text and partial-text levels. • Analyze the structure of a specific paragraph in a text, including the role of particular sentences in developing and clarifying a key concept • Compare and contrast the structure of two texts and analyze how the differing structure of each text contributes to its meaning and style Purpose & Point of View Understand the purpose or purposes for which texts are written, understand point of view, evaluate the choice of narration in literary texts, and analyze bias, stereotypes, and persuasive techniques in texts. • Determine an author’s point of view or purpose in a text • Analyze how the author acknowledges and responds to conflicting evidence or viewpoints • Analyze how differences in the points of view of the characters create such effects as suspense or humor

F.11 Appendices

Grade 8 (continued) Integration of Knowledge and Ideas Arguments Analyze arguments for their claims, counterclaims, reasons, evidence, and soundness of reasoning. • Evaluate the argument and specific claims in a text • Assess whether evidence to support an argument or claim is relevant and sufficient Multiple Texts Make connections between two or more texts in the same medium and format. • Analyze how two or more texts provide contrasting information and identify how the new information influences each text • Analyze the ways in which two or more texts provide archetypal characters and how the presentations of these archetypes may differ Straightforward Somewhat Challenging More Challenging STRAIGHTFORWARD texts will have SOMEWHAT CHALLENGING MORE CHALLENGING texts may have clear purpose and organization with texts will have clear purpose and multi-faceted purpose and organization explicit transitions and include a variety organization, with variety in sentence with varied and often complicated of sentence structures. Vocabulary styles and transitions. Vocabulary may sentence structures. Vocabulary may include some general academic or include some general academic or will include general academic and uncommon words and phrases. These uncommon words and phrases. These uncommon words and phrases. These texts may require inferential reading texts will require inferential reading texts may depict several abstract ideas and contain some abstract concepts and contain abstract concepts as well and concepts and require students and diverse perspectives. as experiences and perspectives that to read at interpretive and evaluative Informational texts will exhibit features may be different that those of the levels. of the field of study (e.g. science), reader. Informational texts will exhibit features including some domain-specific Informational texts will exhibit features of the general discipline (e.g. natural language and assumption of broad of the general discipline (e.g. natural science), including domain-specific content knowledge. science), including domain-specific language and assumption of discipline- Literary texts will have a single narrator language and assumption of everyday specific content knowledge. and conflicts that are straightforward and content knowledge appropriate to Literary texts may have multiple and usually external. Plots may deviate the grade assessed. narrators and well-rounded characters from chronological order with simple, Literary texts may have multiple who may experience subtle internal heavily signaled flash-backs or flash- narrators and non-linear plots, with or external conflicts. Plots may be forwards. These texts may include some shifts in point of view or time clearly non-linear or parallel and may lack non-literal language and simple literary signaled. Characters will be dynamic resolution. Characters are. These devices. Major and minor characters and experience uncomplicated internal texts include non-literal language and may show some depth and the text or external conflicts. These texts literary devices such as symbolism or will offer basic insights into characters, may include non-literal language and irony. Themes and subject matter will people, situations, and events (e.g., literary devices such as symbolism or be challenging, offering deep insights simple motives). irony. Themes and subject matter may into people, situations and events. be challenging, offering insights into people, situations and events.

F.12 Appendices

Early High School (Grades 9-10) 3 passages: Literary + 2 Informational Average Word Count: 500, CCSS FK Range: 8.32-12.12 Key Ideas and Details Close Reading Locate and interpret details, paraphrase information and ideas accurately, and draw logical conclusions based on textual evidence. • Cite strong and thorough textual evidence from somewhat challenging, more challenging, or complex texts to support analysis of what the text says explicitly as well as inferences drawn from the text Central Ideas, Themes & Summaries Identify and infer whole-text-level central (main) ideas and themes, identify and infer central ideas of defined parts of texts, and summarize information and ideas accurately. • Determine a central idea of a somewhat challenging, more challenging, or complex text and analyze its development over the course of the text, including how it emerges and is shaped and refined by specific details • Determine a theme of a somewhat challenging, more challenging, or complex text and analyze its development over the course of the text, including how it emerges and is shaped and refined by specific details • Provide an objective summary of a somewhat challenging, more challenging, or complex text Relationships Identify and infer sequences, comparative relationships (comparisons and contrasts), and cause-effect relationships in texts. • Analyze how the author unfolds an analysis or series of ideas or events, including the order in which the points are made, how they are introduced and developed, and the connections that are drawn between them • Analyze how complex characters develop over the course of a text, interact with other characters, and advance the plot or develop the theme Craft and Structure Word Meanings & Word Choice Determine the meaning of words and phrases and to analyze word choice rhetorically. • Determine the meaning of words and phrases as they are used in a text, including figurative, connotative, and technical meanings • Analyze the cumulative impact of specific word choice on meaning and tone • Analyze how language evokes a sense of time and place • Analyze how language sets a formal or informal tone Text Structure Analyze text structure rhetorically at the whole-text and partial-text levels. • Analyze how an author’s ideas or claims are developed and refined by particular sentences or paragraphs • Analyze how an author’s choices concerning how to order events and manipulate time create such effects as mystery, tension, or surprise Purpose & Point of View Understand the purpose or purposes for which texts are written, understand point of view, evaluate the choice of narration in literary texts, and analyze bias, stereotypes, and persuasive techniques in texts. • Determine an author’s point of view and analyze how an author’s use of language advances that point of view • Determine an author’s purpose and analyze how an author’s use of language advances that point of view • Analyze how a cultural experience is reflected in a work of literature

F.13 Appendices

Early High School (Grades 9-10) (continued) Integration of Knowledge and Ideas Arguments Analyze arguments for their claims, counterclaims, reasons, evidence, and soundness of reasoning. • Evaluate the argument and specific claims in a text, assessing whether the evidence is relevant and sufficient • Identify false statements and fallacious reasoning Multiple Texts Make connections between two or more texts in the same medium and format. • Compare the themes and concepts of two texts of historical and/or literary significance Somewhat Challenging More Challenging Complex SOMEWHAT CHALLENGING texts will MORE CHALLENGING texts may have COMPLEX texts may have complex have clear purpose and organization, multi-faceted purpose and organization structure and purpose that might with variety in sentence styles and with varied and often complicated not be evident upon first reading. transitions. Vocabulary may include sentence structures. Vocabulary Vocabulary and syntax will be some general academic or uncommon will include general academic and sophisticated and may include words and phrases. These texts uncommon words and phrases. These words or structures uncommon will require inferential reading and texts may depict several abstract ideas in contemporary texts. Numerous contain abstract concepts as well as and concepts and require students abstract ideas and concepts will experiences and perspectives that may to read at interpretive and evaluative require students to read at interpretive be different that those of the reader. levels. and evaluative levels. In these texts, Informational texts will exhibit features Informational texts will exhibit features students may encounter experiences, of the general discipline (e.g. natural of the general discipline (e.g. natural values, and perspectives different than science), including domain-specific science), including domain-specific their own. language and assumption of everyday language and assumption of discipline- Informational texts will have high and content knowledge appropriate to specific content knowledge. concept density and consistently use the grade assessed. Literary texts may have multiple domain-specific and general academic Literary texts may have multiple narrators and well-rounded characters vocabulary. narrators and non-linear plots, with who may experience subtle internal Literary texts may have multiple and/ shifts in point of view or time clearly or external conflicts. Plots may be or unreliable narrators and well- signaled. Characters will be dynamic non-linear or parallel and may lack rounded characters with complex and experience uncomplicated internal resolution. Characters are. These conflicts that lack easy resolution. or external conflicts. These texts texts include non-literal language and These texts may use unconventional may include non-literal language and literary devices such as symbolism or language structures (e.g. stream of literary devices such as symbolism or irony. Themes and subject matter will consciousness) and challenging literary irony. Themes and subject matter may be challenging, offering deep insights devices (e.g. extended metaphors, be challenging, offering insights into into people, situations and events. satire, parody). Themes and subject people, situations and events. matter may be mature, offering profound insights or philosophical reflection.

F.14 Appendices

ACT Aspire Writing – Reflective Narrative Rubric Grade 3 Reflective Narrative Development Organization Language Use Score: 5 The response engages The narrative is The response The response demonstrates with the task, and capably developed exhibits a purposeful the ability to capably convey Responses presents a capable through purposeful organizational meaning with clarity. Word at this reflective narrative. conveyance of action, structure, with some choice is usually precise. scorepoint The narrative conveys sensory details, and/or logical progression Sentence structures are demonstrate the significance of character. Reflection within the story. clear and often varied. Voice capable skill the event through on experience Transitions within the and tone are appropriate for in writing a thoughtful reflection and meaning is response clarify the the narrative purpose and reflective on the experience and supported through relationships among are maintained throughout narrative. on the meaning of the apt description and/ elements of the most of the response. While experience. There is or explanation. Details reflective narrative. errors in grammar, usage, purposeful movement enhance the story and mechanics may be between specific and and help to convey its present, they do not impede generalized ideas. significance. understanding. Score: 4 The response is The narrative is The response exhibits The response demonstrates appropriate to the adequately developed a clear organizational the ability to clearly convey Responses task, and presents an through conveyance structure, with a meaning. Word choice at this adequate reflective of action, sensory discernible logic to is sometimes precise. scorepoint narrative. The narrative details, and/or the story. Transitions Sentence structures are demonstrate demonstrates character. Reflection within the response occasionally varied and adequate recognition of the on experience and/ clarify relationships usually clear. Voice and skill in writing significance of or meaning is mostly among the elements tone are appropriate for the a reflective the event through supported through of the reflective narrative purpose, but may narrative. reflection on the description and narrative. be inconsistently maintained. experience and/ explanation. Details While errors in grammar, or on the meaning may enhance the story usage, and mechanics are of the experience. and help to convey its present, they rarely impede Connections significance. understanding. between specific and generalized ideas are mostly clear. Score: 3 The response The narrative is Organization is The response demonstrates demonstrates a limited somewhat developed. somewhat appropriate some ability to convey Responses understanding of the There is some to the task, but may meaning. Word choice is at this task, and presents a conveyance of action, be simplistic or may general and occasionally scorepoint somewhat appropriate sensory details, and/ digress at times. imprecise. Sentence demonstrate reflective narrative. or character, but it Transitions within the structures show little variety some Reflection on the may be limited or only response sometimes and are inconsistently developing experience or on somewhat relevant. clarify relationships clear. Voice and tone are skill in writing the meaning of the Reflection on the among the elements somewhat appropriate a reflective experience is limited experience and/or of the reflective for the narrative purpose narrative. or only somewhat meaning is somewhat narrative. but are inconsistently relevant. Specific and supported through maintained. Distracting generalized ideas description and errors in grammar, usage, are only somewhat explanation. and mechanics are present, connected. and they sometimes impede understanding.

F.15 Appendices

ACT Aspire Writing – Reflective Narrative Rubric Grade 3 (continued) Reflective Narrative Development Organization Language Use Score: 2 The response Development is weak. Organization is The response demonstrates demonstrates Elements of the rudimentary. The logic a weak ability to convey Responses a rudimentary story are reported of the story may be meaning. Word choice is at this understanding of the rather than described. unclear. Transitions rudimentary and vague. scorepoint task, with weak or Reflection on the within the response Sentence structures demonstrate inconsistent skill in experience and/ are often misleading are often unclear. Voice weak or generating a reflective or meaning through or poorly formed. and tone may not be inconsistent narrative. Reflection description or appropriate for the narrative skill in writing on the experience explanation is weak, purpose. Distracting errors a reflective or on the meaning inconsistent, or not in grammar, usage, and narrative. of the experience is clearly relevant. mechanics are present, and unclear or incomplete, they impede understanding. or may be irrelevant. If present, connections between specific and generalized ideas are weak or inconsistent. Score: 1 The response The response is The response shows The response demonstrates demonstrates little or virtually undeveloped, virtually no evidence little or no ability to convey Responses no understanding of with little or no action, of organization. meaning. Word choice is at this the task, with virtually sensory detail, or Transitional devices imprecise, making ideas scorepoint no narrative, and/or character, and little or may be present, but difficult to comprehend. demonstrate virtually no reflection no reflection. they fail to relate Sentence structures are little or no on the experience or elements of the mostly unclear. Voice and skill in writing its meaning. reflective narrative. tone are not appropriate a reflective for the narrative purpose. narrative. Errors in grammar, usage, and mechanics are pervasive and significantly impede understanding. Unscorable The response is blank, voided, off-topic, illegible, or not written in English.

F.16 Appendices

ACT Aspire Writing – Analytical Expository Rubric Grade 4 Analysis Development Organization Language Use Score: 5 The response engages Ideas are capably The response The response demonstrates with the task, and explained, with exhibits a purposeful the ability to capably convey Responses presents a thoughtful purposeful use of organizational strategy. meaning with clarity. Word at this analysis that discusses supporting reasons A logical sequencing choice is usually precise. score point implications and and/or detailed of ideas contributes to Sentence structures are demonstrate complications of the examples. The writer’s the effectiveness of clear and often varied. Voice capable skill subject. There is claims and specific the writer’s analysis. and tone are appropriate for in writing an purposeful movement support are sometimes Transitions between the analytical purpose and analytical between specific integrated. and within paragraphs are maintained throughout essay. details and generalized consistently clarify the most of the response. While ideas. relationships among errors in grammar, usage, ideas. and mechanics may be present, they do not impede understanding.

Score: 4 The response Ideas are adequately The response The response demonstrates is appropriate explained, with exhibits a clear but the ability to clearly convey Responses to the task, and satisfactory use of simple organizational meaning. Word choice at this presents an analysis supporting reasons structure. Ideas are is sometimes precise. score point that recognizes and/or examples. logically grouped. Sentence structures are demonstrate implications and Transitions between occasionally varied and adequate complications of the and within paragraphs usually clear. Voice and skill in writing subject. There is clear clarify the relationships tone are appropriate for the an analytical movement between among ideas. analytical purpose, but may essay. specific details and be inconsistently maintained. generalized ideas. While errors in grammar, usage, and mechanics are present, they rarely impede understanding. Score: 3 The response is Explanations of The response exhibits The response demonstrates somewhat appropriate ideas are limited, but some evidence some developing ability Responses to the task, with include some use of of organizational to convey meaning. Word at this an analysis that is supporting reasons structure. Some ideas choice is general and score point oversimplified or and/or relevant are logically grouped. occasionally imprecise. demonstrate imprecise. Implications examples. Transitions between Sentence structures some or complications are and within paragraphs show little variety and developing only somewhat clear sometimes clarify the are sometimes unclear. skill in writing or relevant. Specific relationships among Voice and tone are an analytical details and generalized ideas. somewhat appropriate essay. ideas are somewhat for the analytical purpose connected. but are inconsistently maintained. Distracting errors in grammar, usage, and mechanics are present, and they sometimes impede understanding.

F.17 Appendices

ACT Aspire Writing – Analytical Expository Rubric Grade 4 (continued) Analysis Development Organization Language Use Score: 2 The response Explanations of The response exhibits The response demonstrates demonstrates ideas are unclear or only a little evidence a weak ability to convey Responses a rudimentary incomplete, with little of organizational meaning. Word choice is at this understanding of the use of supporting structure. Logical rudimentary and frequently score point task, with weak or reasons or examples. grouping of ideas imprecise. Sentence demonstrate inconsistent skill in is inconsistent or structures are often unclear. weak or presenting an analysis. unclear. Transitions Voice and tone may not be inconsistent Implications or between and within appropriate for the analytical skill in writing complications are not paragraphs are often purpose. Distracting errors an analytical clearly relevant. Any missing, misleading, or in grammar, usage, and essay. connections between poorly formed. mechanics are present, and specific details and they impede understanding. generalized ideas are unclear or incomplete. Score: 1 The response Ideas lack explanation, The response The response demonstrates demonstrates little or with virtually no use of exhibits no evidence little or no ability to convey Responses no understanding of supporting reasons or of organizational meaning. Word choice is at this the task. The response relevant examples. structure. Ideas are imprecise and difficult to score point lacks connections not logically grouped. comprehend. Voice and tone demonstrate between specific Transitional devices are not appropriate for the little or no details and generalized may be present, but analytical purpose. Sentence skill in writing ideas. they fail to relate structures are mostly an analytical ideas. unclear. Errors in grammar, essay. usage, and mechanics are pervasive and significantly impede understanding. Unscorable The response is blank, voided, off-topic, illegible, or not written in English.

F.18 Appendices

ACT Aspire Writing – Persuasive Argument Rubric Grade 5 Argument Development Organization Language Use Score: 5 The response engages Ideas are capably The response The response demonstrates with the task, and explained and exhibits a purposeful the ability to capably convey Responses presents a thoughtful supported, with organizational strategy. meaning. Word choice is at this argument driven by apt purposeful use of A logical sequencing usually precise. Sentence scorepoint reasons. The response reasoning and/or of ideas contributes to structures are clear and demonstrate addresses implications, detailed examples. the effectiveness of often varied. Voice and capable skill complications, and/ The writer’s claims and the writer’s argument. tone are appropriate for the in writing a or counterarguments. specific support are Transitions between persuasive purpose and persuasive There is purposeful sometimes integrated. and within paragraphs are maintained throughout argumentative movement between clarify the relationships most of the response. While essay. specific and among ideas. errors in grammar, usage, generalized ideas. and mechanics may be present, they do not impede understanding. Score: 4 The response is Ideas are adequately The response The response demonstrates appropriate to the explained and exhibits a clear but the ability to clearly convey Responses task, and presents supported, with simple organizational meaning. Word choice at this a clear argument, satisfactory use of structure. Ideas are is sometimes precise. scorepoint with satisfactory reasoning and/or logically grouped. Sentence structures are demonstrate reasons for the detailed examples. Transitions between occasionally varied and adequate skill position. The response The writer’s claims and and within paragraphs usually clear. Voice and in writing a demonstrates specific support may clarify the relationships tone are appropriate for the persuasive recognition of be integrated. among ideas. persuasive purpose, but may argumentative implications, be inconsistently maintained. essay complications, and/ While errors in grammar, or counterarguments. usage, and mechanics are There is some present, they rarely impede movement between understanding. specific and generalized ideas. Score: 3 The response is Explanation and The response exhibits The response demonstrates somewhat appropriate support of ideas are some evidence some developing ability Responses to the task, and limited, but include of organizational to convey meaning. Word at this presents a somewhat some use of reasoning structure. Some ideas choice is general and scorepoint clear argument with a and/or examples. are logically grouped. occasionally imprecise. demonstrate vague or oversimplified Transitions between Sentence structures some position. Reasons and within paragraphs show little variety and developing for the position are sometimes clarify the are sometimes unclear. skill in writing somewhat appropriate relationships among Voice and tone are a persuasive and/or somewhat ideas. somewhat appropriate for argumentative relevant. Implications, the persuasive purpose essay. complications, and but are inconsistently counterarguments are maintained. Distracting oversimplified or not errors in grammar, usage, clearly relevant to the and mechanics are present, purpose. Specific and and they sometimes impede generalized ideas may understanding. be only somewhat connected.

F.19 Appendices

ACT Aspire Writing – Persuasive Argument Rubric Grade 5 (continued) Argument Development Organization Language Use Score: 2 The response Explanation and The response exhibits The response demonstrates demonstrates support of ideas are only a little evidence a weak ability to convey Responses a rudimentary unclear or incomplete, of organizational meaning. Word choice is at this understanding of the with little use of structure. Few ideas rudimentary and frequently scorepoint task. The position may reasoning and/or are logically grouped. imprecise. Sentence demonstrate be unclear. Reasons examples. Transitions between structures are often weak or for the position are and within paragraphs unclear. Voice and tone inconsistent unclear, incomplete, or are often missing, may not be appropriate for skill in writing not clearly relevant. If poorly formed, or the persuasive purpose. a persuasive present, implications, misleading. Distracting errors in argumentative complications, or grammar, usage, and essay. counterarguments are mechanics are present, and weak. Any connections they impede understanding between specific and generalized ideas are unclear, incomplete, or irrelevant.. Score: 1 The response Ideas lack explanation The response The response demonstrates demonstrates little or and support, with exhibits no evidence little or no ability to convey Responses no understanding of virtually no use of of organizational meaning. Word choice is at this the task. If a position reasoning or examples. structure. Ideas are imprecise and difficult to scorepoint is taken, there are not logically grouped. comprehend. Voice and demonstrate virtually no reasons for Transitional devices tone are not appropriate little or no the position. may be present, but for the persuasive purpose. skill in writing they fail to relate Sentence structures are a persuasive ideas. mostly unclear. Errors argumentative in grammar, usage, and essay. mechanics are pervasive and significantly impede understanding. Unscorable The response is blank, voided, off-topic, illegible, or not written in English.

F.20 Appendices

ACT Aspire Writing – Reflective Narrative Rubric Grade 6 Reflective Narrative Development Organization Language Use Score: 6 The response The narrative is The response exhibits The response demonstrates critically engages effectively developed a skillful organizational the ability to effectively Responses with the task, and through skillful strategy, with logical convey meaning with clarity. at this presents an effective conveyance of action, progression within Word choice is precise. scorepoint reflective narrative. sensory details, and/or the story. Transitions Sentence structures are demonstrate The narrative conveys character. Reflection within the response varied and clear. Voice and effective skill the significance of on experience and consistently clarify the tone are appropriate for in writing a the event through meaning is well relationships among the narrative purpose and reflective insightful reflection on supported through the elements of the are maintained throughout narrative. the experience and effective description reflective narrative. the response. While a few on the meaning of and/or explanation. errors in grammar, usage, the experience. There Details are integral and mechanics may be is skillful movement to the story and its present, they do not impede between specific and significance. understanding. generalized ideas. Score: 5 The response engages The narrative is The response The response demonstrates with the task, and capably developed exhibits a purposeful the ability to capably convey Responses presents a capable through purposeful organizational meaning with clarity. Word at this reflective narrative. conveyance of action, structure, with some choice is usually precise. scorepoint The narrative conveys sensory details, and/or logical progression Sentence structures are demonstrate the significance of character. Reflection within the story. clear and often varied. Voice capable skill the event through on experience Transitions within the and tone are appropriate for in writing a thoughtful reflection and meaning is response clarify the the narrative purpose and reflective on the experience and supported through relationships among are maintained throughout narrative. on the meaning of the apt description and/ elements of the most of the response. While experience. There is or explanation. Details reflective narrative. errors in grammar, usage, purposeful movement enhance the story and mechanics may be between specific and and help to convey its present, they do not impede generalized ideas. significance. understanding. Score: 4 The response is The narrative is The response exhibits The response demonstrates appropriate to the adequately developed a clear organizational the ability to clearly convey Responses task, and presents an through conveyance structure, with a meaning. Word choice at this adequate reflective of action, sensory discernable logic to is sometimes precise. scorepoint narrative. The narrative details, and/or the story. Transitions Sentence structures are demonstrate demonstrates character. Reflection within the response occasionally varied and adequate recognition of the on experience and/ clarify relationships usually clear. Voice and skill in writing significance of or meaning is mostly among the elements tone are appropriate for the a reflective the event through supported through of the reflective narrative purpose, but may narrative. reflection on the description and narrative. be inconsistently maintained. experience and/ explanation. Details While errors in grammar, or on the meaning may enhance the story usage, and mechanics are of the experience. and help to convey its present, they rarely impede Connections significance. understanding. between specific and generalized ideas are mostly clear.

F.21 Appendices

ACT Aspire Writing – Reflective Narrative Rubric Grade 6 (continued) Reflective Narrative Development Organization Language Use Score: 3 The response The narrative is Organization is The response demonstrates demonstrates a limited somewhat developed. somewhat appropriate some ability to convey Responses understanding of the There is some to the task, but may meaning. Word choice is at this task, and presents a conveyance of action, be simplistic or may general and occasionally scorepoint somewhat appropriate sensory details, and/ digress at times. imprecise. Sentence demonstrate reflective narrative. or character, but it Transitions within the structures show little variety some Reflection on the may be limited or only response sometimes and are inconsistently developing experience or on somewhat relevant. clarify relationships clear. Voice and tone are skill in writing the meaning of the Reflection on the among the elements somewhat appropriate a reflective experience is limited experience and/or of the reflective for the narrative purpose narrative. or only somewhat meaning is somewhat narrative. but are inconsistently relevant. Specific and supported through maintained. Distracting generalized ideas description and errors in grammar, usage, are only somewhat explanation. and mechanics are present, connected. and they sometimes impede understanding. Score: 2 The response Development is weak. Organization is The response demonstrates demonstrates Elements of the rudimentary. The logic a weak ability to convey Responses a rudimentary story are reported of the story may be meaning. Word choice is at this understanding of the rather than described. unclear. Transitions rudimentary and vague. scorepoint task, with weak or Reflection on the within the response Sentence structures demonstrate inconsistent skill in experience and/ are often misleading are often unclear. Voice weak or generating a reflective or meaning through or poorly formed. and tone may not be inconsistent narrative. Reflection description or appropriate for the narrative skill in writing on the experience explanation is weak, purpose. Distracting errors a reflective or on the meaning inconsistent, or not in grammar, usage, and narrative. of the experience is clearly relevant. mechanics are present, and unclear or incomplete, they impede understanding. or may be irrelevant. If present, connections between specific and generalized ideas are weak or inconsistent. Score: 1 The response The response is The response shows The response demonstrates demonstrates little or virtually undeveloped, virtually no evidence little or no ability to convey Responses no understanding of with little or no action, of organization. meaning. Word choice is at this the task, with virtually sensory detail, or Transitional devices imprecise, making ideas scorepoint no narrative, and/or character, and little or may be present, but difficult to comprehend. demonstrate virtually no reflection no reflection. they fail to relate Sentence structures are little or no on the experience or elements of the mostly unclear. Voice and skill in writing its meaning. reflective narrative. tone are not appropriate a reflective for the narrative purpose. narrative. Errors in grammar, usage, and mechanics are pervasive and significantly impede understanding. Unscorable The response is blank, voided, off-topic, illegible, or not written in English.

F.22 Appendices

ACT Aspire Writing – Analytical Expository Rubric Grade 7 Analysis Development Organization Language Use Score: 6 The response critically Ideas are thoroughly The response exhibits The response demonstrates engages with the explained, with skillful a skillful organizational the ability to effectively Responses task, and presents a use of supporting strategy. A logical convey meaning with clarity. at this complex analysis that reasons and/or progression of Word choice is precise. scorepoint addresses implications detailed examples. ideas increases the Sentence structures are demonstrate and complications The writer’s claims and effectiveness of the varied and clear. Voice and effective skill of the subject. There specific support are writer’s analysis. tone are appropriate for in writing an is skillful movement well integrated. Transitions between the analytical purpose and analytical between specific and within paragraphs are maintained throughout essay. details and generalized strengthen the the response. While a few ideas. relationships among errors in grammar, usage, ideas. and mechanics may be present, they do not impede understanding. Score: 5 The response engages Ideas are capably The response The response demonstrates with the task, and explained, with exhibits a purposeful the ability to capably convey Responses presents a thoughtful purposeful use of organizational strategy. meaning with clarity. Word at this analysis that discusses supporting reasons A logical sequencing choice is usually precise. scorepoint implications and and/or detailed of ideas contributes to Sentence structures are demonstrate complications of the examples. The writer’s the effectiveness of clear and often varied. Voice capable skill subject. There is claims and specific the writer’s analysis. and tone are appropriate for in writing an purposeful movement support are sometimes Transitions between the analytical purpose and analytical between specific integrated. and within paragraphs are maintained throughout essay. details and generalized consistently clarify the most of the response. While ideas. relationships among errors in grammar, usage, ideas. and mechanics may be present, they do not impede understanding. Score: 4 The response Ideas are adequately The response The response demonstrates is appropriate explained, with exhibits a clear but the ability to clearly convey Responses to the task, and satisfactory use of simple organizational meaning. Word choice at this presents an analysis supporting reasons structure. Ideas are is sometimes precise. scorepoint that recognizes and/or examples. logically grouped. Sentence structures are demonstrate implications and Transitions between occasionally varied and adequate complications of the and within paragraphs usually clear. Voice and skill in writing subject. There is clear clarify the relationships tone are appropriate for the an analytical movement between among ideas. analytical purpose, but may essay. specific details and be inconsistently maintained. generalized ideas. While errors in grammar, usage, and mechanics are present, they rarely impede understanding.

F.23 Appendices

ACT Aspire Writing – Analytical Expository Rubric Grade 7 (continued) Analysis Development Organization Language Use Score: 3 The response is Explanations of The response exhibits The response demonstrates somewhat appropriate ideas are limited, but some evidence some developing ability Responses to the task, with include some use of of organizational to convey meaning. Word at this an analysis that is supporting reasons structure. Some ideas choice is general and scorepoint oversimplified or and/or relevant are logically grouped. occasionally imprecise. demonstrate imprecise. Implications examples. Transitions between Sentence structures some or complications are and within paragraphs show little variety and developing only somewhat clear sometimes clarify the are sometimes unclear. skill in writing or relevant. Specific relationships among Voice and tone are an analytical details and generalized ideas. somewhat appropriate essay. ideas are somewhat for the analytical purpose connected. but are inconsistently maintained. Distracting errors in grammar, usage, and mechanics are present, and they sometimes impede understanding. Score: 2 The response Explanations of The response exhibits The response demonstrates demonstrates ideas are unclear or only a little evidence a weak ability to convey Responses a rudimentary incomplete, with little of organizational meaning. Word choice is at this understanding of the use of supporting structure. Logical rudimentary and frequently scorepoint task, with weak or reasons or examples. grouping of ideas imprecise. Sentence demonstrate inconsistent skill in is inconsistent or structures are often unclear. weak or presenting an analysis. unclear. Transitions Voice and tone may not be inconsistent Implications or between and within appropriate for the analytical skill in writing complications are not paragraphs are often purpose. Distracting errors an analytical clearly relevant. Any missing, misleading, or in grammar, usage, and essay. connections between poorly formed. mechanics are present, and specific details and they impede understanding generalized ideas are unclear or incomplete. Score: 1 The response Ideas lack explanation, The response The response demonstrates demonstrates little or with virtually no use of exhibits no evidence little or no ability to convey Responses no understanding of supporting reasons or of organizational meaning. Word choice is at this the task. The response relevant examples. structure. Ideas are imprecise and difficult to scorepoint lacks connections not logically grouped. comprehend. Voice and tone demonstrate between specific Transitional devices are not appropriate for the little or no details and generalized may be present, but analytical purpose. Sentence skill in writing ideas. they fail to relate structures are mostly an analytical ideas. unclear. Errors in grammar, essay. usage, and mechanics are pervasive and significantly impede understanding. Unscorable The response is blank, voided, off-topic, illegible, or not written in English.

F.24 Appendices

ACT Aspire Writing – Persuasive Argument Rubric Grade 8 Analysis Development Organization Language Use Score: 6 The response critically Ideas are effectively The response exhibits The response demonstrates engages with the task, explained and a skillful organizational the ability to effectively Responses and presents a skillful supported, with skillful strategy. A logical convey meaning with clarity. at this argument driven by use of reasoning and/ progression of Word choice is precise. scorepoint insightful reasons. or detailed examples. ideas increases the Sentence structures are demonstrate The response critically The writer’s claims and effectiveness of the varied and clear. Voice and effective skill addresses implications, specific support are writer’s argument. tone are appropriate for the in writing a complications, and/ well integrated. Transitions between persuasive purpose and persuasive or counterarguments. and within paragraphs are maintained throughout argumentative There is skillful strengthen the the response. While a few essay. movement between relationships among errors in grammar, usage, specific and ideas. and mechanics may be generalized ideas. present, they do not impede understanding. Score: 5 The response engages Ideas are capably The response The response demonstrates with the task, and explained and exhibits a purposeful the ability to capably convey Responses presents a thoughtful supported, with organizational strategy. meaning. Word choice is at this argument driven by apt purposeful use of A logical sequencing usually precise. Sentence scorepoint reasons. The response reasoning and/or of ideas contributes to structures are clear and demonstrate addresses implications, detailed examples. the effectiveness of often varied. Voice and capable skill complications, and/ The writer’s claims and the writer’s argument. tone are appropriate for the in writing a or counterarguments. specific support are Transitions between persuasive purpose and persuasive There is purposeful sometimes integrated. and within paragraphs are maintained throughout argumentative movement between clarify the relationships most of the response. While essay. specific and among ideas. errors in grammar, usage, generalized ideas. and mechanics may be present, they do not impede understanding. Score: 4 The response is Ideas are adequately The response The response demonstrates appropriate to the explained and exhibits a clear but the ability to clearly convey Responses task, and presents supported, with simple organizational meaning. Word choice at this a clear argument, satisfactory use of structure. Ideas are is sometimes precise. scorepoint with satisfactory reasoning and/or logically grouped. Sentence structures are demonstrate reasons for the detailed examples. Transitions between occasionally varied and adequate skill position. The response The writer’s claims and and within paragraphs usually clear. Voice and in writing a demonstrates specific support may clarify the relationships tone are appropriate for the persuasive recognition of be integrated. among ideas. persuasive purpose, but may argumentative implications, be inconsistently maintained. essay. complications, and/ While errors in grammar, or counterarguments. usage, and mechanics are There is some present, they rarely impede movement between understanding. specific and generalized ideas.

F.25 Appendices

ACT Aspire Writing – Persuasive Argument Rubric Grade 8 (continued) Analysis Development Organization Language Use Score: 3 The response is Explanation and The response exhibits The response demonstrates somewhat appropriate support of ideas are some evidence some developing ability Responses to the task, and limited, but include of organizational to convey meaning. Word at this presents a somewhat some use of reasoning structure. Some ideas choice is general and scorepoint clear argument with a and/or examples. are logically grouped. occasionally imprecise. demonstrate vague or oversimplified Transitions between Sentence structures some position. Reasons and within paragraphs show little variety and developing for the position are sometimes clarify the are sometimes unclear. skill in writing somewhat appropriate relationships among Voice and tone are a persuasive and/or somewhat ideas. somewhat appropriate for argumentative relevant. Implications, the persuasive purpose essay. complications, and but are inconsistently counterarguments are maintained. Distracting oversimplified or not errors in grammar, usage, clearly relevant to the and mechanics are present, purpose. Specific and and they sometimes impede generalized ideas may understanding. be only somewhat connected. Score: 2 The response Explanation and The response exhibits The response demonstrates demonstrates support of ideas are only a little evidence a weak ability to convey Responses a rudimentary unclear or incomplete, of organizational meaning. Word choice is at this understanding of the with little use of structure. Few ideas rudimentary and frequently scorepoint task. The position may reasoning and/or are logically grouped. imprecise. Sentence demonstrate be unclear. Reasons examples. Transitions between structures are often weak or for the position are and within paragraphs unclear. Voice and tone inconsistent unclear, incomplete, or are often missing, may not be appropriate for skill in writing not clearly relevant. If poorly formed, or the persuasive purpose. a persuasive present, implications, misleading. Distracting errors in argumentative complications, or grammar, usage, and essay. counterarguments are mechanics are present, and weak. Any connections they impede understanding between specific and generalized ideas are unclear, incomplete, or irrelevant. Score: 1 The response Ideas lack explanation The response The response demonstrates demonstrates little or and support, with exhibits no evidence little or no ability to convey Responses no understanding of virtually no use of of organizational meaning. Word choice is at this the task. If a position reasoning or examples. structure. Ideas are imprecise and difficult to scorepoint is taken, there are not logically grouped. comprehend. Voice and demonstrate virtually no reasons for Transitional devices tone are not appropriate little or no the position. may be present, but for the persuasive purpose. skill in writing they fail to relate Sentence structures are a persuasive ideas. mostly unclear. Errors argumentative in grammar, usage, and essay. mechanics are pervasive and significantly impede understanding. Unscorable The response is blank, voided, off-topic, illegible, or not written in English.

F.26 Appendices

ACT Aspire Writing – Analytical Expository Rubric Early High School Analysis Development Organization Language Use Score: 6 The response critically Ideas are thoroughly The response exhibits The response demonstrates engages with the explained, with skillful a skillful organizational the ability to effectively Responses task, and presents a use of supporting strategy. A logical convey meaning with clarity. at this complex analysis that reasons and/or progression of Word choice is precise. scorepoint addresses implications detailed examples. ideas increases the Sentence structures are demonstrate and complications The writer’s claims and effectiveness of the varied and clear. Voice and effective skill of the subject. There specific support are writer’s analysis. tone are appropriate for in writing an is skillful movement well integrated. Transitions between the analytical purpose and analytical between specific and within paragraphs are maintained throughout essay. details and generalized strengthen the the response. While a few ideas. relationships among errors in grammar, usage, ideas. and mechanics may be present, they do not impede understanding. Score: 5 The response engages Ideas are capably The response The response demonstrates with the task, and explained, with exhibits a purposeful the ability to capably convey Responses presents a thoughtful purposeful use of organizational strategy. meaning with clarity. Word at this analysis that discusses supporting reasons A logical sequencing choice is usually precise. scorepoint implications and and/or detailed of ideas contributes to Sentence structures are demonstrate complications of the examples. The writer’s the effectiveness of clear and often varied. Voice capable skill subject. There is claims and specific the writer’s analysis. and tone are appropriate for in writing an purposeful movement support are sometimes Transitions between the analytical purpose and analytical between specific integrated. and within paragraphs are maintained throughout essay. details and generalized consistently clarify the most of the response. While ideas. relationships among errors in grammar, usage, ideas. and mechanics may be present, they do not impede understanding. Score: 4 The response Ideas are adequately The response The response demonstrates is appropriate explained, with exhibits a clear but the ability to clearly convey Responses to the task, and satisfactory use of simple organizational meaning. Word choice at this presents an analysis supporting reasons structure. Ideas are is sometimes precise. scorepoint that recognizes and/or examples. logically grouped. Sentence structures are demonstrate implications and Transitions between occasionally varied and adequate complications of the and within paragraphs usually clear. Voice and skill in writing subject. There is clear clarify the relationships tone are appropriate for the an analytical movement between among ideas. analytical purpose, but may essay. specific details and be inconsistently maintained. generalized ideas. While errors in grammar, usage, and mechanics are present, they rarely impede understanding.

F.27 Appendices

ACT Aspire Writing –Analytical Expository Rubric Early High School (continued) Analysis Development Organization Language Use Score: 3 The response is Explanations of The response exhibits The response demonstrates somewhat appropriate ideas are limited, but some evidence some developing ability Responses to the task, with include some use of of organizational to convey meaning. Word at this an analysis that is supporting reasons structure. Some ideas choice is general and scorepoint oversimplified or and/or relevant are logically grouped. occasionally imprecise. demonstrate imprecise. Implications examples. Transitions between Sentence structures some or complications are and within paragraphs show little variety and developing only somewhat clear sometimes clarify the are sometimes unclear. skill in writing or relevant. Specific relationships among Voice and tone are an analytical details and generalized ideas. somewhat appropriate essay. ideas are somewhat for the analytical purpose connected. but are inconsistently maintained. Distracting errors in grammar, usage, and mechanics are present, and they sometimes impede understanding. Score: 2 The response Explanations of The response exhibits The response demonstrates demonstrates ideas are unclear or only a little evidence a weak ability to convey Responses a rudimentary incomplete, with little of organizational meaning. Word choice is at this understanding of the use of supporting structure. Logical rudimentary and frequently scorepoint task, with weak or reasons or examples. grouping of ideas imprecise. Sentence demonstrate inconsistent skill in is inconsistent or structures are often unclear. weak or presenting an analysis. unclear. Transitions Voice and tone may not be inconsistent Implications or between and within appropriate for the analytical skill in writing complications are not paragraphs are often purpose. Distracting errors an analytical clearly relevant. Any missing, misleading, or in grammar, usage, and essay. connections between poorly formed. mechanics are present, and specific details and they impede understanding generalized ideas are unclear or incomplete. Score: 1 The response Ideas lack explanation, The response The response demonstrates demonstrates little or with virtually no use of exhibits no evidence little or no ability to convey Responses no understanding of supporting reasons or of organizational meaning. Word choice is at this the task. The response relevant examples. structure. Ideas are imprecise and difficult to scorepoint lacks connections not logically grouped. comprehend. Voice and tone demonstrate between specific Transitional devices are not appropriate for the little or no details and generalized may be present, but analytical purpose. Sentence skill in writing ideas. they fail to relate structures are mostly an analytical ideas. unclear. Errors in grammar, essay. usage, and mechanics are pervasive and significantly impede understanding. Unscorable The response is blank, voided, off-topic, illegible, or not written in English.

F.28 References

Abedi, J. (2008). Linguistic modification: Part I: Language factors in the assessment of English language learners: The theory and principles underlying the linguistic modification approach. Los Angeles: Department of Education: LEP Partnership at the Davis and the Advance Research and Data Analyses Center, University of California. Retrieved from http://ncela.ed.gov/files/uploads/11/abedi_sato.pdf

Abedi, J. (2009). Assessing high school English language learners. In L. M. Pinkus (Ed.), Meaningful measurement: The role of assessments in improving high school education in the twenty-first century (pp. 143–156). Washington, DC: Alliance for Excellent Education. Retrieved from www.ewcupdate.com/userfiles/ assessmentnetwork_net/file/MeaningfulMeasurement.pdf

Abedi, J., Courtney, M., & Leon, S. (2003b). Research-supported accommodation for English language learners in NAEP (CSE Technical Report No. 586). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing, University of California. Retrieved from http://www.cse.ucla.edu/products/Reports/ TR586.pdf

Abedi, J., Courtney, M., Leon, S., Kao, J., & Assam, T. (2006). English language learners and math achievement: A study of opportunity to learn and language accommodation (CSE Technical Report No. 702). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing, University of California. Retrieved from http://files.eric.ed.gov/fulltext/ED495848.pdf

Abedi, J., Courtney, M., Mirocha, J., Leon, S., & Goldberg, J. (2005). Language accommodations for English language learners in large-scale assessments: Bilingual dictionaries and linguistic modification (CSE Report No. 666). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing, University of California. Retrieved from http://www.cse.ucla.edu/products/reports/r666.pdf

R.1 References

Abedi, J. & Ewers, N. (2013). Accommodations for English language learners and students with disabilities: A research-based decision algorithm. Los Angeles: Smarter Balanced Assessment Consortium, University of California. Retrieved from http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/08/ Accomodations-for-under-represented-students.pdf

Abedi, J., & Herman, J. L. (2010). Assessing English language learners’ opportunity to learn mathematics: Issues and limitations. Teacher’s College Record, 112(3), 723–746.

Abedi, J., Hofstetter, C., Baker, E., & Lord, C. (2001). NAEP math performance and test accommodations: Interactions with student language background (CSE Technical Report No. 536). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing, University of California. Retrieved from http://www. cse.ucla.edu/products/reports/newtr536.pdf

Abedi, J., Hofstetter, C., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1–28.

Acosta, B., Rivera, C., & Willner, L. S. (2008). Best Practices in State Assessment Policies for Accommodating English Language Learners: A Delphi Study. Prepared for the LEP Partnership, U. S. Department of Education. Arlington, VA: The George Washington University Center for Equity and Excellence in Education. Retrieved from http://files.eric.ed.gov/fulltext/ED539759.pdf

ACT. (2006). Ready for college and ready for work: Same or different? Iowa City, IA: ACT. Retrieved from www.act.org/research/policymakers/pdf/ReadinessBrief.pdf

ACT. (2007). National Career Readiness CertificateTM: WorkKeys assessments technical bulletin. Iowa City, IA: ACT. Retrieved from okjobmatch.com/ada/ customization/Kansas/documents/NCRC_Technical_Bulletin.pdf

ACT. (2013a). The ACT Explore® technical manual: 2013–2014. Iowa City, IA: ACT.

ACT. (2013b). The ACT Plan® technical manual: 2013–2014. Iowa City, IA: ACT.

ACT. (2014a). ACT Aspire® exemplar writing questions. Iowa City, IA: ACT.

ACT. (2014b). The ACT® technical manual. Iowa City, IA: ACT.

Allen, J. (2014). Development of predicted paths for ACT Aspire score reports (ACT Working Paper 2014-06). Iowa City, IA: ACT.

Allen, J., & Sconing, J. (2005). Using ACT assessment® scores to set benchmarks for college readiness (ACT Research Report 2005-3). Iowa City, IA: ACT.

R.2 References

Almond, P., Winter, P., Cameto, R., Russell, M., Sato, E., Clarke-Midura, J., Torres, C., Haertel, G., Dolan, R., Beddow, P., & Lazarus, S. (2010). Technology-enabled and universally designed assessment: Considering access in measuring the achievement of students with disabilities—A foundation for research. Journal of Technology, Learning, and Assessment, 10(5), 1–52.

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Angoff, W. H., & Ford, S. F. (1973). Item-race interaction on a test of scholastic aptitude. Journal of Educational Measurement, 10(2), 95–106.

Arditi, A. (2016a). Making text legible: Designing for people with partial sight [Brochure]. New York, NY: Lighthouse International. Retrieved from http://li129-107.members. linode.com/accessibility/design/accessible-print-design/making-text-legible/

Arditi, A. (2016b). Effective color contrast: Designing for people with partial sight and color deficiencies [Brochure]. New York, NY: Lighthouse International. Retrieved from http://li129-107.members.linode.com/accessibility/design/accessible-print- design/effective-color-contrast/

Baker, F. B., & Kim, S. H. (2004). Item response theory: Parameter estimation techniques (2nd ed.). New York, NY: Dekker.

Beddow, P. (2009). TAMI Test Accessibility and Modification Inventory: Quantifying and improving the accessibility of tests and test items. Presented at CCSSO 2009 National Conference on Student Assessment in Los Angeles, CA.

Beddow, P. A., Elliott, S. N., & Kettler, R. J. (2013). Test accessibility: Item reviews and lessons learned from four state assessments. Education Research International, 2013(952704), 1-12. doi: 10.1155/2013/952704.

Beddow, P. A., Elliott, S. N., & Kettler, R. J. (2009). Test Accessibility and Modification Inventory (TAMI). Nashville, TN: Vanderbilt University. Retrieved from http:// peabody.vanderbilt.edu/docs/pdf/PRO/TAMI_Accessibility_Rating_Matrix.pdf

Beddow, P. A., Elliott, S. N., & Kettler, R. J. (2009). TAMI Accessibility Rating Matrix (ARM). Nashville, TN: Vanderbilt University. Retrieved from http://peabody. vanderbilt.edu/docs/pdf/PRO/TAMI_Accessibility_Rating_Matrix.pdf

Beddow, P. A., Elliott, S. N., & Kettler, R. J. (n.d.). Accessibility reviews to improve test score validity. Operational Alternate Assessments of Science Inquiry Standards OAASIS Project. Nashville, TN: Vanderbilt University. Retrieved from http://www. cehd.umn.edu/NCEO/Presentations/Posters/6VanderbiltAERA.pdf

R.3 References

Betebenner, D. W., Van Iwaarden, A., Domingue, B., & Shang, Y. (2016). SGP: Student growth percentiles & percentile growth trajectories. [R package version 1.5-0.0]. Retrieved from cran.r-project.org/web/packages/SGP/index.html

Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.

Bowles, M., & Stansfield, C. W. (2008). Standards-based assessment in the native language: A practical guide to the issues. NLA-LEP Partnership. Retrieved from www.edweek.org/media/maz-guide%20to%20native%20language%20 assessment%20v15-blog.pdf

Brennan, R. L. (2001). Generalizability theory. New York, NY: Springer-Verlag.

Burton, S. J., Sudweeks, R. R., Merrill, P., & Wood, B. (1991). How to prepare better multiple-choice test items: Guidelines for university faculty. Provo, UT: Brigham Young University Testing Services and the Department of Instructional Science. Retrieved from testing.byu.edu/handbooks/betteritems.pdf

Camilli, G. C. (2006). Test fairness. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 221-256). Westport, CT: American Council on Education and Praeger.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105.

CAST. (2011). Universal design for learning guidelines version 2.0. Wakefield, MA: Author. Retrieved from www.udlcenter.org/aboutudl/udlguidelines

Castellano, K. E., & Ho, A. D. (2015). Practical differences among aggregate-level conditional status metrics: From median student growth percentiles to value-added models. Journal of Educational and Behavioral Statistics, 40(1), 35–68.

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276.

Cawthon, S., Ho, E., Patel, P., Potvin, D., & Trundt, K. (2009). Multiple constructs and effects of accommodations on accommodated test scores for students with disabilities. Practical Assessment, Research & Evaluation, 14(18), 1–9. Retrieved from pareonline.net/pdf/v14n18.pdf

Cawthon, S., Leppo, R., Carr, T. G., & Kopriva, R. (2013). Towards accessible assessments: Promises and limitations of test item adaptations for students with disabilities and English language learners. Retrieved from http://iiassessment. wceruw.org/research/researchPapers/related/Towards_accessible_assessments. pdf

R.4 References

Christensen, L., Rogers, C., & Braam, M. (2012). Concluding external validity report of student accessibility assessment system enhanced assessment grant. Research funded by the U. S. Department of Education in support of Enhanced Assessment Grant, Student Accessibility Assessment System (SAAS), 2009-2011.

Christensen, L. L., Shyyan, V., Rogers, C. & Kincaid, A. (2014). Audio support guidelines for accessible assessments: Insights from cognitive labs. Minneapolis: National Center on Educational Outcomes, U.S. Department of Education, University of Minnesota. Retrieved from http://files.eric.ed.gov/fulltext/ED561901.pdf

Colorado Department of Education. (n.d.). Colorado accommodations guide for English learners 2013-2014. Retrieved from www.cde.state.co.us/sites/default/files/ Accommodations%20Guide%20for%20ELs%202013%202014%20final%20 V.2.pdf

Critz, M. (2010). Fonts for iPad & iPhone. Retrieved from http://www.michaelcritz. com/2010/04/02/fonts-for-ipad-iphone/

Critz, M. (n.d.). iOS fonts: A place for happy typography. Retrieved from iosfonts.com

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.

Cronbach, L. J., Schönemann, P., & McKie, D. (1965). Alpha coefficients for stratified- parallel tests. Educational and Psychological Measurement, 25(2), 291–312.

Curran, E. (2003, November 25). Correspondence regarding ACT English test format received from Eileen Curran. Topic: Underlining in English tests. Boston, MA: National Braille Press. de Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: Guilford Press.

Dolan, R. P., Burling, K., Harms, M., Strain-Seymour, E., Way, W., & Rose, D. H. (2013). A universal design for learning-based framework for designing accessible technology- enhanced assessments. (Research Report). Iowa City, IA: Pearson Education Measurement. Retrieved from http://www.cast.org/our-work/publications/2013/ assessment-computer-based-testing-students-disabilities-english-learners- universal-design-udl-dolan.html#.V0kdK03rupo

Duncan, T., Parent, L., Chen, W., Ferrara, S., Johnson, E., Oppler, S., & Shieh, Y-Y. (2005). Study of a dual-language test booklet in eighth-grade mathematics. Applied Measurement in Education, 18(2), 129–161.

Fedorchak, G. (2012). Access by Design—Implications for equity and excellence in education. Internal whitepaper prepared on behalf of the NH Department of Education for the Smarter Balanced Assessment Consortium. (Internal Document – no public link) R.5 References

Fedorchak, G., Higgins, J., Russell, M., & Thurlow, M. (2011). Increasing accessibility to assessments through computer based tools. [PowerPoint Slides]. Orlando, FL: National Conference on Student Assessment. Retrieved from: https://ccsso.confex. com/ccsso/2011/webprogram/Session2220.html

Fletcher, J., Francis, D., O’Malley, K., Copeland, K., Mehta, P., Caldwell, C., Kalinowski, S., Young, V., & Vaughn, S. (2009). Effects of a bundled accommodations package on high-stakes testing for middle school students with reading disabilities. Exceptional Children, 75(4), 447–463.

Francis, D., Rivera, M., Lesaux, N., Kieffer, M., & Rivera, H. (2006) Practical guidelines for the education of English language learner: Research-based recommendations for the use of accommodations in large-scale assessments. (Book 3 of 3). Portsmouth, NH: RMC Research Corporation, Center on Instruction. Retrieved from http://centeroninstruction.org/files/ELL3-Assessments.pdf

Freed, G., Gould, B., Hayward, J., Lally, K., Parkin, M., & Rothberg, M. (2015). Item writer guidelines for greater accessibility. Boston, MA: The Carl and Ruth Shapiro Family National Center for Accessible Media at WGBH (NCAM) Educational Foundation.

Geisinger, K. F. (1991). Using standard-setting data to establish cutoff scores. Educational Measurement: Issues and Practice, 10(2), 17–22.

Gilmer, J. S., & Feldt, L. S. (1983). Reliability estimation for a test with parts of unknown lengths. Psychometrika, 48(1), 99–111.

Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Guzman-Orth, D., Laitusis, C., Thurlow, M., & Christensen, L. (2014). Conceptualizing accessibility for English language proficiency assessments. Educational Testing Service (ETS) and National Center on Educational Outcomes (NCEO). Retrieved from www.ets.org/Media/Research/pdf/K12%20EL%20Accessibility%20Paper. pdf

Hafner, A. L. (2001, April). Evaluating the impact of test accommodations on test scores of LEP students and non-LEP students. Paper presented at the annual meeting of the American Educational Research Association in Seattle, WA. Retrieved from files.eric.ed.gov/fulltext/ED455299.pdf

Hanson, B. A. (1994). Extension of Lord-Wingersky algorithm to computing test score distributions for polytomous items. Unpublished research note.

Hatcher, L. (1994). A step-by-step approach to using the SAS® system for factor analysis and structural equation modeling (1st ed.). Cary, NC: SAS Institute Inc.

R.6 References

Higgins, J., Famularo, L., Bowman, T., & Hall, R. (2015). Research and development of audio and American Sign Language guidelines for creating accessible computer- based assessments. Dover, NH: Measured Progress. Retrieved from: https://www. measuredprogress.org/wp-content/uploads/2015/08/GAAP-Research-and- Development-of-Audio-and-American-Sign-Language-Guidelines-for-Creating- Accessible-Computer-Based-Assessments.pdf

Higgins, J., Fedorchak, G., Katz, M. (2012) Assignment of accessibility tools for digitally delivered assessments: Key findings. Dover: NH: U.S. Department of Education, Measured Progress. (Internal Report for State of New Hampshire–excerpted from the final grant report submitted to the US Department of Education, Office of Elementary and Secondary Education in support of Enhanced Assessment Grant: Examining the feasibility, effect and capacity to provide universal access through computer-based testing (2007-2009).

Holland, P. W., & Dorans, N. J. (2006). Linking and equating. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 187–220). Westport, CT: Praeger.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: Praeger.

Kieffer, M., Rivera, M., & Francis, D. (2012). Practical guidelines for the education of English language learners: Research-based recommendations for the use of accommodations in large-scale assessments research and practice recommendations. Portsmouth, NH: RMC Research Corporation, Center on Instruction. Retrieved from http://files.eric.ed.gov/fulltext/ED537635.pdf

Kingston, N. M. (2008). Comparability of computer- and paper-administered multiple- choice tests for K–12 populations: A synthesis. Applied Measurement in Education, 22(1), 22–37. doi: 10.1080/ 08957340802558326

Kitchel, E. (2013). American Printing House for the Blind. Correspondence received from Dr. Elaine Kitchel. Topic: Low Vision Project.

Kitchel, J. E. (2003). Guidelines to prepare text and graphics for enhanced format documents [PowerPoint slides]. Louisville, KY: American Printing House for the Blind (APH). Retrieved from http://slideplayer.com/slide/712571/

Kitchel, J. E. (n.d.). APH guidelines for print document design. Louisville, KY: American Printing House for the Blind (APH). Retrieved from http://www.aph.org/research/ design-guidelines/

Koenker, R. (2005). Quantile regression. New York, NY: Cambridge University Press.

R.7 References

Kolen, M. J. (1984). Effectiveness of analytic smoothing in equipercentile equating. Journal of Educational and Behavioral Statistics, 9(1), 25–44.

Kolen, M. J., & Brennan, R. L. (2014). Test equating, scaling, and linking: Methods and practices (3rd ed.). New York, NY: Springer-Verlag.

Kwon, M., Legge, G. E., & Dibbels, B. R. (2007). Developmental changes in the visual span for reading. Science Direct Vision Research, 47, 2889–2900. Retrieved from www.sciencedirect.com/science/article/pii/S0042698907003422

Laitusis, C., Buzick, H., Stone, E., Hansen, E., & Hakkinen, M. (2012). Literature review of testing accommodations and accessibility tools for students with disabilities. Princeton, NJ: Educational Testing Service, Smarter Balanced Assessment Consortium. Retrieved from https://www.smarterbalanced.org/wp-content/ uploads/2015/08/Smarter-Balanced-Students-with-Disabilities-Literature- Review.pdf

Legge, G. E. (2007). Psychophysics of reading in normal and low vision. Mahwah, NJ: Lawrence Erlbaum Associates.

Legge, G. E., & Bigelow, C. A. (2011). Does print size matter for reading? A review of findings from vision science and typography. Journal of Vision, 11(5):8, 1–22. doi:10_1167/11.5.8.

Lister, N., & Shaftel, J. (2014, June). Accessibility for Technology-Enhanced Assessments (ATEA). Presentation at the National Conference on Student Assessment in New Orleans, LA.

Lovett, B. J. (2011). Extended time testing accommodations: What does the research say? National Association of School Psychologists Communique, 39(8). Retrieved from www.questia.com/magazine/1P3-2376997951/extended-time-testing- accommodations-what-does-the

Mace, R. L. (1989). Universal design recommendations report (Project 89). Ronald L. Mace Papers, MC 00260, Special Collections Research Center, North Carolina State University Libraries, Raleigh, NC.

Mansfield, J. S., Legge, G. E., & Bane, M. C. (1996). Psychophysics of reading: XV. Font effects in normal and low vision. Investigative Ophthalmology and Visual Science, 37(8), pp. 1492-1501.

Mattern, K., Burrus, J., Camara, W., O’Connor, R., Hansen, M. A., Gambrell, J., Casillas, A., & Bobek, B. (2014). Broadening the definition of college and career readiness: A holistic approach. Iowa City, IA: ACT.

R.8 References

Mislevy, R. J., Almond, R. G., & Lukas J. (2004). A brief introduction to evidence- centered design (CSE Report 632). Los Angeles, CA: University of California-Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing. Retrieved from files.eric.ed.gov/fulltext/ED483399.pdf

Mislevy, R. J., & Haertel, G. (2006). Implications for evidence-centered design for educational assessment. Educational Measurement: Issues and Practice, 25(4), 6–20.

Muraki, E., & Bock, R. D. (2003). PARSCALE 4 for Windows: IRT based test scoring and item analysis for graded items and rating scales [Computer software]. Skokie, IL: Scientific Software International, Inc.

Muthén, L. K., & Muthén, B. O. (1998-2007). Mplus user’s guide (5th ed.). Los Angeles, CA: Muthén & Muthén.

New England Common Assessment Consortium. (2007). Supplement to the 2007 NECAP Technical Manual, Accommodations allowed in NECAP General Assessment: Are they appropriate? What impact do those accommodations have on the test scores? (Response to Federal Peer Review Item 4.0.2)

O’Brien, B. A., Mansfield, J. S., & Legge, G. E. (2005). The effect of print size on reading speed in dyslexia. Journal of research in reading, 28(3), 332-349. doi:10.1111/j.1467-9817.2005.00273.x.

Parker, C. E., Louie, J., and O’Dwyer, L. (2009). New measures of English language proficiency and their relationship to performance on large-scale content assessments (Issues & Answers Report, REL 2009-No. 066). Washington, DC: US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast and Islands. Retrieved from ies.ed.gov/ncee/edlabs/regions/northeast/pdf/ REL_2009066.pdf

Passel, J. S. (2011). Demography of Immigrant Youth: Past, Present and Future. Immigrant Children, 21(1), pp. 19–41. Retrieved from futureofchildren.org/ futureofchildren/publications/journals/journal_details/index.xml?journalid=74

Pennock-Roman, M., & Rivera, C. (2011). Mean effects of test accommodations for ELLs and non-ELLs: A meta-analysis of experimental studies. Educational Measurement: Issues and Practice, 30(3), 10–28.

Pennock-Roman, M., & Rivera, C. (2012). Summary of literature on empirical studies of the validity and effectiveness of test accommodations for ELLs: 2005–012. Prepared for Measured Progress by The George Washington University Center for Equity and Excellence in Education. Retrieved from www.smarterbalanced.org/ wp-content/uploads/2015/08/Smarter-Balanced-ELL-Literature-Review.pdf

R.9 References

Popham, W. J. (2012). Assessment Bias: How to Banish it. Booklet in series Mastering Assessment: A Self-Service System for Educators (2nd ed.). Boston, MA: Pearson Education, Inc., publishing as Allyn & Bacon. Retrieved from ati.pearson.com/ downloads/chapters/Popham_Bias_BK04.pdf

Rogers, C. M., Christian, E. M., & Thurlow, M. L. (2012). A summary of the research on the effects of test accommodations: 2009-2010 (Technical Report 65). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved from www.cehd.umn.edu/NCEO/onlinepubs/Tech65/default.htm

Rogers, J. L., Howard, K. I., & Vessey, J. T. (1993). Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin, 113(3), 553–565.

Russell, M. (2002). Testing on computers: A follow-up study comparing performance on computer and on paper. Technology and Assessment Study Collaborative (inTASC). Originally published in Educational Policy Analysis Archives, 7(20), 1999. Retrieved from www.bc.edu/research/intasc/PDF/TestCompFollowUp.pdf

Russell, M., Goldberg, A., & O’Connor, K. (2003). Computer-based testing and validity: A look back and into the future. Technology and Assessment Study Collaborative (inTASC). Chestnut Hill, MA: Boston College. Retrieved from www.bc.edu/research/ intasc/PDF/ComputerBasedValidity.pdf

Russell-Minda, E., Jutai, J., Strong, G., Campbell, K., Gold, D., Pretty, L., & Wilmot, L. (2006). An evidence-based review of the research on typeface legibility for readers with low vision. London: ON: Vision Rehabilitation Evidence-Based Review (VREBR) and The Canadian National Institute for the Blind (CNIB), University of Western Ontario.

Russell-Minda, E., Jutai, J., Strong, J. G., Campbell, K., Gold, D., Pretty, L., & Wilmot, L. (2007). The legibility of typefaces for readers with low vision: A research review. Journal of Visual Impairment & Blindness, 101(7), 402–415.

Sato, E., Rabinowitz, S., Gallagher, C., & Huang, C. W. (2010). Accommodations for English language learner students: the effect of linguistic modification of math test item sets (NCEE 2009–4079). Washington, DC: US Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from ies.ed.gov/ncee/edlabs/regions/west/pdf/ REL_20094079.pdf

Serlin, R. C., & Lapsley, D. K. (1985). Rationality in psychological research: The good- enough principle. American Psychologist, 40(1), 73–83.

Shaftel, J., Belton-Kocher, E., Glasnapp, D., & Poggio, J. (2006). The impact of language characteristics in mathematics test items on the performance of English language learners and students with disabilities. Educational Assessment, 11(2), R.10 105–126. References

Sireci, S. G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 19–37). Charlotte, NC: Information Age Publishing.

Solano-Flores, G. (2012). Translation accommodations framework for testing English language learners in mathematics. Boulder: Smarter Balanced Assessment Consortium, University of Colorado. Retrieved from https://www.smarterbalanced. org/wp-content/uploads/2015/08/Translation-Accommodations-Framework-for- Testing-ELL-Math.pdf

Thompson, S. J., Johnstone, C. J., & Thurlow, M. L. (2002). Universal design applied to large scale assessments (NCEO Synthesis Report 44). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved from www.cehd. umn.edu/nceo/onlinepubs/Synthesis44.html

Thurlow, M. L., Laitusis, C. C., Dillon, D. R., Cook, L. L., Moen, R. E., Abedi, J., & O’Brien, D. G. (2009). Accessibility principles for reading assessments. Minneapolis, MN: National Accessible Reading Assessment Projects. Retrieved from http://files.eric. ed.gov/fulltext/ED510369.pdf

US Department of Education. (2003). Fact sheet: NCLB provisions ensure flexibility and accountability for limited English proficient students. Retrieved from www2. ed.gov/nclb/accountability/schools/factsheet-english.html. Related information listed in Standards and assessments: Non-regulatory guidance, retrieved from www. ed.gov/policy/elsec/guid/saaguidance03.doc

Wang, M. W., & Stanley, J. C. (1970). Differential weighting: A review of methods and empirical studies. Review of Educational Research, 40(5), 663–705.

Wang, T., Kolen, M. J., & Harris, D. J. (2000). Psychometric properties of scale scores and performance levels for performance assessments using polytomous IRT. Journal of Educational Measurement, 37(2), 141–162.

Wells, C. S., Sireci, S. G., & Bahry, L. M. (2014). The effect of conditioning years on the reliability of SGPs (Center for Educational Assessment Research Report 869). Amherst, MA: University of Massachusetts Amherst, Center for Educational Assessment. Retrieved from www.umass.edu/remp/pdf/WSB_NCME2014.pdf

Wolf, M. K., Herman, J. L., & Dietel, R. (2010). Improving the validity of English language learner assessment systems (Policy Brief 10). Los Angeles, CA: University of California-Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing. Retrieved from www.cse.ucla.edu/products/policy/policybrief10_ frep_ellpolicy.pdf

R.11 References

Wolf, M. K., Kim, J., Kao, J. C. & Rivera, N. (2009). Examining the effectiveness and validity of glossary and read-aloud accommodations for English language learners in a math assessment. Los Angeles, CA: University of California-Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing. Retrieved from www.cde.state.co.us/sites/default/files/documents/research/download/pdf/ cresstglossaryaccomstudy09.pdf

Yen, W. M., & Fitzpatrick, A. R. (2006). Item response theory. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 111–153). Westport, CT: Praeger.

R.12 2016 Updates

Previous chapters in this technical manual include ACT Aspire test development, test administration, and psychometric analyses from the early development stage to the operational administrations. This section presents updated information and statistical analyses based on data collected from recent operational administrations and a few updates that will be implemented starting from spring 2016. Specifically, Section 1 of this chapter presents the updated nationally representative norms used for reporting from the fall 2015 operational administration; Section 2 presents raw score reliabilities of forms administered in spring 2015; and Section 3 presents a series of program improvements that will be implemented starting from the spring 2016 operational administration.

Section 1: Updating ACT Aspire Norms ACT Aspire started with the goal of reporting three-year rolling user norms with equal weights to each student. Beginning with fall 2015 reporting, the norm data have been statistically weighted to more closely match a national distribution in terms of selected student demographics and achievement.

This section describes the development of ACT Aspire norms based on weighted data from tests administered through spring 2015. First, the inclusion rules for the samples are described, followed by a description of a weighting methodology designed to produce nationally representative norms. Then, the updated ACT Aspire norm tables, summary statistics, and demographic information for the weighted samples are presented.

ACT Aspire Norm Samples For grades 3–8, the norm samples included students who took ACT Aspire on-grade subject tests in spring 2013, spring 2014, and spring 2015, as consistent with the goal to include three years of rolling data.

U.1 2016 Updates

For grades 9 and 10, the norm samples were restricted to students who took ACT Aspire in consecutive years. The grade 9 samples included students who took grade 8 ACT Aspire and grade 9 ACT Aspire subject tests approximately one year apart. Similarly, the grade 10 samples included students who took ACT Aspire in grade 9 and grade 10 approximately one year apart. As described later, these longitudinal samples were used to anchor the grade 9 and grade 10 norm samples to the grade 8 and grade 9 score distributions, respectively. Table U.1 provides the sample sizes by grade level and subject area.

Table U.1. Sample Sizes Used for Norm Establishment

Subject area Grade level English Mathematics Reading Science Writing 3 102,213 176,117 176,128 73,337 89,557 4 100,592 173,117 173,383 74,618 94,322 5 100,037 172,635 172,437 99,966 93,023 6 98,838 172,865 172,723 72,081 94,949 7 103,297 179,967 179,501 104,468 99,005 8 112,999 190,450 190,086 89,559 107,996 9 3,875 4,149 3,925 3,877 3,364 10 6,436 6,634 6,600 6,172 4,862

Weighting Methodology for ACT Aspire Norms Students assessed with ACT Aspire are not representative of the national population of US elementary and secondary school students. To support interpretations of nationally representative norms, weights were assigned to the ACT Aspire-tested samples so that the weighted samples were similar to the national population of US elementary and secondary school students on school affiliation (public vs. private), and race/ethnicity and academic achievement among public school students.

For grades 3–8, we first estimated the population of US elementary and secondary public school students with respect to race/ethnicity and district mean achievement level. Then, the public school ACT Aspire-tested students were weighted to match on these characteristics. Finally, the weighted public school data was combined with the private school data with the final weighting reflecting the frequency of school affiliation (public or private) in the population. An important assumption of this approach is that a district’s mean achievement level can be measured by its mean ACT Composite score1, and that this measure reflects achievement in the lower grade levels. To examine this assumption, correlations between district mean ACT Composite score and district mean ACT Aspire scores in the lower grade levels were examined

1 When a district tests virtually all students with the ACT, as is often done with ACT state and district testing, a district’s U.2 mean ACT Composite score measures the mean achievement level of the district’s graduating students. 2016 Updates

(Table U.2). For most subject areas and grade levels, district mean ACT Composite score was highly correlated with district mean ACT Aspire score. For writing, the correlations were less consistent across grade levels and were generally lower. The correlations in Table U.2 suggest that district mean ACT Composite score reflects achievement in lower grade levels.

Table U.2. Correlation of District Mean ACT Composite Scores with District Mean ACT Aspire Score

Subject area Grade level English Mathematics Reading Science Writing 3 0.69 0.61 0.72 0.52 0.51 4 0.61 0.70 0.74 0.57 0.08 5 0.63 0.69 0.73 0.66 0.59 6 0.74 0.65 0.68 0.58 0.62 7 0.70 0.79 0.66 0.71 0.38 8 0.73 0.81 0.58 0.66 0.26 9 0.79 0.79 0.73 0.72 0.61 10 0.72 0.74 0.66 0.70 0.47

Note. Districts with at least 50 ACT Aspire-tested students were included.

For grade 9, a sample of students tested with ACT Aspire in grades 8 and 9 was used. After the norms were developed for grade 8, the longitudinal samples (one for each subject area) were weighted to match the national grade 8 score distribution. The weighted score distribution of the grade 9 scores were then calculated. For grade 10, the same approach was used, but with the national grade 9 score distribution serving as the anchor.

The weighting procedure for grades 3–8 is described in detail below. In what follows, target population refers to the population of US elementary and secondary school students. The following steps were taken to weight each ACT Aspire norming sample2: 1. Determine the target population’s relative frequency of students enrolled in public and private schools.3 2. Among public school students within the target population, determine the relative frequency by race/ethnicity and school percent eligible for free or reduced lunch (school FRL). Table U.3 shows the categorizations of these variables and relative

2 There was one norming sample for each grade level and subject area. 3 Estimated from Market Data Retrieval (MDR) data, the target population included 90.5% public-school and 9.5% private-school enrollees. U.3 2016 Updates

frequencies in the public school target population.4 3. Identify all public school districts that administer the ACT test to all 11th grade students; we refer to ACT-tested students within these districts as the ACT state and district testing public school sample. Weight this sample to the public school target population on race/ethnicity and school FRL. Weights were determined by the ratio of the population and sample percentages of each combination of race/ ethnicity and school FRL:

We refer to this weighted sample as the ACT public school population. The ACT public school population was nearly identical to the public school target population with respect to relative frequencies of race/ethnicity and school FRL groups (Table U.3). 4. For each student in the ACT public school population, determine the mean ACT Composite score within their district. Categorize district mean ACT score5 and student race/ethnicity, and calculate the relative frequency of each combination of race/ethnicity and district mean ACT Composite score level (DMACT) within the ACT public school population. 5. For each ACT Aspire norm sample, determine public/private school affiliation and, for public school students, race/ethnicity and DMACT. For students enrolled in public schools, calculate the sample percentage for each combination of race/ ethnicity and DMACT. Weight the sample to the target population on public/private school affiliation and, for public school students, race/ethnicity and DMACT. Weights were determined by the ratio of the population and sample percentages:

For an example norm sample (grade 5 English), the relative frequencies of the weighting variables are provided in Table U.4 for the target population, the unweighted sample, and the weighted sample. The weighted sample was identical to the target population with respect to school affiliation, and the public school sample was identical to the public school target population on race/ethnicity and district mean ACT Composite score. As shown in Table U.4, the weighted mean ACT Aspire grade 5 English score was higher than the unweighted mean. This was expected because the unweighted district mean ACT Composite score was lower than the target population’s district mean ACT Composite score.

4 The public school target population relative frequencies for race/ethnicity and school FRL were derived from the Common Core of Public School Universe Survey Data provided by the National Center for Education Statistics. U.4 5 See Table U.3 for the categorization of district mean ACT Composite score. 2016 Updates

Table U.3. Target Public School Population and ACT Public School Population Population of public ACT state and district testing U.S. elementary public school sample and secondary Variable students Unweighted ACT public school population* Race/ethnicity Asian 4.7% 3.3% 4.7% Black/African American 15.5% 14.6% 15.5% Hispanic 25.0% 13.1% 25.0% Native American 1.1% 0.7% 1.1% White 50.6% 55.4% 50.6% Unknown/Other** 3.1% 12.9% 3.1% FRL % of enrolled school 0–10% 6.8% 5.7% 6.8% 10–20% 8.9% 12.0% 8.9% 20–35% 14.9% 23.1% 14.9% 35–50% 17.4% 23.0% 17.4% 50–65% 16.8% 18.8% 16.8% 65–80% 15.0% 10.1% 15.0% 80–90% 9.5% 3.6% 9.5% 90–100% 10.0% 3.3% 10.0% Unknown 0.8% 0.5% 0.8% District mean ACT score <17.5 10.1% 16.1% 17.5–18.2 9.2% 9.5% 18.2–18.7 11.3% 15.9% 18.7–19.2 10.3% 9.6% 19.2–19.5 8.6% 7.7% 19.5–20.1 9.9% 8.5% 20.1–20.8 12.4% 9.7% 20.8–21.4 8.3% 6.3% 21.4–22.6 9.3% 7.2% >22.6 10.6% 9.6% Mean (SD) district mean ACT score 19.9 (2.5) 19.3 (2.8) Mean (SD) ACT Composite score 19.9 (5.3) 19.3 (5.3)

Note. Shading represents variables used in weighting. * ACT state and district testing public school sample weighted to national population of U.S. elementary and secondary students on race/ethnicity and school FRL. ** For the public school target population, the relative frequency for Unknown/Other was obtained by subtracting the sum of the individual race/ethnicity counts from total enrollment counts. For the ACT data, the Unknown/Other category included Native Hawaiian/Other Pacific Islander, two or more races, prefer not to respond, and no response. U.5 2016 Updates

Table U.4. One Example of ACT Aspire Norm Sample Weighting: Grade 5 English

ACT Aspire norm sample: Population of U.S. grade 5 English elementary and Variable secondary students Unweighted Weighted

Female 49.0% 49.0% 48.5% Male 51.0% 51.0% 51.5% School affiliation Public 90.5% 95.4% 90.5% Private 9.5% 4.6% 9.5% Public school frequencies Race/ethnicity Asian 4.7% 1.6% 4.7% Black/African American 15.5% 32.5% 15.5% Hispanic 25.0% 6.9% 25.0% Native 1.1% 0.7% 1.0% White 50.6% 49.9% 50.6% Unknown/Other 3.1% 8.4% 3.1% FRL % of enrolled school 0–10% 6.8% 1.5% 7.6% 10–20% 8.9% 2.8% 3.0% 20–35% 14.9% 10.3% 15.6% 35–50% 17.4% 12.0% 11.6% 50–65% 16.8% 25.6% 27.2% 65–80% 15.0% 19.0% 14.1% 80–90% 9.5% 11.4% 7.9% 90–100% 10.0% 10.4% 6.3% Unknown 0.8% 7.1% 6.6% District mean ACT score* <17.5 16.1% 27.5% 16.1% 17.5-18.2 9.5% 25.8% 9.5% 18.2-18.7 15.9% 11.3% 15.9% 18.7-19.2 9.6% 13.3% 9.6% Note. Shading represents variables used in weighting. * Frequencies from ACT public school population.

U.6 2016 Updates

Table U.4. (continued) Population of U.S. ACT Aspire norm sample: elementary and grade 5 English Variable secondary students Unweighted Weighted 19.2-19.5 7.7% 2.8% 7.7% 19.5-20.1 8.5% 1.5% 8.5% 20.1-20.8 9.7% 6.4% 9.7% 20.8-21.4 6.3% 7.7% 6.3% 21.4-22.6 7.2% 3.0% 7.2% >22.6 9.6% 0.7% 9.6% Mean (SD) district 19.3 (2.8) 18.4 (1.7) 19.6 (2.5) mean ACT score Mean (SD) ACT Aspire 422.3 (7.1) 423.1 (7.1) score Note. Shading represents variables used in weighting. * Frequencies from ACT public school population.

Figure U.1 summarizes the weighting procedure for the ACT Aspire norms. Tables U.5–U.9 present the demographic information on the weighted samples for each grade and subject test.

Norm Tables and Summary Statistics The norm table includes the cumulative percentages of students scoring at or below each scale score in the weighted norm sample. A smoothing process was applied to the empirical cumulative percentages to reduce the sampling error and increase the precision of the norms. The norm tables are presented in Tables U.10–U.14 for each subject respectively. The mean scale score values for students in each grade and subject test are given in Table U.15. These represent estimated mean scores if all students in the target population (all U.S. elementary and secondary school students) were to take the test. For the same target population, Table U.16 also shows the estimated percentages of students meeting ACT Readiness Benchmark in each grade and subject test.

U.7 2016 Updates

Population of U.S. ACT Aspire ACT Aspire elementary ACT public ACT Aspire grade 8-9 grade 9-10 and secondary school grade 3-8 longitudinal longitudinal students population norm samples samples samples

School School affiliation affiliation = = (public or (public or private) private)

Race/ethnicity Race/ethnicity Race/ethnicity = = (public) (public) (public)

School FRL % School FRL % = (public) (public)

District mean District mean ACT score = ACT score (public) (public)

Grade 8 Grade 8 score = score distribution distribution

Grade 9 Grade 9 score = score distribution distribution

Grade 10 score distribution

Note. Variables used for weighting are included in shaded boxes.

Figure U.1. Conceptual diagram of chain-linked weighting strategy

U.8 2016 Updates

Table U.5. 2015 ACT Aspire Weighted English Norm Group Demographics

Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 102,213) 100,592) 100,037) 98,838) 103,297) 112,999) 3,875) 6,436) Gender F 48.99 48.56 48.11 48.81 47.89 48.6 44.32 48.96 M 49.91 50.32 51 49.76 50.57 50.32 40.73 48.64 No response 1.10 1.12 0.89 1.43 1.54 1.09 14.96 2.40 State AL 47.91 49.07 49.33 44.03 36.9 33.92 0.59 - AR ------0.36 0.17 AZ - 0.19 - - - 0.25 - - CA 1.36 1.50 2.51 2.51 1.37 1.11 - - CO 2.87 2.36 2.00 1.85 1.93 1.92 1.51 1.17 CT ------3.45 FL 0.28 0.35 0.22 0.16 0.14 0.24 - - GA - - - 0.05 0.07 0.09 - - IA 0.17 0.18 0.14 0.15 0.28 1.17 0.89 0.55 IL 0.13 0.13 0.16 0.14 0.12 0.56 8.24 9.59 IN - - - 0.13 0.11 0.20 - - KS 1.73 1.78 1.89 0.78 0.70 3.03 6.34 3.35 KY 0.04 0.05 0.04 0.22 0.76 0.49 2.69 0.09 LA 6.08 6.07 5.07 4.93 4.57 4.47 11.93 7.93 MI 0.41 0.35 0.37 0.52 6.79 10.17 14.75 13.06 MN - - - - - 0.10 - - MO - - - - 0.54 0.06 0.08 2.16 MS 0.10 0.14 0.11 0.13 1.13 1.25 1.84 0.59 ND - - - - - 0.21 - 0.58 NE - - - - - 0.00 - 5.02 NJ ------0.02 NV 0.16 0.19 0.18 0.18 0.25 0.19 - - OH - - 0.02 0.05 0.26 0.50 - 0.25 Note. - indicates no students tested.

U.9 2016 Updates

Table U.5. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 102,213) 100,592) 100,037) 98,838) 103,297) 112,999) 3,875) 6,436) OK 0.29 0.26 0.28 0.29 0.27 0.25 - - OR - - - - - 0.01 - - PA ------3.10 SC 36.72 36.07 36.08 42.27 40.24 36.82 15.08 - SD 0.03 - 0.02 - 0.02 - - - TN 0.03 0.05 0.03 0.04 0.03 0.20 2.10 1.17 TX 0.57 0.58 0.48 0.73 0.87 0.75 1.78 0.27 UT 1.05 0.62 0.98 0.59 1.33 0.49 2.30 - VA ------0.03 - WI 0.06 0.07 0.07 0.27 1.33 1.51 29.20 47.25 No response - - - - - 0.05 0.29 0.26 Race/Ethnicity Asian 4.38 4.36 4.36 4.37 4.36 4.34 0.87 1.72 Black/African American 14.30 14.30 14.32 14.31 14.36 14.20 2.37 2.93 Hispanic/Latino 22.87 22.78 22.81 22.77 22.92 22.73 6.85 7.32 Native American 0.97 0.97 0.95 0.95 0.96 0.98 0.71 0.76 Other 0.90 0.81 0.86 0.67 0.49 0.37 1.42 1.61 White 47.90 47.53 47.55 47.68 48.30 47.88 48.33 69.42 Unknown 8.68 9.25 9.14 9.24 8.62 9.50 39.46 16.22 Note. - indicates no students tested.

U.10 2016 Updates

Table U.6. 2015 ACT Aspire Weighted Mathematics Norm Group Demographics

Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 176,117) 173,117) 172,635) 172,865) 179,967) 190,450) 4,149) 6,634) Gender F 48.72 48.73 48.43 48.38 48.04 48.96 45.46 50.30 M 50.17 50.22 50.71 50.30 50.55 50.05 40.31 47.24 No response 1.11 1.05 0.86 1.32 1.41 0.99 14.24 2.46 State AL 66.56 66.84 67.00 64.51 61.19 58.80 1.63 - AR ------0.38 0.19 AZ - 0.19 - - - 0.24 - - CA 1.34 1.48 2.43 2.56 1.46 1.19 - - CO 1.09 1.06 0.98 0.96 0.94 1.17 1.58 1.15 CT ------3.27 FL 0.27 0.35 0.22 0.16 0.14 0.23 - - GA - - - 0.05 0.07 0.09 - - IA 0.17 0.17 0.14 0.15 0.28 0.70 0.88 0.62 IL 0.09 0.09 0.10 0.10 0.10 0.48 7.30 9.80 IN - - - 0.12 0.11 0.18 - - KS 0.54 0.59 0.55 0.51 0.46 2.27 6.20 3.07 KY 0.04 0.05 0.04 0.22 0.61 0.48 3.05 0.10 LA 5.98 6.00 5.05 4.89 4.62 4.40 11.69 8.47 MI 0.36 0.28 0.39 0.47 2.52 4.15 13.83 12.77 MN - - - - - 0.09 - - MO - - - - 0.36 0.05 0.06 2.43 MS 0.08 0.12 0.10 0.12 0.51 0.77 1.48 0.53 ND - - - - - 0.13 - 0.51 NE - - - - - 0.00 - 4.69 NJ ------0.02 NV 0.25 0.27 0.26 0.24 0.25 0.19 - - OH - - 0.02 0.04 0.24 0.34 - 0.26 Note. - indicates no students tested.

U.11 2016 Updates

Table U.6. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 176,117) 173,117) 172,635) 172,865) 179,967) 190,450) 4,149) 6,634) OK 0.17 0.16 0.16 0.17 0.16 0.14 - - OR - - - - - 0.01 - - PA ------2.90 SC 21.58 21.30 21.18 23.45 22.68 21.42 14.30 0.01 SD 0.03 - 0.02 - 0.02 - - - TN 0.03 0.05 0.03 0.04 0.03 0.20 1.80 1.18 TX 0.31 0.31 0.27 0.37 1.12 0.66 3.54 0.27 UT 1.04 0.62 0.98 0.60 1.33 0.49 2.09 - VA ------0.03 - WI 0.06 0.07 0.07 0.26 0.81 1.08 29.86 47.51 No response - - - - - 0.05 0.31 0.24 Race/Ethnicity Asian 4.38 4.36 4.37 4.37 4.36 4.34 1.00 1.60 Black/African American 14.30 14.30 14.32 14.31 14.34 14.19 2.88 2.99 Hispanic/Latino 22.87 22.78 22.81 22.77 22.92 22.73 6.61 7.02 Native American 0.97 0.97 0.97 0.98 0.98 0.99 0.74 0.80 Other 0.58 0.56 0.61 0.47 0.38 0.30 1.29 1.61 White 47.95 47.59 47.61 47.73 48.27 47.99 50.24 69.45 Unknown 8.95 9.44 9.31 9.37 8.75 9.46 37.24 16.52 Note. - indicates no students tested.

U.12 2016 Updates

Table U.7. 2015 ACT Aspire Weighted Reading Norm Group Demographics

Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 176,128) 173,383) 172,437) 172,723) 179,501) 190,086) 3,925) 6,600) Gender F 48.74 48.76 48.49 48.39 48.10 49.00 47.15 51.04 M 50.2 50.25 50.71 50.32 50.6 50.00 38.87 46.54 No response 1.05 0.99 0.8 1.29 1.3 1.00 13.98 2.42 State AL 66.59 66.87 67.07 64.43 61.81 58.72 1.80 - AR ------0.40 0.18 AZ - 0.19 - - - 0.25 - - CA 1.34 1.48 2.50 2.61 1.48 1.22 - - CO 1.10 1.07 0.97 0.99 0.95 1.20 1.58 1.05 CT ------3.04 FL 0.28 0.35 0.22 0.16 0.14 0.24 - - GA - - - 0.05 0.07 0.09 - - IA 0.17 0.17 0.14 0.15 0.28 0.72 1.02 0.61 IL 0.09 0.09 0.10 0.10 0.10 0.49 7.98 9.83 IN - - - 0.12 0.11 0.18 - - KS 0.53 0.57 0.54 0.50 0.47 2.26 5.96 2.94 KY 0.04 0.05 0.04 0.22 0.60 0.49 0.20 0.10 LA 5.98 6.01 5.00 4.91 4.60 4.51 13.03 7.99 MI 0.37 0.35 0.39 0.47 2.56 4.13 14.06 12.62 MN - - - - - 0.10 - - MO - - - - 0.36 0.05 0.08 2.37 MS 0.08 0.12 0.10 0.12 0.52 0.70 1.20 0.47 ND - - - - - 0.13 - 0.59 NE - - - - - 0.00 - 4.64 NJ ------0.02 NV 0.25 0.27 0.26 0.25 0.25 0.19 - - OH - - 0.02 0.04 0.24 0.34 - 0.25 Note. - indicates no students tested.

U.13 2016 Updates

Table U.7. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 176,128) 173,383) 172,437) 172,723) 179,501) 190,086) 3,925) 6,600) OK 0.18 0.16 0.16 0.18 - 0.14 - - OR - - - - - 0.01 - - PA ------3.09 SC 21.52 21.21 21.15 23.45 22.62 21.48 14.01 - SD 0.03 - 0.02 - 0.02 - - - TN 0.03 0.05 0.03 0.04 0.03 0.20 2.15 1.14 TX 0.33 0.31 0.25 0.43 0.64 0.53 2.02 0.29 UT 1.03 0.62 0.98 0.54 1.34 0.49 2.51 - VA ------0.02 - WI 0.06 0.07 0.07 0.25 0.81 1.09 31.63 48.44 No response - - - - - 0.05 0.36 0.34 Race/Ethnicity Asian 4.38 4.36 4.37 4.37 4.36 4.34 1.14 1.67 Black/African American 14.3 14.30 14.32 14.31 14.34 14.18 2.45 2.67 Hispanic/Latino 22.87 22.78 22.81 22.76 22.92 22.73 6.2 6.47 Native American 0.97 0.97 0.97 0.98 0.98 0.98 0.69 0.69 Other 0.58 0.57 0.62 0.48 0.38 0.30 1.24 1.58 White 47.94 47.6 47.59 47.69 48.28 47.87 49.42 70.76 Unknown 8.95 9.42 9.32 9.42 8.75 9.60 38.86 16.16 Note. - indicates no students tested.

U.14 2016 Updates

Table U.8. 2015 ACT Aspire Weighted Science Norm Group Demographics

Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 73,337) 74,618) 99,966) 72,081) 104,468) 89,559) 3,877) 6,172) Gender F 49.39 48.00 48.34 48.89 48.01 48.63 45.46 50.29 M 49.44 50.87 50.76 49.71 50.45 50.21 39.81 47.14 No response 1.17 1.13 0.90 1.40 1.54 1.15 14.74 2.57 State AL 61.88 63.4 71.56 61.06 64.81 46.75 0.37 - AR ------0.39 0.18 AZ - 0.19 - - - 0.25 - - CA 1.37 1.50 2.48 2.63 1.50 1.23 - - CO 1.65 1.59 1.37 1.53 1.39 1.59 1.47 1.05 CT ------1.26 FL 0.28 0.35 0.22 0.16 0.14 0.24 - - GA - - - 0.05 0.07 0.09 - - IA 0.17 0.18 0.14 0.15 0.28 1.06 0.89 0.65 IL 0.14 0.13 0.13 0.15 0.12 0.69 8.30 10.23 IN - - - 0.14 0.12 0.20 - - KS 2.30 2.38 0.90 1.11 0.79 3.91 6.05 3.03 KY 0.04 0.05 0.04 0.23 0.67 0.48 3.30 0.10 LA 6.11 6.12 5.11 5.03 4.64 4.43 12.63 8.77 MI 0.42 0.26 0.42 0.44 3.78 8.91 12.30 11.75 MN - - - - - 0.09 - - MO - - - - 0.65 0.07 0.08 2.51 MS 0.06 0.08 0.08 0.09 0.51 1.19 1.15 0.49 ND - - - - - 0.34 - 0.60 NE - - - - - 0.00 - 4.87 NJ ------0.01 NV 0.17 0.19 0.18 0.18 0.25 0.19 - - OH - - 0.03 0.05 0.27 0.48 - 0.26 Note. - indicates no students tested.

U.15 2016 Updates

Table U.8. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 73,337) 74,618) 99,966) 72,081) 104,468) 89,559) 3,877) 6,172) OK 0.22 0.22 0.22 0.23 0.24 0.21 - - OR - - - - - 0.01 - - PA ------3.32 SC 23.62 22.12 15.56 25.29 16.34 24.05 14.83 - SD 0.03 - 0.03 - 0.02 - - - TN 0.03 0.04 0.03 0.04 0.03 0.20 2.10 1.26 TX 0.40 0.49 0.43 0.54 0.77 0.76 3.81 0.28 UT 1.06 0.63 0.99 0.60 1.35 0.48 2.32 - VA ------0.03 - WI 0.06 0.07 0.07 0.3 1.26 2.05 29.65 49.06 No response - - - - - 0.05 0.34 0.31 Race/Ethnicity Asian 4.38 4.36 4.36 4.37 4.36 4.34 0.95 1.64 Black/African American 14.30 14.30 14.32 14.32 14.34 14.19 2.37 2.53 Hispanic/Latino 22.88 22.78 22.81 22.78 22.92 22.74 6.53 6.45 Native American 0.97 0.97 0.97 0.92 0.98 0.95 0.73 0.79 Other 0.50 0.49 0.49 0.40 0.26 0.26 1.24 1.66 White 47.9 47.54 47.55 47.72 48.3 47.99 48.77 69.8 Unknown 9.06 9.55 9.50 9.49 8.84 9.53 39.42 17.13 Note. - indicates no students tested.

U.16 2016 Updates

Table U.9. 2015 ACT Aspire Weighted Writing Norm Group Demographics

Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 89,557) 94,322) 93,023) 94,949) 99,005) 107,996) 3,364) 4,862) Gender F 50.31 49.46 47.92 49.12 47.84 49.00 46.84 53.53 M 48.87 49.76 51.33 49.82 50.92 49.91 37.50 45.15 No response 0.82 0.78 0.75 1.06 1.25 1.09 15.66 1.31 State AL 39.35 40.46 40.25 37.92 34.34 30.01 0.59 - AR ------0.44 0.28 AZ - 0.19 - - - 0.31 - - CA 1.46 1.57 2.71 3.03 1.81 1.60 - - CO 2.63 2.10 2.16 2.20 2.49 2.25 1.84 1.23 FL 0.28 0.37 0.24 0.18 0.16 0.32 - - GA - - - 0.06 0.08 0.12 - - IA 0.15 0.15 0.12 0.13 0.31 0.11 - - IL 0.25 0.23 0.24 0.17 0.14 0.50 10.02 9.43 IN - - - 0.23 0.19 0.30 - - KS 8.1 9.34 8.11 0.74 0.68 2.94 6.37 3.97 KY 0.04 0.05 0.04 0.26 0.95 0.64 3.68 0.15 LA 6.05 6.07 4.83 5.32 4.64 3.94 9.73 8.50 MI 0.43 0.27 0.44 0.51 2.95 12.71 13.8 9.14 MN - - - - - 0.12 - - MO - - - - 0.53 0.07 0.12 3.52 MS 0.02 0.01 0.04 0.03 0.01 0.70 - 0.60 ND - - - - - 0.25 - - NE ------0.30 NJ ------0.03 NV 0.19 0.20 0.28 0.20 0.30 0.14 - - OH - - 0.24 - 0.30 0.16 - 0.35 OK - - - 0.28 0.25 0.20 - - Note. - indicates no students tested.

U.17 2016 Updates

Table U.9. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n= (n= (n= (n= (n= (n= (n= (n= 89,557) 94,322) 93,023) 94,949) 99,005) 107,996) 3,364) 4,862) OR - - - - - 0.01 - - PA ------4.23 SC 39.37 37.71 38.64 47.14 45.66 39.26 15.85 - SD 0.03 - 0.03 - 0.02 - - - TN 0.03 0.05 0.03 0.05 0.03 0.03 1.54 0.71 TX 0.41 0.49 0.46 0.58 0.89 0.74 2.13 0.35 UT 1.12 0.65 1.06 0.68 1.58 0.63 2.50 - VA ------0.03 - WI 0.07 0.08 0.09 0.31 1.69 1.89 31.2 56.89 No response - - - - - 0.07 0.16 0.32 Race/Ethnicity Asian 3.41 4.36 3.38 3.68 3.66 3.39 0.92 1.38 Black/African American 14.48 14.29 14.46 15.2 15.15 13.97 2.43 3.01 Hispanic/Latino 23.19 22.79 23.06 24.01 24.12 23.08 7.32 5.57 Native American 0.96 0.95 0.96 1.04 1.05 1.01 0.60 0.88 Other 1.01 1.18 1.00 0.79 0.54 0.41 1.44 1.69 White 48.39 47.5 47.92 45.47 45.92 48.42 49.56 72.03 Unknown 8.55 8.93 9.21 9.81 9.56 9.73 37.72 15.43 Note. - indicates no students tested.

U.18 2016 Updates

Table U.10. 2015 ACT Aspire English Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 2 1 1 1 1 1 1 1 406 3 1 1 1 2 1 1 1 407 5 1 1 2 2 1 1 1 408 8 2 1 2 3 1 1 2 409 12 4 2 3 3 2 2 3 410 16 5 3 4 4 2 3 3 411 22 8 4 5 5 3 4 4 412 27 10 6 7 6 4 5 5 413 34 14 9 8 8 5 6 7 414 40 18 11 11 9 7 8 8 415 46 23 15 13 11 8 9 10 416 52 28 18 16 13 10 11 11 417 58 33 22 19 15 12 14 13 418 64 39 27 23 18 15 16 15 419 69 45 32 26 21 18 19 17 420 74 51 37 31 24 21 21 20 421 78 57 42 35 27 24 24 22 422 82 63 48 40 31 28 27 25 423 86 68 53 44 35 31 31 27 424 88 74 58 49 39 36 34 30 425 91 78 63 54 43 40 37 33 426 93 83 68 59 47 44 41 36 427 95 86 73 64 52 48 44 39 428 96 89 77 68 56 53 48 42

U.19 2016 Updates

Table U.10. (continued) Grade Scale Score 3 4 5 6 7 8 9 10

429 97 92 81 72 61 57 51 46 430 98 94 84 76 65 61 55 49 431 99 96 87 80 69 66 58 52 432 99 97 90 83 73 70 62 56 433 99 98 92 86 77 73 65 59 434 99 99 94 88 80 77 69 63 435 100 99 95 91 83 80 72 66 436 99 97 92 86 83 76 69 437 99 98 94 89 86 79 73 438 100 98 95 91 89 82 76 439 99 96 93 91 84 79 440 99 97 95 93 87 82 441 99 98 96 94 89 85 442 100 99 97 96 92 87 443 99 98 97 93 90 444 99 99 98 95 92 445 99 99 98 96 93 446 99 99 99 97 95 447 99 99 99 98 96 448 100 99 99 99 97 449 99 99 99 98 450 100 99 99 99 451 99 99 99 452 100 99 99 453 99 99 454 99 99 455 99 99 456 100 100

U.20 2016 Updates

Table U.11. 2015 ACT Aspire Mathematics Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 2 1 1 1 1 1 1 1 405 4 1 1 1 1 1 1 1 406 6 1 1 1 1 1 1 1 407 9 2 1 1 2 1 1 1 408 12 3 2 2 3 1 1 1 409 17 5 3 2 5 2 1 1 410 24 8 5 4 6 3 2 2 411 31 13 8 6 9 5 3 3 412 40 19 12 8 12 7 4 4 413 49 27 18 11 15 10 6 6 414 59 36 24 15 20 14 9 8 415 68 46 32 20 24 18 12 10 416 76 55 40 25 29 22 15 13 417 83 65 48 31 35 27 19 16 418 89 73 56 37 40 32 24 20 419 93 79 64 44 46 37 28 24 420 96 85 70 50 52 43 33 28 421 98 89 76 56 57 48 38 32 422 99 92 81 63 62 53 43 36 423 99 94 85 68 67 57 48 40 424 99 96 88 73 72 62 52 45 425 99 97 91 78 76 66 57 49 426 99 98 93 82 79 69 61 53 427 99 99 94 85 83 73 65 57 428 99 99 96 88 85 76 69 61 429 99 99 97 91 88 79 73 65 U.21 2016 Updates

Table U.11. (continued) Grade Scale Score 3 4 5 6 7 8 9 10

430 99 99 97 93 90 81 76 68 431 99 99 98 94 92 84 80 72 432 99 99 99 95 93 86 82 75 433 99 99 99 97 95 88 85 79 434 100 99 99 97 96 90 88 82 435 99 99 98 97 92 90 85 436 99 99 99 98 93 92 88 437 99 99 99 98 95 94 90 438 99 99 99 99 96 95 92 439 99 99 99 99 97 96 94 440 100 99 99 99 98 97 96 441 99 99 99 98 98 97 442 99 99 99 99 99 98 443 99 99 99 99 99 99 444 99 99 99 99 99 99 445 99 99 99 99 99 99 446 100 99 99 99 99 99 447 99 99 99 99 99 448 99 99 99 99 99 449 99 99 99 99 99 450 99 99 99 99 99 451 100 99 99 99 99 452 99 99 99 99 453 100 99 99 99 454 99 99 99 455 99 99 99 456 100 99 99 457 99 99 458 99 99 459 99 99 460 100 100 U.22 2016 Updates

Table U.12. 2015 ACT Aspire Reading Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 4 1 1 1 1 1 1 1 405 8 3 1 1 1 1 1 1 406 13 5 2 2 1 1 1 1 407 19 8 4 3 2 1 1 2 408 25 13 6 5 3 2 2 3 409 32 17 9 7 5 3 4 4 410 38 23 12 9 7 4 5 5 411 44 28 16 12 10 6 7 7 412 50 34 21 15 13 8 10 9 413 56 40 26 19 16 11 13 12 414 62 46 32 22 19 13 16 15 415 67 52 38 26 23 16 19 18 416 73 58 43 30 27 19 23 21 417 78 64 49 35 32 23 26 24 418 83 69 55 40 36 26 30 27 419 87 75 60 45 41 30 34 30 420 91 80 66 50 47 34 38 34 421 93 84 71 56 52 39 42 37 422 96 88 76 62 58 44 47 41 423 97 91 80 67 64 49 51 44 424 98 93 84 73 70 54 55 48 425 99 95 88 78 76 60 59 52 426 99 97 91 83 81 65 64 56 427 99 98 93 87 86 71 68 61 428 99 99 95 90 90 76 73 65

U.23 2016 Updates

Table U.12. (continued) Grade Scale Score 3 4 5 6 7 8 9 10

429 100 99 97 93 93 81 77 70 430 99 98 95 95 86 82 75 431 100 99 97 97 90 86 80 432 99 98 98 93 90 85 433 99 99 99 95 93 90 434 100 99 99 97 95 93 435 99 99 98 97 96 436 100 99 99 98 98 437 99 99 99 99 438 100 99 99 99 439 99 99 99 440 100 99 99 441 99 99 442 100 100

U.24 2016 Updates

Table U.13. 2015 ACT Aspire Science Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 3 1 1 1 1 1 1 1 405 5 2 1 1 1 1 1 1 406 8 4 2 1 1 1 1 1 407 12 6 3 2 2 1 1 1 408 17 8 4 3 3 2 1 1 409 22 11 6 5 5 3 2 2 410 27 15 8 6 7 4 3 3 411 33 18 10 8 9 6 4 4 412 38 23 13 11 12 7 5 5 413 43 28 16 14 15 9 7 6 414 49 33 20 17 19 12 8 8 415 54 38 24 20 22 15 11 10 416 59 43 28 24 26 18 13 12 417 64 49 33 28 30 21 16 15 418 69 55 38 33 34 24 19 17 419 74 60 44 37 38 28 22 20 420 79 66 50 42 42 32 26 23 421 83 71 56 47 46 36 29 26 422 87 76 62 53 50 40 33 30 423 91 81 69 58 54 45 37 33 424 94 85 75 64 59 50 42 36 425 96 89 80 69 63 54 46 40 426 98 92 85 74 67 59 50 43 427 99 94 89 79 71 64 54 47 428 99 96 92 83 75 68 59 51

U.25 2016 Updates

Table U.13. (continued) Grade Scale Score 3 4 5 6 7 8 9 10

429 99 97 94 87 79 72 63 55 430 99 98 96 90 83 77 67 59 431 99 99 98 93 87 81 71 63 432 99 99 98 95 90 84 75 67 433 100 99 99 97 93 87 79 71 434 99 99 98 95 90 82 74 435 99 99 99 97 92 85 78 436 100 99 99 98 94 88 82 437 99 99 99 96 91 85 438 100 99 99 97 93 88 439 99 99 98 95 91 440 100 99 99 97 93 441 99 99 98 95 442 99 99 99 97 443 100 99 99 98 444 99 99 99 445 99 99 99 446 100 99 99 447 99 99 448 99 99 449 100 100

U.26 2016 Updates

Table U.14. 2015 ACT Aspire Writing Norms: Percentage of Students at or below Each Scale Score

Grade Scale Score 3 4 5 6 7 8 9 10 408 1 1 1 1 1 1 1 1 409 1 1 1 1 1 1 1 1 410 2 1 2 2 2 2 2 1 411 3 1 3 2 3 2 3 1 412 4 1 4 3 4 3 3 2 413 6 2 5 4 6 4 5 3 414 9 2 7 5 8 6 6 4 415 12 4 9 7 10 7 8 5 416 16 6 12 9 13 10 10 6 417 21 8 15 11 16 13 12 8 418 27 12 20 14 20 16 15 10 419 33 17 25 17 25 20 18 12 420 40 24 30 21 30 26 21 14 421 48 31 36 26 36 32 25 17 422 55 40 43 32 42 38 29 19 423 62 49 50 37 49 45 33 22 424 69 58 57 44 56 52 38 25 425 75 66 63 50 62 60 43 28 426 81 74 70 56 69 67 48 32 427 85 80 75 62 74 73 54 37 428 89 85 80 68 79 78 60 42 429 92 90 84 73 84 83 66 48 430 94 93 87 78 87 87 73 56 431 96 95 90 82 90 90 80 64 432 97 96 92 85 93 93 86 73 433 98 98 93 88 95 95 91 83 434 99 98 95 91 96 96 95 90 435 99 99 96 93 97 97 98 96 436 99 99 97 94 98 98 99 99

U.27 2016 Updates

Table U.14. (continued) Grade Scale Score 3 4 5 6 7 8 9 10

437 99 99 98 96 99 98 99 99 438 99 99 98 97 99 99 99 99 439 99 99 99 97 99 99 99 99 440 100 100 100 98 99 99 99 99 441 98 99 99 99 99 442 99 99 99 99 99 443 99 99 99 99 99 444 99 99 99 99 99 445 99 99 99 99 99 446 99 99 99 99 99 447 99 99 99 99 99 448 100 100 100 100 100

Table U.15. National Average Scale Scores Using 2015 Norm Sample

Subject area Grade level English Mathematics Reading Science Writing 3 417 413 413 415 422 4 420 416 415 418 424 5 423 418 418 420 424 6 425 421 420 422 426 7 427 421 421 422 424 8 428 423 423 424 424 9 429 425 423 426 426 10 430 426 424 428 428

U.28 2016 Updates

Table U.16. Percentage of Students Who Met Subject Benchmark in the 2015 Norm Sample

Grade English Mathematics Reading Science Writing

3 73 60 38 36 15 4 72 54 42 40 20 5 73 52 40 44 25 6 74 56 50 47 38 7 76 43 42 41 26 8 76 38 51 41 27 9 63 35 45 37 46 10 61 28 39 37 63

U.29 2016 Updates

Section 2: Reliability for Spring 2015 ACT Aspire Forms This section presents the reliability estimate of Cronbach’s alpha for ACT Aspire test forms administered in spring 2015. Forms with the same collection of operational items were combined in the analysis. To obtain a national representative sample, data from the spring 2014 and spring 2015 administrations were combined in the computation for forms administered in both years. Reliability estimates for each grade and subject are shown in Table U.17.

Table U.17. Cronbach’s Alpha of ACT Aspire Forms Administered in Spring 2015

Online Paper Form 1 Form 2 Form 1 Form 2 Grade Subject Items Items Items Items 3 English 0.80 0.73 0.82 0.76 Mathematics 0.80 0.83 0.78 0.82 Reading 0.85 0.84 0.84 0.84 Science 0.88 0.85 0.88 0.85 4 English 0.78 0.78 0.79 0.80 Mathematics 0.69 0.79 0.66 0.80 Reading 0.84 0.82 0.82 0.83 Science 0.84 0.84 0.84 0.84 5 English 0.76 0.79 0.78 0.80 Mathematics 0.70 0.78 0.68 0.78 Reading 0.84 0.81 0.82 0.82 Science 0.84 0.86 0.83 0.86 6 English 0.82 0.81 0.82 0.82 Mathematics 0.79 0.76 0.77 0.73 Reading 0.83 0.83 0.82 0.83 Science 0.87 0.86 0.87 0.86 7 English 0.81 0.86 0.82 0.87 Mathematics 0.83 0.81 0.81 0.78 Reading 0.80 0.81 0.80 0.81 Science 0.88 0.88 0.87 0.86 Note. EHS = Early High School

U.30 2016 Updates

Table U.17. (continued) Online Paper Form 1 Form 2 Form 1 Form 2 Grade Subject Items Items Items Items 8 English 0.85 0.86 0.85 0.86 Mathematics 0.87 0.86 0.86 0.85 Reading 0.82 0.84 0.81 0.83 Science 0.87 0.88 0.87 0.86 EHS English 0.89 0.90 0.91 0.91 Mathematics 0.87 0.88 0.88 0.87 Reading 0.86 0.86 0.87 0.85 Science 0.88 0.89 0.89 0.89 Note. EHS = Early High School

U.31 2016 Updates

Section 3: Changes to the Program for 2016 The ACT Aspire testing program is an effective and efficient measure of college and career readiness. The grade-level tests were designed to be given in a short amount of time to avoid the phenomenon of over-testing and allow for more valuable classroom learning experience. The program is also designed to allow for continuous improvement. The goal of ACT Aspire is to promote fluency in the College and Career Readiness Standards (CCRS), while optimizing the testing experience for students.

The following three categories of improvement will be implemented in the spring 2016 operational administration: 1. Testing time 2. Number of test items for grades 3-5 English and mathematics 3. Reporting category name change for mathematics Testing Time ACT Aspire assessments are timed tests, meaning that students have a certain amount of time allowed to complete their answers. Virtually all timed tests of academic achievement can be considered to be speeded to some degree, as students working at a very deliberate pace may feel they do not have sufficient time to carefully consider each test question

ACT conducted extensive research to inform the adjustments to testing time for future administrations of ACT Aspire. A number of factors related to speededness were analyzed. After completing several types of analyses based on mode, item location, item latency and omit rates, very specific adjustments to testing time were made.

Table U.18 shows the changes in testing time for the spring 2016 operational administration.

Table U.18. ACT Aspire Summative Testing Time Changes (in minutes)

English English Math Math Reading Reading Science Science Grade (New) (Old) (New) (Old) (New) (Old) (New) (Old) 3 40 30 65 55 65 60 60 55 4 40 30 65 55 65 60 60 55 5 40 30 65 55 65 60 60 55 6 40 35 70 60 65 60 60 55 7 40 35 70 60 65 60 60 55 8 40 35 75 65 65 60 60 55 EHS 45 40 75 65 65 60 60 55

Note. NEW assessments are those administered in 2016; OLD assessments are those administered in 2015. Assessments for grades 6–EHS did not undergo an assessment design change.

U.32 2016 Updates

To ensure ACT Aspire provides sufficient evidence for reliable scores, ACT monitors many statistical factors to ensure test scores are valid, reliable, and fair. The following changes to these tests will be implemented in the spring 2016 operational:

English • Add 6 MC Items to grades 3, 4 and 5

Mathematics • Add 6 MC Items to grades 3, 4 and 5 • Remove a single CR from grades 3, 4 and 5 • The reporting category “Foundation” has been renamed to “Integrating Essential Skills” on all score reports.

When adding these items, item latency analysis is also performed to ensure the test timing is controlled to avoid speededness. Secondly, items are added proportionally to reporting categories in order to collect sufficient and high quality evidence for content coverage.

The following tables show content and item type coverage for the mathematics and English tests. They provide information as follows:

English • Table U.19 shows the increase in MC items for English grades 3, 4 and 5 by item type • Table U.20 shows the increase in points by reporting category for English MC items in grades 3, 4 and 5 • Note: Grades 6 through EHS are not changed in spring 2016 and are included here for your reference

Mathematics • Table U.21 shows the increase in MC items for Mathematics grades 3, 4 and 5 by item type. Also shown is the decrease of a single CR item in these grades • Table U.22 shows the increase in points by reporting category for Mathematics MC items in grades 3, 4 and 5, and a decrease in the Justification and Explanation(JE) category for these grades based on reducing by one CR item (equal to 4 points) • Note: Grades 6 through EHS are not changed in spring 2016 and are included here for your reference

U.33 2016 Updates

English Table U.19. English: Number of Items by Item Type Grade

3 3 4 4 5 5 6 7 8 EHS

NEW OLD NEW OLD NEW OLD Item Type Number of Items MC 27–28 21–22 27–28 21–22 27–28 21–22 31–33 31–33 31–33 46–48 TE 3–4 3–4 3–4 3–4 3–4 3–4 2–4 2–4 2–4 2–4 Total 31 25 31 25 31 25 35 35 35 50 Note. EHS – Early High School (Grades 9 and 10); MC – Multiple Choice; TE – Technology Enhanced. Paper-and-pencil tests do not have TE items. MC items are used in their place.

Table U.20. English: Number of Points by Reporting Category Grade

3 3 4 4 5 5 6 7 8 EHS

NEW OLD NEW OLD NEW OLD Item Type Number of Points POW 12–14 9–11 8–10 6–8 8–10 6–8 10–12 9–11 9–11 12–14 KLA 3–5 2–4 3–5 2–4 3–5 4–6 4–6 6–8 CSE 17–19 14–16 17–19 14–16 17–19 14–16 19–21 19–21 19–21 29–31 Total 31 25 31 25 31 25 35 35 35 50 Note. POW – Production of Writing; KLA – Knowledge of Language; CSE – Conventions of Standard English

Mathematics Table U.21. Mathematics: Number of Items by Item Type Grade

3 3 4 4 5 5 6 7 8 EHS

NEW OLD NEW OLD NEW OLD Item Type Number of Points MC 21–23 15–16 21–23 15–16 21–23 15–16 23–25 23–25 27–29 27–29 TE 4–6 5–6 4–6 5–6 4–6 5–6 5–7 5–7 5–7 5–7 CR 3 4 3 4 3 4 4 4 5 5 Total 30 25 30 25 30 25 34 34 38 38 MC – Multiple Choice; TE – Technology Enhanced; CR – Constructed Response. Paper-and-pencil tests do not have TE items. MC items are used in their place.

U.34 2016 Updates

Table U.22. Mathematics: Number of Points by Reporting Category Grade

3 3 4 4 5 5 6 7 8 EHS

NEW OLD NEW OLD NEW OLD Item Type Number of Points Base 10 5–7 5–7 5–8 3–5 5–8 3–5 1–3 1–3 1–3 0–2 Fractions 3–5 2–4 6–8 4–6 6–8 4–6 1–3 1–3 1–3 0–2 NS 3–5 3–5 2–4 1–3 NQ 1–3 OAT 6–8 3–5 4–6 3–5 3–5 3–5 1–3 1–3 0–2 0–2 EE 3–5 3–5 5–7 2–4 RP 3–5 3–5 0–2 1–3 ALG 2–4 Functions 3–5 3–5 Meas. 0–2 0–2 1–3 1–3 GEO 3–5 3–5 3–5 3–5 4–6 3–5 5–7 4–6 6–8 5–7 MD 5–7 3–5 3–5 3–5 3–5 3–5 Data 0–2 1–3 1–3 1–3 SP 3–5 3–5 4–6 4–7 JE 12 16 12 16 12 16 16 16 20 20 Total 39 37 39 37 39 37 46 46 53 53

ACT Aspire Reporting ACT Aspire Reporting Abbreviation Category Abbreviation Category Base 10 Number & Operations in SP Statistics & Probability Base 10 JE Justification & Explanation NS The Number System Fractions Number & Operations– OAT Operations & Algebraic Fractions Thinking NQ Number & Quantity RP Ratios & Proportional EE Expressions & Equations Relationships ALG Algebra Functions Functions GEO Geometry Meas. Measurement Data Data MD Measurement & Data U.35

2017 Updates

This chapter presents updated information and statistical analyses based on data collected from recent operational administrations and a few updates that will be implemented starting from spring 2017. Specifically, Section 1 of this chapter presents the updated national norms used for reporting starting from the fall 2016 operational administration; Section 2 presents raw score reliabilities of forms administered in spring 2016; and Section 3 presents a program update that will be implemented starting from the spring 2017 operational administration.

Section 1: Updating ACT Aspire Norms ACT Aspire Norm Samples ACT Aspire norming analysis was conducted annually after the spring administration and the updated norm tables were used for reporting in the following academic year. This section presents the norm tables and summary statistics of the norm samples used for reporting in the 2016–2017 academic year. The same methodology as in the previous year was implemented. Methodology details are documented in the chapter of 2016 Updates in this technical manual.

The norm samples used for reporting in the academic year of 2016–2017 (i.e., fall 2016 and spring 2017 administrations) included students who took ACT Aspire in the most recent three years. More specifically, for grades 3–8, the norm samples included students who took ACT Aspire on-grade subject tests in spring 2014, spring 2015, and spring 2016. For grades 9 and 10, the norm samples were restricted to students who took ACT Aspire in consecutive years. The grade 9 samples included students who took grade 8 ACT Aspire and grade 9 ACT Aspire subject tests approximately one year apart, with the grade 9 test taken during the 2013–2014, 2014–2015, or 2015–2016 academic years. Similarly, the grade 10 samples included students who took ACT Aspire in grade 9 and grade 10 approximately one year apart, with the grade 10 test taken during the aforementioned three academic years. These longitudinal samples were used to anchor the grade 9 and grade 10 norm samples V.1 2017 Updates

to the grade 8 and grade 9 score distributions, respectively. Table V.1 provides the sample sizes by grade level and subject area, and Tables V.2–V.6 present the demographic information on the weighted samples for each grade and subject test.

Table V.1. Sample Sizes Used for Norm Establishment

Grade Subject level English Mathematics Reading Science Writing 3 172,904 279,776 279,375 148,359 162,295 4 168,419 272,553 272,637 147,023 166,484 5 167,884 272,281 270,693 197,599 164,527 6 174,957 281,720 281,315 153,940 174,282 7 184,689 291,035 290,210 213,694 180,115 8 207,723 313,556 312,530 186,395 199,845 9 22,055 22,131 21,338 20,463 16,053 10 109,704 109,058 107,837 105,792 88,909

Norm Tables and Summary Statistics The norm table includes the cumulative percentages of students scoring at or below each scale score in the weighted norm sample. Tables V.7–V.11 present the norm tables for each subject respectively. Table V.12 presents the mean scale score values of the weighted norm samples. Table V.13 presents the percentages of students meeting ACT Readiness Benchmark in each grade and subject test.

V.2 2017 Updates

Table V.2. 2016 ACT Aspire Weighted English Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 172,904) 168,419) 167,884) 174,957) 184,689) 207,723) 22,055) 109,704) Gender F 48.92 48.65 48.56 49.01 48.51 49.02 48.42 48.25 M 50.39 50.62 50.73 50.21 50.72 50.09 46.94 49.78 No response 0.70 0.73 0.71 0.77 0.77 0.89 4.63 1.97 State AL 36.41 37.02 37.22 36.20 34.13 32.19 1.03 0.17 AR 23.55 22.66 22.77 21.59 20.01 18.67 36.37 14.22 AS 0.01 0.01 0.01 0.01 0.01 0.00 - - AZ 0.06 0.12 0.05 0.12 0.12 0.18 - 0.01 CA 1.60 1.74 2.16 4.05 3.76 2.59 0.82 0.74 CO 1.05 1.07 0.93 0.93 0.86 1.36 0.22 4.08 CT ------<0.01 0.23 DC ------0.42 0.05 FL 0.15 0.15 0.14 0.20 0.19 0.25 0.05 1.88 GA 0.01 0.02 0.02 0.04 0.03 0.04 0.01 - GU 0.14 0.01 0.01 0.18 0.03 0.02 - - HI - - - - 0.12 1.91 0.59 1.04 IA 0.06 0.08 0.05 0.06 0.09 0.46 0.16 0.05 IL 2.67 2.76 2.59 2.43 2.45 2.34 5.12 3.96 IN - - - 0.17 0.13 0.19 2.39 0.52 KS 0.94 0.99 0.95 0.73 0.70 2.67 10.51 2.34 KY 0.02 0.03 0.03 0.11 0.48 0.28 1.16 0.18 LA 3.62 3.43 3.10 2.90 2.96 3.05 8.81 2.56 MA ------0.01 MD 0.02 0.02 0.01 0.02 0.03 0.02 0.01 0.01 ME ------MI 0.31 0.29 0.31 0.44 3.10 4.35 2.73 1.33 MN 0.01 0.01 0.02 0.02 0.02 0.04 - - MO - - - 0.04 0.61 0.33 0.50 0.78 MP 0.48 0.47 0.45 0.52 0.35 0.34 - - MS 0.24 0.23 0.26 0.27 0.60 1.02 3.80 0.82 MT - - - - 0.03 0.02 0.16 0.04 NC 0.01 0.02 0.01 0.01 0.02 0.01 0.03 - ND - - - - - 0.04 0.01 0.41 NE - - - - - 0.00 0.46 2.40 NJ - - - - 0.00 0.01 - 0.11 NV 0.18 0.24 0.20 0.23 0.21 0.38 - -

Note: - indicates no students tested. V.3 2017 Updates

Table V.2. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 172,904) 168,419) 167,884) 174,957) 184,689) 207,723) 22,055) 109,704) NY - - - 0.04 0.05 0.08 0.28 0.38 OH - 0.14 0.22 0.22 0.40 0.44 3.26 1.35 OK 0.23 0.29 0.34 0.35 0.36 0.31 0.89 0.29 OR 0.02 0.01 0.01 0.01 0.01 0.01 0.04 0.79 PA 0.14 0.14 0.14 0.13 0.12 0.10 - 0.73 SC 27.40 27.23 27.17 27.00 26.5 24.41 8.30 0.96 SD 0.02 0.05 0.14 0.10 0.02 0.10 0.18 0.17 TN 0.09 0.09 0.09 0.19 0.17 0.22 1.25 0.34 TX 0.31 0.49 0.31 0.46 0.59 0.49 2.17 0.76 UT 0.23 0.17 0.23 0.06 0.13 0.07 - - VA - 0.01 0.01 0.01 0.02 0.04 0.01 - WA - - - - - 0.00 0.61 0.10 WI 0.04 0.04 0.07 0.18 0.60 0.96 7.30 55.85 WY ------0.26 0.20 No response ------0.11 0.13 Race/Ethnicity Asian 4.52 4.52 4.50 4.50 4.50 4.47 2.64 3.08 Black/ African American 14.42 14.44 14.41 14.43 14.42 14.33 6.27 7.07 Hispanic/ Latino 23.79 23.74 23.66 23.66 23.67 23.59 7.73 9.50 Native American 0.94 0.94 0.94 0.94 0.95 0.95 0.30 0.75 Other 0.91 0.88 0.86 0.64 0.52 0.52 3.31 2.16 White 47.42 47.62 47.48 47.46 47.53 47.36 30.45 57.18 Unknown 7.99 7.85 8.14 8.37 8.42 8.79 49.30 20.26

Note: - indicates no students tested.

V.4 2017 Updates

Table V.3. 2016 ACT Aspire Weighted Mathematics Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n= (n = (n = (n = (n = (n = (n = (n = 279,776) 272,553) 272,281) 281,720) 291,035) 313,556) 22,131) 109,058) Gender F 49.00 48.80 48.65 48.89 48.55 49.02 48.61 48.51 M 50.35 50.51 50.69 50.40 50.72 50.12 45.85 49.44 No response 0.65 0.70 0.66 0.72 0.72 0.86 5.54 2.05 State AL 56.05 56.14 56.19 54.53 53.00 50.70 1.20 0.18 AR 14.65 14.21 14.31 13.68 13.06 12.43 30.82 13.05 AS 0.00 0.00 0.00 0.01 0.00 0.00 - - AZ 0.06 0.11 0.05 0.11 0.11 0.17 - 0.03 CA 1.56 1.71 2.09 3.95 3.64 2.55 3.98 0.67 CO 0.60 0.62 0.55 0.67 0.63 0.95 0.22 4.17 CT ------<0.01 0.22 DC ------0.44 0.06 FL 0.14 0.14 0.13 0.19 0.18 0.24 0.05 1.93 GA 0.01 0.01 0.01 0.03 0.03 0.04 <0.01 - GU 0.14 0.01 0.01 0.18 0.03 0.02 - - HI - - - - 0.11 1.52 0.60 1.07 IA 0.05 0.07 0.05 0.05 0.09 0.31 0.16 0.20 IL 2.59 2.67 2.51 2.39 2.42 2.25 5.95 4.05 IN - - - 0.15 0.12 0.17 2.00 0.54 KS 0.73 0.77 0.79 0.58 0.58 2.32 10.36 2.47 KY 0.02 0.03 0.03 0.10 0.42 0.28 1.28 0.18 LA 3.55 3.38 3.05 2.87 2.94 3.03 9.07 2.60 MA ------0.01 MD 0.02 0.02 0.01 0.02 0.02 0.02 0.01 0.01 ME ------MI 0.29 0.26 0.32 0.38 1.72 2.79 2.71 1.32 MN 0.01 0.01 0.01 0.01 0.01 0.04 - - MO - 0.01 - 0.02 0.35 0.22 0.53 0.79 MP 0.27 0.27 0.25 0.44 0.31 0.30 - - MS 0.22 0.21 0.24 0.25 0.48 0.74 3.86 0.88 MT - - - - 0.02 0.01 0.16 0.04 NC 0.01 0.02 0.01 0.01 0.01 0.01 0.03 - ND - - - - - 0.03 0.01 0.40 NE - - - - - 0.00 1.43 2.44 NJ - - - - 0.00 0.01 - 0.11 NV 0.43 0.53 0.25 0.69 0.43 0.33 - -

Note: - indicates no students tested. V.5 2017 Updates

Table V.3. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n= (n = (n = (n = (n = (n = (n = (n = 279,776) 272,553) 272,281) 281,720) 291,035) 313,556) 22,131) 109,058) NY - - - 0.03 0.05 0.07 0.29 0.39 OH - 0.09 0.12 0.14 0.26 0.32 3.26 1.32 OK 0.14 0.18 0.22 0.24 0.26 0.23 0.89 0.28 OR 0.01 0.01 0.01 0.01 0.01 0.01 0.03 0.85 PA 0.13 0.14 0.14 0.13 0.12 0.10 - 0.79 SC 17.70 17.72 17.58 17.35 17.15 16.21 8.36 1.00 SD 0.02 0.05 0.14 0.09 0.02 0.10 0.18 0.17 TN 0.09 0.09 0.10 0.18 0.17 0.22 1.26 0.35 TX 0.21 0.31 0.51 0.32 0.64 0.41 2.18 0.79 UT 0.23 0.17 0.23 0.06 0.13 0.07 - - VA - 0.01 0.01 0.01 0.02 0.03 0.01 - WA - - - - - 0.00 0.58 0.10 WI 0.04 0.04 0.06 0.15 0.45 0.73 7.76 56.20 WY ------0.33 0.20 No response ------0.13 Race/Ethnicity Asian 4.53 4.52 4.50 4.50 4.50 4.47 2.79 3.07 Black/ African American 14.45 14.48 14.45 14.48 14.46 14.36 6.52 7.65 Hispanic/ Latino 23.79 23.74 23.66 23.65 23.66 23.58 8.42 9.90 Native American 0.94 0.94 0.94 0.94 0.95 0.95 0.34 0.81 Other 0.65 0.62 0.65 0.52 0.43 0.46 3.43 2.25 White 47.44 47.64 47.51 47.45 47.52 47.36 32.91 56.97 Unknown 8.20 8.05 8.29 8.45 8.48 8.82 45.06 19.34

Note: - indicates no students tested.

V.6 2017 Updates

Table V.4. 2016 ACT Aspire Weighted Reading Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 279,375) 272,637) 270,693) 281,315) 290,210) 312,530) 21,338) 107,837) Gender F 49.00 48.79 48.65 48.87 48.58 49.03 49.24 48.51 M 50.34 50.51 50.69 50.41 50.69 50.10 45.40 49.44 No response 0.65 0.70 0.66 0.72 0.73 0.87 5.36 2.05 State AL 56.06 56.13 56.40 54.46 53.17 50.53 1.25 0.18 AR 14.65 14.19 14.36 13.70 13.10 12.48 30.98 12.80 AS 0.00 0.00 0.00 0.01 0.00 0.00 - - AZ 0.06 0.12 0.05 0.12 0.11 0.17 - 0.04 CA 1.57 1.71 2.13 3.96 3.67 2.54 4.00 0.65 CO 0.60 0.62 0.55 0.65 0.62 0.94 0.24 4.21 CT ------0.19 DC ------0.40 0.06 FL 0.14 0.14 0.13 0.19 0.18 0.25 0.05 1.85 GA 0.01 0.01 0.01 0.03 0.03 0.04 0.01 - GU 0.14 0.01 0.01 0.18 0.03 0.02 - - HI - - - - 0.11 1.52 0.62 1.03 IA 0.05 0.07 0.05 0.05 0.09 0.32 0.17 0.20 IL 2.59 2.67 2.51 2.39 2.43 2.26 6.21 3.95 IN - - - 0.15 0.13 0.18 2.18 0.58 KS 0.73 0.77 0.76 0.56 0.57 2.30 10.29 2.43 KY 0.02 0.03 0.03 0.10 0.42 0.28 0.81 0.18 LA 3.55 3.38 3.04 2.87 2.94 3.03 9.64 2.64 MA ------0.01 MD 0.02 0.02 0.01 0.02 0.02 0.02 0.01 0.01 ME ------MI 0.29 0.29 0.32 0.37 1.73 2.84 2.76 1.35 MN 0.01 0.01 0.01 0.01 0.01 0.04 - - MO - 0.01 - 0.02 0.35 0.22 0.53 0.81 MP 0.27 0.27 0.25 0.44 0.31 0.30 - - MS 0.22 0.21 0.25 0.25 0.48 0.73 3.71 0.85 MT - - - - 0.02 0.01 0.18 0.04 NC 0.01 0.02 0.01 0.01 0.01 0.01 0.03 - ND - - - - - 0.03 0.01 0.40 NE - - - - - 0.00 1.43 2.48 NJ - - - - 0.00 0.01 - 0.12 NV 0.43 0.53 0.25 0.70 0.44 0.33 0.29 0.40

Note: - indicates no students tested. V.7 2017 Updates

Table V.4. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 279,375) 272,637) 270,693) 281,315) 290,210) 312,530) 21,338) 107,837) NY - - - 0.03 0.05 0.07 - - OH - 0.09 0.12 0.14 0.28 0.33 3.29 1.39 OK 0.14 0.18 0.23 0.24 0.18 0.23 0.94 0.28 OR 0.01 0.01 0.01 0.01 0.01 0.01 0.03 0.87 PA 0.13 0.14 0.14 0.13 0.12 0.10 - 0.74 SC 17.67 17.68 17.60 17.37 17.12 16.28 8.55 0.98 SD 0.02 0.05 0.14 0.10 0.02 0.10 0.19 0.13 TN 0.09 0.09 0.08 0.18 0.17 0.22 1.38 0.35 TX 0.22 0.34 0.24 0.35 0.48 0.42 0.84 0.48 UT 0.23 0.17 0.23 0.06 0.13 0.07 - - VA - 0.01 0.01 0.01 0.02 0.03 0.01 - WA - - - - - 0.00 0.59 0.10 WI 0.04 0.04 0.06 0.15 0.45 0.73 7.99 56.85 WY ------0.36 0.21 No response ------0.14 Race/Ethnicity Asian 4.53 4.52 4.50 4.50 4.50 4.47 2.83 3.15 Black/ African American 14.45 14.48 14.45 14.48 14.46 14.36 6.30 7.01 Hispanic/ Latino 23.79 23.74 23.66 23.65 23.66 23.59 8.12 9.41 Native American 0.94 0.94 0.94 0.94 0.95 0.95 0.35 0.78 Other 0.65 0.63 0.66 0.52 0.43 0.47 3.46 2.19 White 47.44 47.64 47.50 47.45 47.52 47.36 33.42 58.71 Unknown 8.20 8.05 8.29 8.45 8.48 8.81 45.51 18.76

Note: - indicates no students tested.

V.8 2017 Updates

Table V.5. 2016 ACT Aspire Weighted Science Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 148,359) 147,023) 197,599) 153,940) 213,694) 186,395) 20,463) 105,792) Gender F 48.95 48.37 48.64 48.95 48.56 49.08 48.41 48.24 M 50.29 50.84 50.65 50.27 50.67 50.03 45.56 49.71 No response 0.75 0.79 0.71 0.79 0.77 0.89 6.02 2.05 State AL 45.19 46.99 57.82 45.62 54.44 38.75 1.11 0.18 AR 28.37 26.48 20.24 26.09 18.55 21.49 30.32 12.82 AS 0.00 0.01 0.00 0.01 0.01 0.00 - - AZ 0.07 0.12 0.05 0.12 0.12 0.18 - 0.01 CA 1.63 1.74 2.14 4.25 3.88 2.65 3.16 0.11 CO 1.24 1.27 0.76 1.18 0.84 1.70 0.24 4.15 CT ------<0.01 0.09 FL 0.15 0.15 0.14 0.22 0.19 0.22 0.04 1.87 GA 0.01 0.01 0.02 0.03 0.03 0.04 <0.01 - GU ------HI - - - - 0.12 2.06 0.62 0.74 IA 0.06 0.08 0.05 0.06 0.09 0.44 0.17 0.19 IL 2.76 2.80 2.59 2.49 2.46 2.39 6.52 4.14 IN - - - 0.17 0.13 0.19 2.21 0.58 KS 1.04 1.11 0.97 0.80 0.68 2.91 10.92 2.53 KY 0.02 0.03 0.03 0.11 0.45 0.28 1.39 0.19 LA 3.71 3.47 3.12 2.98 2.98 3.07 9.78 2.65 MA ------0.01 MD 0.02 0.02 0.01 0.02 0.03 0.02 0.01 0.01 ME ------MI 0.30 0.22 0.32 0.43 2.19 4.89 2.50 1.26 MN 0.02 0.02 0.02 0.01 0.01 0.04 - - MO - - - 0.04 0.53 0.36 0.57 0.79 MP 0.43 0.43 0.34 0.63 0.36 0.36 - - MS 0.25 0.23 0.25 0.27 0.48 1.00 4.02 0.80 MT - - - - 0.02 0.02 0.18 0.04 NC 0.01 0.02 0.01 0.01 0.02 0.01 0.03 - ND - - - - - 0.05 0.01 0.42 NE - - - - - 0.00 1.51 2.49 NJ - - - - 0.00 0.00 - 0.13 NV 0.09 0.09 0.09 0.14 0.14 0.36 - - NY - - - 0.04 0.05 0.09 0.30 0.41

Note: - indicates no students tested. V.9 2017 Updates

Table V.5. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 148,359) 147,023) 197,599) 153,940) 213,694) 186,395) 20,463) 105,792) OH - 0.21 0.19 0.31 0.36 0.48 3.49 1.39 OK 0.21 0.27 0.24 0.32 0.28 0.30 0.96 0.25 OR 0.01 0.01 0.01 0.01 0.01 0.01 0.03 0.81 PA 0.14 0.14 0.14 0.13 0.12 0.10 - 0.79 SC 13.63 13.27 9.63 12.51 8.97 13.42 8.92 0.98 SD 0.02 0.05 0.14 0.10 0.02 0.10 0.19 0.17 TN 0.10 0.10 0.11 0.20 0.18 0.23 1.37 0.36 TX 0.24 0.43 0.28 0.40 0.52 0.47 0.89 0.46 UT 0.24 0.17 0.23 0.06 0.13 0.07 - - VA - 0.01 0.01 0.02 0.02 0.05 0.01 - WA - - - - - 0.00 0.48 0.07 WI 0.04 0.04 0.07 0.19 0.59 1.18 7.77 57.75 WY ------0.26 0.21 No response ------0.13 Race/Ethnicity Asian 4.52 4.52 4.50 4.51 4.50 4.47 2.82 3.14 Black/ African American 14.43 14.45 14.42 14.44 14.42 14.33 6.23 7.62 Hispanic/ Latino 23.81 23.75 23.67 23.67 23.67 23.59 8.94 9.58 Native American 0.94 0.94 0.94 0.94 0.95 0.95 0.34 0.80 Other 0.71 0.72 0.59 0.51 0.38 0.44 3.35 2.21 White 47.45 47.63 47.49 47.51 47.55 47.38 33.98 58.34 Unknown 8.14 7.98 8.38 8.42 8.53 8.84 44.34 18.32

Note: - indicates no students tested.

V.10 2017 Updates

Table V.6. 2016 ACT Aspire Weighted Writing Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 162,295) 166,484) 164,527) 174,282) 180,115) 199,845) 16,053) 88,909) Gender F 49.75 48.95 49.33 49.67 48.94 49.83 51.08 48.20 M 49.60 50.35 49.99 49.61 50.32 49.25 43.48 50.30 No response 0.65 0.70 0.68 0.72 0.74 0.92 5.44 1.50 State AL 38.03 39.33 38.35 38.51 35.82 36.38 1.37 0.27 AR 23.58 22.22 22.56 21.17 19.92 18.86 27.41 11.39 AS 0.01 0.01 0.01 0.01 0.01 0.00 - - AZ 0.07 0.12 0.06 0.12 0.13 0.19 - 0.01 CA 1.67 1.77 2.25 4.02 3.84 2.78 5.17 1.00 CO 0.95 0.89 0.83 0.81 0.85 1.31 0.01 1.37 DC ------0.46 0.07 FL 0.15 0.15 0.14 0.21 0.20 0.28 0.08 2.90 GA 0.01 0.02 0.02 0.03 0.03 0.03 - - GU ------HI - - - - 0.12 0.85 0.67 1.11 IA 0.05 0.05 0.04 0.05 0.09 0.05 <0.01 - IL 2.85 2.89 2.72 2.60 2.63 2.54 6.57 4.65 IN - - - 0.18 0.14 0.21 2.87 0.64 KS 0.83 0.87 0.84 0.59 0.61 2.34 9.07 2.28 KY 0.02 0.03 0.03 0.12 0.49 0.32 1.90 0.19 LA 3.66 3.50 3.04 3.01 2.82 2.77 7.78 2.38 MA ------0.02 MD 0.02 0.02 0.01 0.01 0.02 0.01 0.01 0.01 MI 0.33 0.27 0.33 0.45 3.29 2.83 3.10 1.33 MN 0.01 0.01 0.02 0.01 0.01 0.04 - - MO - - - 0.04 0.51 0.34 0.15 0.64 MP 0.34 0.35 0.33 0.53 0.37 0.35 - - MS 0.10 0.08 0.14 0.11 0.14 0.78 1.64 0.46 MT - - - - 0.02 0.02 0.17 0.05 NC 0.02 0.01 0.01 0.00 0.01 0.01 0.04 - ND - - - - - 0.04 0.01 - NE - - - - - 0.00 0.51 0.23 NJ - - - - 0.00 0.00 - 0.18 NV 0.14 0.14 0.14 0.15 0.16 0.21 - - NY - - - 0.04 0.06 0.09 0.42 0.65 OH - 0.14 0.31 0.19 0.39 0.25 4.16 0.77

Note: - indicates no students tested. V.11 2017 Updates

Table V.6. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 162,295) 166,484) 164,527) 174,282) 180,115) 199,845) 16,053) 88,909) OK 0.01 0.15 0.18 0.32 0.35 0.31 0.99 0.35 OR 0.02 0.01 0.01 0.01 0.01 0.01 0.03 1.29 PA 0.15 0.15 0.15 0.14 0.13 0.12 - 0.77 SC 26.30 26.10 26.60 25.60 25.35 23.74 11.74 1.35 SD 0.02 0.06 0.15 0.10 0.02 0.11 - 0.15 TN 0.09 0.10 0.09 0.20 0.19 0.17 1.81 0.44 TX 0.28 0.38 0.34 0.41 0.51 0.50 2.44 1.02 UT 0.24 0.17 0.24 0.07 0.14 0.08 - - VA - - - 0.01 0.02 0.04 0.01 - WA - - - - - 0.00 0.75 0.11 WI 0.04 0.04 0.07 0.18 0.60 1.03 8.30 61.67 WY ------0.35 0.19 No response ------0.06 Race/Ethnicity Asian 4.53 4.53 4.50 4.51 4.50 4.48 3.06 2.85 Black/ African American 14.43 14.45 14.41 14.45 14.42 14.35 6.78 7.00 Hispanic/ Latino 23.83 23.78 23.70 23.70 23.71 23.65 7.22 9.27 Native American 0.94 0.91 0.91 0.94 0.95 0.92 0.31 0.81 Other 0.94 0.92 0.91 0.69 0.58 0.50 3.20 2.13 White 47.39 47.67 47.46 47.51 47.51 47.36 31.55 57.71 Unknown 7.94 7.75 8.11 8.19 8.32 8.74 47.88 20.24

Note: - indicates no students tested.

V.12 2017 Updates

Table V.7. 2016 ACT Aspire English Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 2 1 1 1 1 1 1 1 406 3 1 1 1 2 1 1 1 407 5 2 1 2 2 1 1 1 408 8 3 1 2 3 2 2 2 409 12 4 2 3 3 2 2 2 410 16 6 3 4 4 3 3 3 411 22 9 5 5 5 4 4 4 412 27 12 6 7 6 5 6 5 413 33 15 9 9 8 6 7 6 414 40 20 11 11 9 8 9 7 415 46 24 14 14 11 9 11 9 416 52 30 18 16 13 11 13 10 417 58 35 22 20 16 14 15 12 418 63 41 27 23 18 16 18 14 419 68 47 31 27 21 19 20 16 420 73 52 36 31 24 22 23 18 421 77 58 42 36 28 25 26 21 422 81 64 47 40 31 29 29 23 423 84 69 52 45 35 33 33 26 424 87 74 58 50 39 37 36 29 425 90 78 63 54 44 41 39 31 426 92 83 68 59 48 45 43 34 427 94 86 73 64 52 49 46 37 428 96 89 77 68 57 54 49 40 429 97 92 81 72 61 58 53 43 430 98 94 84 76 65 62 56 47 431 99 96 87 79 69 66 60 50 432 99 97 90 82 73 70 63 53 433 99 98 92 85 77 74 66 56 434 99 99 94 88 81 78 69 60 435 100 99 95 90 84 81 73 63 436 99 97 92 86 84 76 66 437 99 98 93 89 86 78 70 438 100 98 95 91 89 81 73 V.13 2017 Updates

Table V.7. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 96 93 91 84 76 440 99 97 95 93 86 79 441 99 98 96 94 88 82 442 100 98 97 96 90 84 443 99 98 97 92 87 444 99 99 97 94 89 445 99 99 98 95 91 446 99 99 99 96 93 447 99 99 99 97 95 448 100 99 99 98 96 449 99 99 99 97 450 100 99 99 98 451 99 99 99 452 100 99 99 453 99 99 454 99 99 455 99 99 456 100 100

V.14 2017 Updates

Table V.8. 2016 ACT Aspire Mathematics Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 3 1 1 1 1 1 1 1 405 4 1 1 1 1 1 1 1 406 6 1 1 1 2 1 1 1 407 9 2 1 1 3 1 1 1 408 13 3 2 2 4 1 1 1 409 18 5 3 3 5 2 1 1 410 24 8 6 4 7 4 2 2 411 31 13 9 6 10 5 4 3 412 40 19 13 8 13 8 5 5 413 49 27 19 11 17 11 8 7 414 58 35 25 15 21 14 10 9 415 68 45 33 20 26 18 14 11 416 76 54 41 25 31 23 17 14 417 83 63 49 30 36 28 21 18 418 89 72 57 37 42 33 26 21 419 93 78 64 43 47 38 30 25 420 96 84 70 50 53 43 35 28 421 98 88 76 56 58 48 39 32 422 99 92 81 62 63 53 44 36 423 99 94 85 68 68 58 48 40 424 99 96 88 73 72 62 52 43 425 99 97 91 77 76 66 57 47 426 99 98 93 82 79 70 61 51 427 99 99 94 85 82 73 64 54 428 99 99 96 88 85 76 68 58 429 99 99 97 91 88 79 71 61 430 99 99 97 93 90 82 75 65 431 99 99 98 94 92 84 78 68 432 99 99 99 96 93 86 81 71 433 99 99 99 97 94 88 83 75 434 100 99 99 98 96 90 86 78 435 99 99 98 97 92 88 81 436 99 99 99 97 93 90 84 437 99 99 99 98 94 92 86 438 99 99 99 99 96 94 89 V.15 2017 Updates

Table V.8. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 99 97 95 91 440 100 99 99 99 98 97 93 441 99 99 99 98 98 95 442 99 99 99 99 98 96 443 99 99 99 99 99 98 444 99 99 99 99 99 98 445 99 99 99 99 99 99 446 100 99 99 99 99 99 447 99 99 99 99 99 448 99 99 99 99 99 449 99 99 99 99 99 450 99 99 99 99 99 451 100 99 99 99 99 452 99 99 99 99 453 100 99 99 99 454 99 99 99 455 99 99 99 456 100 99 99 457 99 99 458 99 99 459 99 99 460 100 100

V.16 2017 Updates

Table V.9. 2016 ACT Aspire Reading Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 4 2 1 1 1 1 1 1 405 8 3 1 1 1 1 1 1 406 14 6 3 2 1 1 1 1 407 20 9 4 4 2 1 2 2 408 26 13 7 6 4 2 3 3 409 33 18 10 8 6 3 5 4 410 39 23 14 11 8 5 6 6 411 45 29 18 14 11 7 9 8 412 51 34 23 17 14 9 12 10 413 56 40 28 21 17 11 15 13 414 62 46 33 25 21 14 18 16 415 67 52 39 29 25 17 21 19 416 73 58 45 34 29 20 25 22 417 78 63 50 38 33 24 29 25 418 82 69 56 43 38 27 33 28 419 87 74 61 48 43 31 36 31 420 90 79 67 53 48 36 40 35 421 93 84 72 58 53 40 44 38 422 95 87 76 64 59 45 48 42 423 97 91 81 69 65 50 52 45 424 98 93 84 74 71 55 56 49 425 99 95 88 79 76 61 61 53 426 99 97 91 83 81 66 65 58 427 99 98 94 87 86 71 69 62 428 99 99 96 91 90 77 74 67 429 100 99 97 93 93 82 78 71 430 99 98 95 95 86 82 76 431 100 99 97 97 90 86 81 432 99 98 98 93 90 86 433 99 99 99 95 93 90 434 100 99 99 97 95 93 435 99 99 98 97 96 436 100 99 99 99 98 437 99 99 99 99 438 100 99 99 99 V.17 2017 Updates

Table V.9. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 440 100 99 99 441 99 99 442 100 100

V.18 2017 Updates

Table V.10. 2016 ACT Aspire Science Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 3 2 1 1 1 1 1 1 405 6 3 1 1 1 1 1 1 406 9 4 2 2 2 1 1 1 407 13 6 3 2 2 1 1 1 408 18 9 4 4 4 2 2 2 409 23 11 6 5 5 3 3 2 410 28 15 8 7 7 5 4 3 411 33 19 11 9 10 6 5 5 412 38 23 14 12 13 8 7 6 413 44 27 17 14 16 10 9 8 414 48 32 21 18 19 13 11 10 415 53 37 25 21 23 16 14 13 416 58 42 29 25 26 19 17 15 417 63 48 34 29 30 22 21 18 418 68 53 39 33 34 26 24 21 419 72 59 45 38 38 30 28 25 420 77 64 50 42 42 33 32 28 421 81 70 56 47 46 38 36 32 422 85 75 62 53 50 42 40 35 423 89 80 68 58 55 46 44 38 424 92 84 74 63 59 51 48 42 425 95 88 79 68 63 55 52 45 426 97 91 84 73 67 60 56 49 427 98 93 88 78 71 65 60 52 428 99 95 91 82 76 69 64 56 429 99 97 94 86 80 73 67 59 430 99 98 96 89 83 77 71 62 431 99 99 97 92 87 81 74 66 432 99 99 98 94 90 85 78 69 433 100 99 99 96 93 88 81 72 434 99 99 97 95 90 84 76 435 99 99 98 97 93 87 79 436 100 99 99 98 95 89 82 437 99 99 99 96 92 85 438 100 99 99 97 94 88 V.19 2017 Updates

Table V.10. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 98 96 91 440 100 99 99 97 93 441 99 99 98 95 442 99 99 99 97 443 100 99 99 98 444 99 99 99 445 99 99 99 446 100 99 99 447 99 99 448 99 99 449 100 100

V.20 2017 Updates

Table V.11. 2016 ACT Aspire Writing Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 408 1 1 1 1 1 1 1 1 409 1 1 1 1 2 1 1 1 410 2 1 2 1 3 2 2 2 411 3 1 3 2 4 3 3 3 412 4 2 3 3 5 4 5 5 413 6 2 4 4 7 5 6 6 414 8 3 6 5 8 6 8 8 415 11 4 7 7 11 8 10 10 416 15 6 10 9 13 10 13 12 417 19 8 12 11 17 12 15 14 418 24 12 16 13 20 15 18 16 419 30 16 20 16 25 19 21 19 420 37 22 26 20 30 24 24 21 421 44 29 32 24 36 30 28 24 422 51 38 39 28 42 36 32 27 423 59 47 47 33 48 43 36 30 424 66 56 54 38 55 50 40 34 425 72 65 61 44 61 57 45 38 426 78 73 68 50 67 64 50 43 427 82 79 74 57 72 71 56 48 428 87 85 79 63 77 77 62 54 429 90 89 83 68 82 82 68 60 430 92 92 87 74 85 86 74 67 431 94 94 89 79 88 90 79 74 432 96 96 92 83 91 92 84 80 433 97 97 93 87 93 94 89 86 434 98 98 95 90 95 96 93 90 435 99 98 96 92 96 97 96 94 436 99 99 97 94 97 98 97 97 437 99 99 98 95 98 98 99 98 438 99 99 98 97 98 99 99 99 439 99 99 99 97 99 99 99 99 440 100 100 100 98 99 99 99 99 441 98 99 99 99 99 442 99 99 99 99 99 443 99 99 99 99 99 444 99 99 99 99 99 445 99 99 99 99 99 446 99 99 99 99 99 447 99 99 99 99 99 V.21 448 100 100 100 100 100 2017 Updates

Table V.12. National Average Scale Scores Using 2016 Norm Sample Subject Grade level English Mathematics Reading Science Writing 3 417 413 413 415 422 4 420 416 415 418 424 5 423 418 418 420 424 6 425 421 420 422 426 7 427 421 420 422 424 8 428 423 423 424 424 9 429 425 423 425 425 10 431 427 424 427 426

Table V.13. Percentage of Students Who Met Subject Benchmark in the 2016 Norm Sample Subject Grade level English Mathematics Reading Science Writing 3 72 60 38 37 17 4 70 55 42 41 21 5 73 51 39 44 26 6 73 57 47 47 44 7 75 43 41 41 28 8 74 37 50 40 29 9 61 36 44 32 44 10 62 32 38 34 52

V.22 2017 Updates

Section 2: Reliability for Spring 2016 ACT Aspire Forms This section presents the reliability estimate of Cronbach’s alpha for ACT Aspire test forms administered in spring 2016. Ranges of reliability estimates for each grade and subject are shown in Table V.14.

Table V.14. Ranges of Cronbach’s Alpha of ACT Aspire Forms Administered in Spring 2016 Cronbach’s Alpha Grade Subject Online Paper 3 English 0.78–0.83 0.81–0.84 Mathematics 0.85–0.88 0.86–0.86 Reading 0.83–0.85 0.83–0.84 Science 0.85–0.87 0.86–0.86 4 English 0.83–0.86 0.84–0.85 Mathematics 0.82–0.87 0.83–0.86 Reading 0.82–0.84 0.81–0.83 Science 0.84–0.86 0.85–0.86 5 English 0.80–0.82 0.82–0.82 Mathematics 0.83–0.86 0.81–0.85 Reading 0.80–0.83 0.81–0.81 Science 0.86–0.87 0.86–0.87 6 English 0.81–0.86 0.82–0.85 Mathematics 0.78–0.86 0.77–0.83 Reading 0.82–0.84 0.82–0.84 Science 0.87–0.89 0.87–0.88 7 English 0.84–0.86 0.85–0.87 Mathematics 0.83–0.86 0.81–0.85 Reading 0.81–0.84 0.82–0.82 Science 0.88–0.89 0.87–0.89 8 English 0.86–0.87 0.86–0.86 Mathematics 0.87–0.88 0.86–0.87 Reading 0.81–0.84 0.81–0.83 Science 0.88–0.89 0.87–0.89 EHS* English 0.90–0.91 0.91–0.91 Mathematics 0.86–0.89 0.88–0.88 Reading 0.79–0.86 0.82–0.86 Science 0.89–0.90 0.89–0.90

* EHS = Early High School

V.23 2017 Updates

Section 3: Changes to the Program for 2017 The ACT Aspire testing program is an effective and efficient measure of college and career readiness. The grade-level tests were designed to be given in a short amount of time to avoid the phenomenon of over-testing and allow for more valuable classroom learning experience. The program is also designed to allow for continuous improvement. The goal of ACT Aspire is to promote fluency in the College and Career Readiness Standards (CCRS), while optimizing the testing experience for students.

The only category of improvement that will be implemented in the spring 2017 operational administration is a change to testing time for math grades 6 and 7 and all grades of writing.

ACT Aspire assessments are timed tests, meaning that students have a certain amount of time allowed to complete their answers. Virtually all timed tests of academic achievement can be considered to be speeded to some degree, as students working at a very deliberate pace may feel they do not have sufficient time to carefully consider each test question.

Table V.15 shows the changes in testing time for the spring 2017 operational administration.

Table V.15. ACT Aspire Summative Testing Time Changes (in minutes) Math Math Writing Writing Grade English (New) (Old) Reading Science (New) (Old) 3 40 65 65 65 60 45 30 4 40 65 65 65 60 45 30 5 40 65 65 65 60 45 30 6 40 75 70 65 60 40 30 7 40 75 70 65 60 40 30 8 40 75 75 65 60 40 30 EHS 45 75 75 65 60 40 30

Note. NEW assessments are those administered in 2017; OLD assessments are those administered in 2016.

V.24 2018 Updates

This chapter presents updated information and statistical analyses based on data collected from recent operational administrations and a few program updates that will be implemented starting from fall 2017 and also from spring 2018. Specifically, Sections 1–2 of this chapter presents the updated national norms and updated ACT Readiness Benchmarks for ELA and STEM, which were implemented in reporting starting from the fall 2017 operational administration; Section 3 presents raw score reliabilities of forms administered in spring 2017; and Section 4 presents a program update that will be implemented starting from the fall 2017 and also from spring 2018 operational administration.

This section describes the development of ACT Aspire norms based on weighted data from tests administered through spring 2017; these norms are used for reporting starting from fall 2017. First, the inclusion rules for the samples are described, followed by a description of a weighting methodology designed to produce nationally representative norms. Then, the demographic information for the weighted samples, updated ACT Aspire norm tables, and summary statistics are presented.

ACT Aspire Norm Samples For grades 3–8, the norm samples included students who took ACT Aspire on-grade subject tests in spring 2015, spring 2016, and spring 2017, as consistent with the goal to include three years of rolling data.

For grades 9 and 10, the norm samples were restricted to students who took ACT Aspire in consecutive years. As described later, these longitudinal samples were W.1 used to anchor the grade 9 and grade 10 norm samples to the grade 8 and grade 9 2018 Updates

score distributions, respectively. The grade 9 samples included students who took grade 8 ACT Aspire and grade 9 ACT Aspire subject tests approximately one year apart, with the grade 9 test taken during the 2013–2014, 2014–2015, 2015–2016, or 2016–2017 academic years. Similarly, the grade 10 samples included students who took ACT Aspire in grade 9 and grade 10 approximately one year apart, with the grade 10 test taken during the 2013–2014, 2014–2015, 2015–2016, or 2016–2017 academic years. Table W.1 provides the sample sizes by grade level and subject area.

Table W.1. Sample Sizes Used for Norm Establishment

Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 228,246 337,428 336,854 213,523 204,897 213,248 4 223,648 330,498 330,373 211,524 207,392 211,198 5 220,025 323,521 322,106 289,658 205,851 288,438 6 233,698 340,480 339,884 223,213 215,128 222,563 7 253,703 354,786 353,552 318,229 232,495 316,188 8 275,554 373,398 372,399 261,125 239,633 245,972 259,697 9 83,763 84,606 83,725 80,005 63,502 79,293 76,712 10 245,195 245,718 243,819 237,974 207,065 235,755 230,907

Note: Composite score is available for grade 8 and above.

Weighting methodology for ACT Aspire Norms Students assessed with ACT Aspire are not representative of the national population of U.S. elementary and secondary school students. To support interpretations of nationally representative norms, weights were assigned to the ACT Aspire-tested samples so that the weighted samples are similar to the national population of U.S. elementary and secondary school students on school affiliation (public vs. private), and race/ethnicity and academic achievement among public school students.

For grades 3–8, we first estimated the population of U.S. elementary and secondary public school students with respect to race/ethnicity and district mean achievement level. Then, the public school ACT Aspire-tested students were weighted to match on these characteristics. Finally, the weighted public school data was combined with the private school data with the final weighting reflecting the frequency of school affiliation (public or private) in the population. An important assumption of this approach is that a district’s mean achievement level can be measured by its mean ACT Composite score1, and that this measure reflects achievement in the lower grade levels. To examine this assumption, correlations between district mean ACT Composite score and district mean ACT Aspire scores in the lower grade levels were examined (Table W.2). Across subject areas and grade levels, district mean ACT Composite scores are highly correlated with district mean ACT Aspire score. The correlations in Table W.2 suggest that district mean ACT Composite score reflects achievement in lower grade levels.

W.2 1 When a district tests virtually all students with the ACT, as is often done with ACT state and district testing, a district’s mean ACT Composite score measures the mean achievement level of the district’s graduating students. 2018 Updates

Table W.2. Correlation of District Mean ACT Composite Scores with District Mean ACT Aspire Score Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 0.56 0.56 0.64 0.53 0.58 0.53 4 0.53 0.60 0.63 0.53 0.50 0.54 5 0.56 0.61 0.63 0.61 0.60 0.63 6 0.56 0.55 0.59 0.52 0.58 0.50 7 0.59 0.71 0.62 0.66 0.60 0.69 8 0.69 0.72 0.61 0.64 0.67 0.61 0.69 9 0.65 0.70 0.62 0.65 0.68 0.66 0.69 10 0.75 0.77 0.68 0.73 0.77 0.73 0.77

Note: Districts with at least 50 ACT Aspire-tested students are included.

The weighting procedure for grades 3–8 is described in detail below. In what follows, target population refers to the population of U.S. elementary and secondary school students. The following steps were taken to weight each ACT Aspire norming sample2:

1. Determine the target population’s relative frequency of students enrolled in public and private schools.3 2. Among public school students within the target population, determine the relative frequency by race/ethnicity and school percent eligible for free or reduced lunch (school FRL). Table W.3 shows the categorizations of these variables and relative frequencies in the public school target population4. 3. Identify all public school districts that administer the ACT test to all 11th grade students; we refer to ACT-tested students within these districts as the ACT state and district testing public school sample. Weight this sample to the public school target population on race/ethnicity and school FRL. Weights were determined by the ratio of the population and sample percentages of each combination of race/ ethnicity and school FRL: Population % (Race = x, FRL = y) weight (Race = x, FRL = y) = Sample % (Race = x, FRL = y)

2 There is one norming sample for each grade level and subject area. 3 Estimated from Market Data Retrieval (MDR) data, the target population includes 90.6% public-school and 9.4% private-school enrollees. 4 The public school target population relative frequencies for race/ethnicity and school FRL were derived from the W.3 Common Core of Public School Universe Survey Data provided by the National Center for Education Statistics. 2018 Updates

We refer to this weighted sample as the ACT public school population. The ACT public school population was nearly identical to the public school target population with respect to relative frequencies of race/ethnicity and school FRL groups (Table W.3). 4. For each student in the ACT public school population, determine the mean ACT Composite score within their district. Categorize district mean ACT score5 and student race/ethnicity, and calculate the relative frequency of each combination of race/ethnicity and district mean ACT Composite score level (DMACT) within the ACT public school population. 5. For each ACT Aspire norm sample, determine public/private school affiliation and, for public school students, race/ethnicity and DMACT. For students enrolled in public schools, calculate the sample percentage for each combination of race/ ethnicity and DMACT. Weight the sample to the target population on public/ private school affiliation and, for public school students, race/ethnicity and DMACT. Weights were determined by the ratio of the population and sample percentages:

Public weight (Race = x, DMACT = y) = Pop.% public 3 ACT Public Pop.% (Race = x, DMACT = y) Sample % public Public Sample % (Race = x, DMACT = y)

Private weight = Pop.% public Sample % private

For an example norm sample (grade 5 English), the relative frequencies of the weighting variables are provided in Table W.4 for the target population, the ACT public school population, the unweighted sample, and the weighted sample. The weighted sample was identical to the target population with respect to school affiliation, and the public school sample was identical to the ACT public school population on race/ethnicity and district mean ACT Composite score. As shown in Table W.4, the weighted mean ACT Aspire grade 5 English score was higher than the unweighted mean. This was expected because the unweighted district mean ACT Composite score was lower than the target population’s district mean ACT Composite score.

W.4 5 See Table W.3 for the categorization of district mean ACT Composite score. 2018 Updates

Table W.3. Target Public School Population and ACT Public School Population67 Population ACT state and district testing of public public school sample U.S. elementary and secondary ACT public school Variable students Unweighted population6 Race/ethnicity Asian 4.8% 3.1% 4.8% Black/African American 15.5% 14.0% 15.5% Hispanic 25.5% 15.3% 25.5% Native American 1.0% 0.9% 1.0% White 49.8% 53.0% 49.8% Unknown/Other7 3.4% 13.7% 3.4% FRL % of enrolled school 0–10% 6.6% 3.9% 6.6% 10–20% 8.8% 11.5% 8.8% 20–35% 14.7% 22.9% 14.7% 35–50% 16.9% 23.9% 16.9% 50–65% 16.6% 20.1% 16.6% 65–80% 15.0% 10.7% 15.0% 80–90% 9.7% 4.4% 9.7% 90–100% 10.8% 2.4% 10.8% Unknown 1.0% 0.2% 1.0% District mean ACT score <16.72 4.5% 9.8% 16.72–17.46 8.2% 10.2% 17.46–18.07 8.7% 9.8% 18.07–18.67 10.3% 10.5% 18.67–19.19 10.9% 9.4% 19.19–19.44 7.8% 10.0% 19.44–20.09 12.9% 10.0% 20.09–20.97 13.4% 10.1% 20.97–22.38 13.4% 10.1% >22.38 10.0% 10.0% Mean (SD) district mean ACT score 19.7 (2.5) 19.1 (2.8) Mean (SD) ACT Composite score 19.7 (5.3) 19.3 (5.3)

Note: Shading represents variables used in weighting.

6 ACT state and district testing public school sample weighted to national population of U.S. elementary and secondary students on race/ethnicity and school FRL. 7 For the public school target population, the relative frequency for Unknown/Other is obtained by subtracting the sum of the individual race/ethnicity counts from total enrollment counts. For the ACT data, the unknown/other category includes Native Hawaiian/Other Pacific Islander, two or more races, prefer not to respond, and no response. W.5 2018 Updates

Table W.4. An Example of ACT Aspire Norm Sample Weighting: Grade 5 English8

Population of ACT Aspire norm sample: U.S. elementary and ACT public school grade 5 English Variable secondary students population8 Unweighted Weighted Sex Female 48.6% 50.3% 48.6% 48.9% Male 51.4% 48.6% 50.6% 50.7% Unknown 1.1% 0.8% 0.4% School affiliation Public 90.6% 85.3% 90.6% Private 9.4% 14.7% 9.4% Public school frequencies Race/ethnicity Asian 4.8% 4.8% 1.9% 4.8% Black/African American 15.5% 15.5% 28.0% 15.5% Hispanic 25.5% 25.5% 9.0% 25.5% Native American 1.0% 1.0% 0.7% 1.0% White 49.8% 49.8% 52.7% 49.8% Unknown/Other 3.4% 3.4% 7.6% 3.4% FRL % of enrolled school 0–10% 6.6% 6.6% 1.1% 1.1% 10–20% 8.8% 8.8% 2.2% 2.4% 20–35% 14.7% 14.7% 7.7% 17.4% 35–50% 16.9% 16.9% 14.0% 13.3% 50–65% 16.6% 16.6% 25.7% 22.3% 65–80% 15.0% 15.0% 22.1% 19.2% 80–90% 9.7% 9.7% 11.6% 11.3% 90–100% 10.8% 10.8% 9.7% 8.1% Unknown 1.0% 1.0% 6.0% 4.9% District mean ACT score <16.72 9.8% 11.5% 9.8% 16.72–17.46 10.2% 8.8% 10.2% 17.46–18.07 9.8% 15.1% 9.8% 18.07–18.67 10.5% 12.6% 10.5% 18.67–19.19 9.4% 14.6% 9.4% 19.19–19.44 10.0% 7.4% 10.0% 19.44–20.09 10.0% 8.9% 10.0% 20.09–20.97 10.1% 12.4% 10.1%

Note: Shading represents variables used in weighting.

8 ACT state and district testing public school sample weighted to national population of U.S. elementary and secondary students on race/ethnicity and school FRL.

W.6 2018 Updates

Table W.4. (continued)

Population of ACT Aspire norm sample: U.S. elementary and ACT public school grade 5 English Variable secondary students population8 Unweighted Weighted 20.97–22.38 10.1% 8.2% 10.1% >22.38 10.0% 0.7% 10.0% Mean (SD) district mean ACT score 19.1 (2.8) 18.7(1.7) 19.2(2.0) Mean (SD) ACT Aspire score 423.0(7.1) 423.2(7.1)

Note: Shading represents variables used in weighting.

To illustrate the weighting process for grade 9, the relative frequencies of the weighting variables are provided for mathematics for the target population, the ACT public school population, the grade 8 weighted sample, the grade 9 unweighted sample, and the grade 9 weighted sample (Table W.5). The weighted sample is identical to the grade 8 weighted sample with respect to grade 8 ACT Aspire mathematics score categories. In turn, the grade 8 weighted sample is identical to the target population with respect to school affiliation, and the public school sample is identical to the ACT public school population on race/ethnicity and district mean ACT Composite score.

W.7 2018 Updates

Table W.5. An Example of ACT Aspire Norm Sample Weighting: Grade 9 Mathematics9

ACT Aspire norm ACT Aspire norm sample: Population of U.S. ACT public sample: grade 8 grade 9 Mathematics elementary and school Mathematics, Variable secondary students population9 Weighted Unweighted Weighted Sex Female 48.6% 50.3% 49.1% 50.4% 50.2% Male 51.4% 48.6% 50.4% 49.0% 49.2% Unknown 1.1% 0.5% 0.7% 0.7% School affiliation Public 90.6% 90.6% 84.5% 84.6% Private 9.4% 9.4% 12.4% 12.3% Unknown 3.1% 3.1% Grade 8 ACT Aspire score range 400–403 0.1% 0.1% 0.1% 404–406 0.4% 0.4% 0.4% 407–409 1.8% 1.9% 1.8% 410–412 5.3% 4.6% 5.3% 413–415 10.3% 10.5% 10.3% 416–418 14.2% 14.7% 14.2% 419–421 15.1% 14.5% 15.1% 422–424 13.5% 14.5% 13.5% 425–427 10.9% 11.8% 10.9% 428–430 8.5% 8.9% 8.5% 431–433 6.7% 7.1% 6.7% 434–436 5.3% 4.9% 5.3% 437–439 4.0% 3.7% 4.0% 440–442 2.5% 1.8% 2.5% 443–445 1.1% 0.5% 1.1% 446–448 0.3% 0.1% 0.3% 449–451 0.0% 0.0% 0.0% 452–454 0.0% 0.0% 0.0% 455–456 0.0% 0.0% 0.0% Mean (SD) grade 8 423.2 (8.3) 422.9 (7.8) 423.2 (8.3) ACT Aspire score

9 ACT state and district testing public school sample weighted to national population of U.S. elementary and secondary students on race/ethnicity and school FRL. W.8 2018 Updates

Table W.5. (continued) ACT Aspire norm ACT Aspire norm sample: Population of U.S. ACT public sample: grade 8 grade 9 Mathematics elementary and school Mathematics, Variable secondary students population9 Weighted Unweighted Weighted Public school frequencies Race/ethnicity Asian 4.8% 4.8% 4.8% 3.0% 3.0% Black/African 11.5% 11.4% American 15.5% 15.5% 15.5% Hispanic 25.5% 25.5% 25.5% 10.1% 9.9% Native American 1.0% 1.0% 1.0% 0.7% 0.7% White 49.8% 49.8% 49.8% 43.8% 43.9% Unknown/Other 3.4% 3.4% 3.4% 27.4% 27.5% FRL % of enrolled school 0–10% 6.6% 6.6% 4.1% 1.4% 1.4% 10–20% 8.8% 8.8% 2.9% 4.4% 4.6% 20–35% 14.7% 14.7% 12.8% 6.4% 6.4% 35–50% 16.9% 16.9% 13.3% 15.5% 15.5% 50–65% 16.6% 16.6% 23.4% 23.7% 23.6% 65–80% 15.0% 15.0% 20.8% 20.2% 20.1% 80–90% 9.7% 9.7% 6.8% 8.4% 8.3% 90–100% 10.8% 10.8% 7.6% 7.6% 7.5% Unknown 1.0% 1.0% 8.3% 2.8% 2.9% District mean ACT score <16.72 9.8% 9.8% 10.1% 10.1% 16.72–17.46 10.2% 10.2% 4.5% 4.4% 17.46–18.07 9.8% 9.8% 6.9% 6.9% 18.07–18.67 10.5% 10.5% 14.3% 14.3% 18.67–19.19 9.4% 9.4% 14.9% 14.9% 19.19–19.44 10.0% 10.0% 5.5% 5.5% 19.44–20.09 10.0% 10.0% 7.8% 7.8% 20.09–20.97 10.1% 10.1% 13.6% 13.6% 20.97–22.38 10.1% 10.1% 11.0% 11.1% >22.38 10.0% 10.0% 2.6% 2.7% Unknown 8.8% 8.9% Mean (SD) district 19.1 (2.8) 19.4 (2.3) 19.1 (1.8) 19.2 (1.8) mean ACT score Mean (SD) grade 9 424.5 (8.3) 424.8 (8.6) ACT Aspire score

W.9 2018 Updates

Figure W.1 summarizes the weighting procedure for the ACT Aspire norms. Tables W.6–W.12 present the demographic information on the weighted samples for each grade and subject test.

Norm Tables and Summary Statistics The norm table includes the cumulative percentages of students scoring at or below each scale score in the weighted norm sample. A smoothing process was applied to the empirical cumulative percentages to reduce the sampling error and increase the precision of the norms. The norm tables are presented in Tables W.13–W.19 for each subject respectively. The mean scale score values for students in each grade and subject test are given in Table W.20. These represent estimated mean scores if all students in the target population (all U.S. elementary and secondary school students) were to take the test. For the same target population, Table W.21 also shows the estimated percentages of students meeting ACT Readiness Benchmark in each grade and subject test.

Population of ACT Aspire ACT Aspire U.S. elementary ACT public ACT Aspire grade 8–9 grade 9–10 and secondary school grade 3–8 norm longitudinal longitudinal students population samples samples samples

School affiliation School affiliation (public or (public or private) = = private)

Race/ethnicity =Race/ethnicity = Race/ethnicity (public) (public) (public)

School FRL % = School FRL % (public) (public)

District mean = District mean ACT score ACT score (public) (public)

Grade 8 = Grade 8 score score distribution distribution

Grade 9 = Grade 9 score score distribution distribution

Grade 10 score distribution

Note: Variables used for weighting are included in boxes.

Figure W.1. Conceptual diagram of chain-linked weighting strategy W.10 2018 Updates

Table W.6. 2017 ACT Aspire Weighted English Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 228,246) 223,648) 220,025) 233,698) 253,703) 275,554) 83,763) 245,195) Gender F 49.01 48.90 48.91 49.06 49.03 48.98 50.37 50.29 M 50.63 50.73 50.73 50.43 50.54 50.51 49.18 49.08 No response 0.36 0.38 0.36 0.51 0.44 0.51 0.46 0.62 State AL 24.49 24.68 24.46 22.69 27.31 25.76 0.50 0.18 AR 40.82 40.28 39.82 37.68 27.79 26.82 60.61 26.07 AS <0.01 0.01 0.01 0.01 <0.01 <0.01 - - AZ 0.07 0.10 0.08 0.10 0.42 0.12 1.10 <0.01 CA 1.12 1.21 1.21 4.38 4.47 2.95 1.23 1.54 CO 0.01 0.01 0.02 0.44 0.54 0.66 4.07 1.93 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.11 DC ------0.16 0.05 FL 0.22 0.22 0.23 0.29 0.28 0.31 0.01 1.83 GA 0.04 0.05 0.06 0.08 0.06 0.09 0.10 0.03 GU 0.16 0.11 0.11 0.17 0.10 0.09 0.13 0.13 HI - - - 0.01 0.04 2.22 2.72 2.50 IA 0.02 0.04 0.02 0.03 0.03 0.29 0.11 0.02 IL 3.00 3.08 3.03 3.05 2.96 2.82 1.81 1.97 IN - - - 0.22 0.27 0.28 1.30 0.47 KS 1.53 1.44 1.61 2.18 2.37 4.07 5.27 2.24 KY 0.05 0.07 0.08 0.12 0.27 0.28 0.65 0.19 LA 3.66 3.50 3.23 2.92 2.63 2.76 4.63 1.70 MA 0.01 0.01 <0.01 <0.01 <0.01 <0.01 - 0.02 MD 0.04 0.04 0.04 0.06 0.05 0.05 0.01 0.01 ME ------<0.01 MI 0.16 0.18 0.18 0.17 3.08 2.82 0.77 0.75 MN 0.01 0.02 0.02 0.04 0.90 0.06 0.38 0.14 MO 0.03 0.02 0.02 0.04 0.48 0.22 0.78 0.43 MP 0.39 0.38 0.38 0.33 0.27 0.16 - - MS 0.25 0.27 0.28 0.27 0.47 0.71 2.55 0.66 MT - - - 0.01 0.02 0.02 0.10 0.05 NC 0.06 0.06 0.06 0.06 0.05 0.05 0.04 0.03 ND - - - 0.01 <0.01 0.03 0.02 0.35 NE <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.13 1.12 NJ - - - - <0.01 0.01 - 0.05 NV 0.20 0.38 0.22 0.29 0.20 0.56 - 0.01

Note: - indicates no students tested. W.11 2018 Updates

Table W.6. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 228,246) 223,648) 220,025) 233,698) 253,703) 275,554) 83,763) 245,195) NY 0.01 0.02 0.01 0.03 0.04 0.08 0.14 0.20 OH - 0.19 0.22 0.21 0.28 0.55 2.00 1.36 OK 0.29 0.35 0.77 0.72 0.80 0.81 0.49 0.23 OR 0.01 0.01 0.01 0.02 0.02 0.03 0.01 0.36 PA 0.11 0.11 0.12 0.10 0.10 0.09 0.08 0.59 SC 22.44 22.31 22.66 22.06 20.92 18.79 2.24 0.55 SD 0.06 0.10 0.13 0.12 0.12 0.09 0.05 0.08 TN 0.19 0.20 0.26 0.43 0.41 0.45 0.75 0.36 TX 0.19 0.28 0.23 0.26 0.58 1.86 0.73 0.54 UT 0.29 0.16 0.30 0.17 0.28 0.16 <0.01 - VA <0.01 0.04 0.03 0.01 0.05 0.18 0.01 <0.01 WA - - - - - <0.01 0.24 0.09 WI 0.04 0.05 0.06 0.18 1.32 2.67 3.66 48.35 WY ------0.41 2.70 No response 0.01 0.01 0.01 0.01 0.01 0.01 - - Race/Ethnicity Asian 4.47 4.51 4.51 4.51 4.51 4.49 2.91 3.71 Black/ African 14.43 14.43 14.42 14.43 14.41 14.35 11.48 8.51 American Hispanic/ 23.90 23.86 23.80 23.77 23.78 23.70 9.94 11.70 Latino Native 0.94 0.94 0.94 0.95 0.95 0.95 0.69 0.73 American Other 1.40 1.21 1.27 0.96 0.46 0.52 3.62 3.13 White 47.62 47.68 47.80 47.71 47.65 47.50 43.31 59.31 Unknown 7.23 7.36 7.27 7.67 8.24 8.48 28.04 12.91

Note: - indicates no students tested.

W.12 2018 Updates

Table W.7. 2017 ACT Aspire Weighted Mathematics Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 337,428) 330,498) 323,521) 340,480) 354,786) 373,398) 84,606) 245,718) Gender F 49.06 49.11 48.76 48.87 48.81 49.09 50.17 49.95 M 50.61 50.54 50.91 50.69 50.79 50.44 49.15 49.42 No response 0.33 0.35 0.33 0.44 0.40 0.48 0.68 0.63 State AL 48.77 48.61 47.67 45.25 44.69 43.11 0.64 0.18 AR 23.29 23.17 23.81 22.12 20.41 20.04 59.38 25.90 AS <0.01 <0.01 <0.01 0.01 <0.01 <0.01 - - AZ 0.08 0.10 0.08 0.10 0.39 0.12 0.89 0.01 CA 1.09 1.19 1.19 4.30 4.39 2.90 1.98 1.58 CO 0.01 0.01 0.01 0.44 0.45 0.70 4.13 1.96 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.10 DC ------0.22 0.05 FL 0.20 0.21 0.23 0.25 0.26 0.30 0.01 1.80 GA 0.04 0.05 0.05 0.07 0.06 0.09 0.10 0.03 GU 0.16 0.10 0.11 0.17 0.10 0.09 0.09 0.14 HI - - - 0.01 0.03 1.90 2.69 2.55 IA 0.02 0.04 0.02 0.03 0.03 0.23 0.11 0.09 IL 2.97 3.05 3.00 3.01 2.91 2.78 2.03 1.98 IN - - - 0.21 0.23 0.23 1.20 0.46 KS 1.26 1.19 1.33 1.70 1.97 3.60 5.17 2.30 KY 0.05 0.07 0.08 0.12 0.27 0.28 0.64 0.18 LA 3.54 3.38 3.11 2.83 2.54 2.67 4.55 1.65 MA <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.02 MD 0.04 0.04 0.04 0.05 0.05 0.05 0.01 0.01 ME ------<0.01 MI 0.16 0.18 0.18 0.18 0.91 0.86 0.76 0.73 MN 0.01 0.01 0.02 0.03 0.34 0.06 0.39 0.14 MO 0.03 0.03 0.02 0.04 0.42 0.17 0.77 0.42 MP 0.30 0.29 0.30 0.30 0.26 0.15 - - MS 0.25 0.26 0.27 0.26 0.42 0.60 2.48 0.67 MT - - - 0.01 0.01 0.01 0.10 0.05 NC 0.05 0.04 0.04 0.05 0.04 0.04 0.04 0.03 ND - - - 0.01 <0.01 0.02 0.02 0.34 NE <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.37 1.11 NJ - - - - <0.01 0.01 - 0.05 NV 0.59 0.74 0.34 0.90 0.72 0.46 <0.01 0.01

Note: - indicates no students tested. W.13 2018 Updates

Table W.7. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 337,428) 330,498) 323,521) 340,480) 354,786) 373,398) 84,606) 245,718) NY 0.01 0.01 0.01 0.03 0.04 0.07 0.15 0.20 OH - 0.12 0.14 0.14 0.39 0.60 1.95 1.35 OK 0.23 0.28 0.60 0.58 0.67 0.68 0.47 0.21 OR 0.01 <0.01 0.01 0.01 0.02 0.03 0.01 0.38 PA 0.11 0.11 0.11 0.10 0.10 0.09 0.09 0.64 SC 16.00 15.95 16.09 15.56 15.05 13.68 2.24 0.56 SD 0.05 0.09 0.12 0.12 0.11 0.09 0.05 0.07 TN 0.17 0.19 0.26 0.40 0.38 0.43 0.73 0.35 TX 0.18 0.22 0.36 0.24 0.51 1.56 0.73 0.54 UT 0.29 0.16 0.29 0.18 0.28 0.16 <0.01 - VA <0.01 0.03 0.03 0.01 0.05 0.16 0.02 0.01 WA - - - - - <0.01 0.24 0.09 WI 0.04 0.04 0.06 0.15 0.48 0.90 4.05 48.34 WY ------0.51 2.73 No response 0.01 0.01 0.01 0.01 0.01 0.01 - - Race/Ethnicity Asian 4.47 4.52 4.51 4.51 4.51 4.49 3.04 3.74 Black/ African American 14.44 14.44 14.42 14.43 14.41 14.35 11.38 9.09 Hispanic/ Latino 23.90 23.86 23.80 23.77 23.78 23.70 9.93 12.12 Native American 0.94 0.94 0.94 0.95 0.95 0.95 0.68 0.77 Other 0.83 0.75 0.83 0.56 0.39 0.48 3.59 3.20 White 47.62 47.68 47.80 47.70 47.65 47.50 43.91 58.41 Unknown 7.79 7.82 7.69 8.08 8.32 8.53 27.48 12.66

Note: - indicates no students tested.

W.14 2018 Updates

Table W.8. 2017 ACT Aspire Weighted Reading Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n= (n = 336,854) 330,373) 322,106) 339,884) 353,552) 372,399) 83,725) 243,819) Gender F 49.07 49.12 48.79 48.87 48.84 49.07 50.84 50.52 M 50.60 50.52 50.88 50.68 50.76 50.45 48.54 48.84 No response 0.33 0.35 0.33 0.45 0.40 0.48 0.62 0.63 State AL 48.70 48.58 47.72 45.19 44.73 42.96 0.65 0.18 AR 23.33 23.18 23.86 22.14 20.52 20.05 59.59 25.57 AS <0.01 0.01 <0.01 0.01 <0.01 <0.01 - - AZ 0.07 0.10 0.08 0.10 0.39 0.12 0.89 0.02 CA 1.09 1.19 1.19 4.30 4.39 2.91 1.99 1.53 CO 0.01 0.01 0.01 0.44 0.31 0.71 4.12 1.96 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.09 DC ------0.15 0.06 FL 0.20 0.21 0.23 0.25 0.26 0.30 0.01 1.77 GA 0.04 0.05 0.05 0.07 0.06 0.09 0.10 0.03 GU 0.16 0.10 0.11 0.17 0.10 0.09 0.10 0.13 HI - - - 0.01 0.03 1.91 2.66 2.51 IA 0.02 0.04 0.02 0.03 0.03 0.24 0.11 0.09 IL 2.97 3.05 3.00 3.01 2.92 2.79 2.04 1.97 IN - - - 0.22 0.24 0.24 1.20 0.49 KS 1.27 1.20 1.31 1.69 1.98 3.59 5.12 2.28 KY 0.05 0.07 0.08 0.12 0.27 0.28 0.52 0.19 LA 3.55 3.38 3.11 2.85 2.55 2.67 4.74 1.69 MA <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.02 MD 0.04 0.04 0.04 0.05 0.05 0.05 0.01 0.01 ME ------<0.01 MI 0.16 0.18 0.18 0.18 0.91 0.93 0.75 0.74 MN 0.01 0.01 0.02 0.03 0.34 0.06 0.38 0.14 MO 0.03 0.03 0.02 0.04 0.43 0.18 0.79 0.43 MP 0.31 0.29 0.30 0.30 0.26 0.15 - - MS 0.25 0.26 0.27 0.26 0.42 0.59 2.51 0.66 MT - - - 0.01 0.01 0.01 0.10 0.05 NC 0.05 0.04 0.04 0.05 0.04 0.04 0.04 0.03 ND - - - 0.01 <0.01 0.02 0.02 0.35 NE <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.38 1.14 NJ - - - - <0.01 0.01 - 0.05 NV 0.59 0.74 0.34 0.92 0.73 0.45 <0.01 0.01

Note: - indicates no students tested. W.15 2018 Updates

Table W.8. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n= (n = 336,854) 330,373) 322,106) 339,884) 353,552) 372,399) 83,725) 243,819) NY 0.01 0.01 0.01 0.03 0.04 0.07 0.14 0.20 OH - 0.12 0.15 0.14 0.41 0.63 1.98 1.37 OK 0.24 0.29 0.62 0.59 0.58 0.68 0.50 0.22 OR 0.01 <0.01 0.01 0.01 0.02 0.03 0.01 0.39 PA 0.11 0.11 0.11 0.10 0.10 0.09 0.10 0.59 SC 15.99 15.94 16.13 15.58 15.04 13.69 2.24 0.56 SD 0.05 0.09 0.12 0.12 0.11 0.09 0.05 0.06 TN 0.17 0.19 0.25 0.40 0.38 0.43 0.76 0.35 TX 0.18 0.24 0.22 0.24 0.52 1.59 0.40 0.41 UT 0.29 0.16 0.29 0.17 0.28 0.16 <0.01 - VA <0.01 0.04 0.03 0.01 0.05 0.16 0.03 0.01 WA - - - - - <0.01 0.24 0.09 WI 0.04 0.04 0.06 0.15 0.48 0.90 3.99 48.78 WY ------0.57 2.77 No response 0.01 0.01 0.01 0.01 0.01 0.01 - - Race/Ethnicity Asian 4.47 4.52 4.51 4.51 4.51 4.49 2.98 3.76 Black/ African American 14.44 14.44 14.42 14.43 14.41 14.35 11.28 8.47 Hispanic/ 23.80 23.77 23.78 23.70 9.91 11.71 Latino 23.90 23.86 Native 0.94 0.95 0.95 0.95 0.68 0.75 American 0.94 0.94 Other 0.84 0.75 0.84 0.56 0.40 0.49 3.58 3.13 White 47.62 47.68 47.80 47.71 47.65 47.51 44.22 59.88 Unknown 7.79 7.81 7.68 8.06 8.31 8.51 27.34 12.30

Note: - indicates no students tested.

W.16 2018 Updates

Table W.9. 2017 ACT Aspire Weighted Science Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 213,523) 211,524) 289,658) 223,213) 318,229) 261,125) 80,005) 237,974) Gender F 49.11 48.93 48.82 49.09 48.77 49.06 50.34 50.00 M 50.53 50.68 50.83 50.39 50.81 50.43 48.94 49.37 No response 0.36 0.38 0.35 0.51 0.42 0.51 0.72 0.63 State AL 37.54 38.66 53.40 34.77 49.88 33.08 0.53 0.18 AR 39.89 39.03 27.13 37.82 23.64 29.59 58.94 25.14 AS 0.01 0.01 0.01 0.01 <0.01 0.01 - - AZ 0.07 0.10 0.08 0.11 0.43 0.13 0.94 <0.01 CA 1.14 1.22 1.44 4.58 4.51 3.00 1.81 1.35 CO 0.01 0.01 0.01 0.48 0.46 0.64 4.30 1.96 CT <0.01 <0.01 0.01 <0.01 <0.01 <0.01 0.01 0.04 DC ------0.11 0.03 FL 0.23 0.23 0.23 0.28 0.27 0.29 0.01 1.79 GA 0.03 0.04 0.06 0.07 0.06 0.09 0.11 0.03 GU ------HI - - - 0.01 0.04 2.51 2.80 2.33 IA 0.02 0.04 0.03 0.03 0.03 0.28 0.11 0.09 IL 3.10 3.15 3.07 3.14 3.00 2.89 2.16 2.02 IN - - - 0.23 0.26 0.28 1.24 0.49 KS 1.62 1.55 1.49 2.27 2.25 4.28 5.34 2.32 KY 0.05 0.07 0.08 0.12 0.28 0.29 0.69 0.19 LA 3.72 3.53 3.19 2.98 2.64 2.81 4.91 1.71 MA <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.02 MD 0.02 0.02 0.02 0.04 0.03 0.03 0.01 0.01 ME ------<0.01 MI 0.14 0.15 0.15 0.20 0.83 2.12 0.70 0.71 MN 0.02 0.02 0.02 0.04 0.39 0.07 0.40 0.14 MO 0.03 0.02 0.02 0.05 0.50 0.22 0.82 0.42 MP 0.35 0.34 0.32 0.33 0.26 0.16 - - MS 0.25 0.27 0.27 0.27 0.44 0.71 2.63 0.64 MT - - - 0.01 0.02 0.02 0.11 0.05 NC 0.06 0.06 0.04 0.07 0.04 0.05 0.04 0.03 ND - - - 0.01 <0.01 0.02 0.02 0.37 NE <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.39 1.15 NJ - - - - <0.01 <0.01 - 0.05 NV 0.10 0.15 0.09 0.19 0.15 0.49 - 0.01

Note: - indicates no students tested. W.17 2018 Updates

Table W.9. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 213,523) 211,524) 289,658) 223,213) 318,229) 261,125) 80,005) 237,974) NY 0.01 0.01 0.01 0.03 0.04 0.08 0.15 0.20 OH - 0.18 0.15 0.21 0.43 1.06 2.10 1.40 OK 0.30 0.39 0.66 0.80 0.73 0.89 0.51 0.21 OR 0.01 0.01 0.01 0.02 0.02 0.03 0.01 0.37 PA 0.11 0.12 0.12 0.11 0.10 0.09 0.10 0.61 SC 10.32 9.70 6.86 9.49 6.38 8.97 2.33 0.56 SD 0.06 0.10 0.12 0.12 0.11 0.09 0.05 0.08 TN 0.21 0.24 0.28 0.44 0.40 0.46 0.78 0.36 TX 0.20 0.30 0.22 0.26 0.55 1.89 0.43 0.41 UT 0.30 0.17 0.30 0.18 0.29 0.17 <0.01 - VA <0.01 0.05 0.03 0.02 0.06 0.19 0.01 <0.01 WA - - - - - <0.01 0.21 0.07 WI 0.04 0.05 0.06 0.19 0.47 2.02 3.75 49.72 WY ------0.42 2.72 No response 0.01 0.01 0.01 0.01 0.01 0.01 - - Race/Ethnicity Asian 4.47 4.52 4.51 4.52 4.51 4.49 3.06 3.79 Black/ African 14.43 14.44 14.42 14.36 8.82 American 14.44 14.45 11.74 Hispanic/ 23.82 23.79 23.79 23.71 11.66 Latino 23.93 23.88 10.35 Native 0.94 0.95 0.95 0.95 0.76 American 0.94 0.94 0.71 Other 1.10 0.95 0.73 0.70 0.34 0.48 3.69 3.17 White 47.65 47.70 47.82 47.78 47.71 47.57 45.67 60.46 Unknown 7.46 7.57 7.74 7.82 8.28 8.44 24.78 11.33

Note: - indicates no students tested.

W.18 2018 Updates

Table W.10. 2017 ACT Aspire Weighted Composite Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = 239,633) 76,712) 230,907) Gender F 48.95 50.42 49.97 M 50.53 49.15 49.39 No response 0.53 0.42 0.64 State AL 28.50 0.53 0.18 AR 32.29 60.03 24.86 AS 0.01 - - AZ 0.13 0.96 <0.01 CA 3.00 1.32 1.33 CO 0.65 4.47 1.87 CT <0.01 0.01 0.02 DC - 0.06 0.03 FL 0.29 0.01 1.80 GA 0.09 0.11 0.03 GU - - - HI 2.55 2.80 2.20 IA 0.27 0.12 0.02 IL 2.92 1.87 2.02 IN 0.29 1.20 0.43 KS 4.33 5.31 2.19 KY 0.29 0.55 0.20 LA 2.85 4.92 1.66 MA <0.01 - 0.02 MD 0.03 0.01 0.01 ME - - <0.01 MI 2.59 0.71 0.70 MN 0.07 0.42 0.15 MO 0.24 0.83 0.42 MP 0.16 - - MS 0.74 2.64 0.64 MT 0.02 0.11 0.05 NC 0.05 0.04 0.03 ND 0.03 0.03 0.33 NE 0.01 0.14 1.16 NJ <0.01 - 0.05 NV 0.52 - 0.01

Note: - indicates no students tested. Composite score is available for grade 8 and above. W.19 2018 Updates

Table W.10. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = 239,633) 76,712) 230,907) NY 0.08 0.15 0.20 OH 0.52 2.01 1.38 OK 0.91 0.49 0.20 OR 0.03 0.02 0.36 PA 0.09 0.09 0.56 SC 9.84 2.40 0.55 SD 0.10 0.05 0.06 TN 0.47 0.76 0.35 TX 1.97 0.42 0.41 UT 0.16 <0.01 - VA 0.20 0.01 <0.01 WA <0.01 0.22 0.07 WI 2.70 3.81 50.67 WY - 0.39 2.76 No response 0.01 - - Race/Ethnicity Asian 4.49 3.10 3.79 Black/ African 14.36 12.15 8.70 American Hispanic/ 23.72 10.33 11.60 Latino Native 0.95 0.73 0.76 American Other 0.48 3.69 3.10 White 47.58 46.07 61.34 Unknown 8.42 23.93 10.71

Note: - indicates no students tested. Composite score is available for grade 8 and above.

W.20 2018 Updates

Table W.11. 2017 ACT Aspire Weighted ELA Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 204,897) 207,392) 205,851) 215,128) 232,495) 245,972) 63,502) 207,065) Gender F 49.74 49.31 49.22 49.51 48.86 49.13 51.07 50.83 M 49.99 50.40 50.50 50.11 50.78 50.36 48.52 48.90 No response 0.28 0.30 0.28 0.38 0.36 0.52 0.41 0.28 State AL 22.04 22.16 21.95 20.72 21.15 20.35 0.56 0.18 AR 43.06 42.48 41.70 39.66 29.72 29.23 66.75 24.08 AS <0.01 0.01 0.01 0.01 <0.01 <0.01 - - AZ 0.07 0.10 0.08 0.11 0.47 0.14 1.11 <0.01 CA 1.15 1.23 1.26 4.32 4.50 3.08 1.56 1.60 CO 0.01 0.02 0.02 0.15 0.13 0.42 <0.01 0.52 CT <0.01 <0.01 0.01 <0.01 0.01 <0.01 0.01 <0.01 DC ------0.16 0.05 FL 0.22 0.23 0.25 0.32 0.29 0.34 0.02 2.00 GA 0.05 0.06 0.06 0.08 0.06 0.10 0.13 0.03 GU ------HI - - - 0.01 0.04 0.99 1.16 1.56 IA 0.02 0.02 0.02 0.02 0.02 0.06 <0.01 - IL 3.22 3.28 3.24 3.30 3.18 3.09 2.14 1.91 IN - - - 0.24 0.30 0.30 1.50 0.51 KS 1.50 1.40 1.53 1.61 1.78 3.50 4.89 1.88 KY 0.05 0.07 0.08 0.13 0.28 0.32 0.70 0.13 LA 3.72 3.60 3.23 3.08 2.71 2.69 4.31 1.33 MA 0.01 0.01 0.01 0.01 <0.01 <0.01 - 0.02 MD 0.02 0.02 0.02 0.02 0.02 0.02 0.01 0.01 ME ------MI 0.17 0.19 0.19 0.18 0.28 0.32 0.84 0.68 MN 0.02 0.02 0.02 0.03 2.86 0.07 0.49 0.16 MO 0.03 0.02 0.02 0.05 0.48 0.23 0.73 0.31 MP 0.36 0.36 0.37 0.35 0.29 0.19 - - MS 0.15 0.14 0.17 0.14 0.16 0.46 1.04 0.32 MT - - - 0.01 0.02 0.02 0.12 0.06 NC 0.04 0.03 0.03 0.04 0.03 0.03 0.05 0.03 ND - - - 0.01 <0.01 0.03 0.03 0.01 NE <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.12 0.07 NJ - - - - <0.01 <0.01 - 0.05 NV 0.14 0.21 0.10 0.19 0.15 0.44 - 0.01

Note: - indicates no students tested. W.21 2018 Updates

Table W.11. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 204,897) 207,392) 205,851) 215,128) 232,495) 245,972) 63,502) 207,065) NY 0.01 0.02 0.02 0.04 0.05 0.09 0.19 0.23 OH - 0.20 0.22 0.21 0.29 0.50 1.82 0.89 OK 0.10 0.27 0.82 0.87 0.83 0.91 0.57 0.23 OR 0.01 0.01 0.01 0.02 0.02 0.04 0.02 0.40 PA 0.12 0.12 0.12 0.11 0.11 0.10 0.11 0.47 SC 22.91 22.98 23.45 22.80 21.92 20.25 2.85 0.61 SD 0.06 0.10 0.13 0.13 0.13 0.10 - 0.05 TN 0.16 0.17 0.22 0.37 0.36 0.39 0.92 0.36 TX 0.20 0.22 0.23 0.27 0.57 1.96 0.48 0.45 UT 0.31 0.17 0.32 0.18 0.30 0.18 - - VA <0.01 0.02 0.01 0.01 0.05 0.05 0.02 <0.01 WA - - - - - <0.01 0.31 0.10 WI 0.05 0.05 0.06 0.18 6.43 9.00 3.86 55.72 WY ------0.42 2.98 No response 0.01 0.01 0.01 0.01 0.01 0.01 - - Race/Ethnicity Asian 4.47 4.52 4.51 4.52 4.52 4.51 2.59 3.46 Black/ African 14.43 14.43 14.42 14.45 14.43 14.38 13.69 8.64 American Hispanic/ 23.94 23.90 23.85 23.82 23.85 23.78 11.08 11.93 Latino Native 0.94 0.94 0.94 0.95 0.92 0.92 0.83 0.78 American Other 1.52 1.37 1.42 1.07 0.52 0.57 3.49 2.86 White 47.63 47.73 47.79 47.83 47.75 47.61 48.72 63.01 Unknown 7.07 7.11 7.07 7.35 8.01 8.23 19.61 9.31

Note: - indicates no students tested.

W.22 2018 Updates

Table W.12. 2017 ACT Aspire Weighted STEM Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 213,248) 211,198) 288,438) 222,563) 316,188) 259,697) 79,293) 235,755) Gender F 49.12 48.94 48.81 49.10 48.77 49.07 50.23 49.89 M 50.52 50.68 50.84 50.38 50.80 50.41 49.04 49.48 No response 0.36 0.38 0.35 0.52 0.42 0.52 0.72 0.63 State AL 37.51 38.65 53.41 34.72 49.79 33.24 0.53 0.18 AR 39.92 39.07 27.31 37.90 23.75 29.68 59.30 25.13 AS 0.01 0.01 0.01 0.01 <0.01 0.01 - - AZ 0.07 0.10 0.08 0.11 0.43 0.13 0.94 <0.01 CA 1.14 1.22 1.21 4.57 4.51 3.01 1.78 1.36 CO 0.01 0.01 0.01 0.48 0.45 0.60 4.36 1.92 CT <0.01 <0.01 0.01 <0.01 <0.01 <0.01 0.01 0.04 DC ------0.12 0.03 FL 0.23 0.23 0.24 0.28 0.27 0.29 0.01 1.78 GA 0.03 0.04 0.06 0.07 0.06 0.09 0.11 0.03 GU ------HI - - - 0.01 0.04 2.49 2.80 2.33 IA 0.02 0.04 0.03 0.03 0.03 0.27 0.11 0.09 IL 3.10 3.15 3.09 3.14 3.01 2.90 2.14 2.04 IN - - - 0.23 0.26 0.28 1.22 0.46 KS 1.62 1.55 1.49 2.27 2.25 4.24 5.30 2.29 KY 0.05 0.07 0.08 0.12 0.28 0.29 0.68 0.19 LA 3.73 3.52 3.19 2.98 2.64 2.81 4.81 1.67 MA <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.02 MD 0.02 0.02 0.02 0.04 0.03 0.03 0.01 0.01 ME ------<0.01 MI 0.14 0.15 0.15 0.20 0.83 1.94 0.69 0.70 MN 0.02 0.02 0.02 0.04 0.39 0.07 0.41 0.15 MO 0.03 0.02 0.02 0.05 0.49 0.22 0.81 0.42 MP 0.35 0.34 0.32 0.33 0.26 0.16 - - MS 0.25 0.27 0.27 0.27 0.44 0.71 2.60 0.64 MT - - - 0.01 0.02 0.02 0.11 0.05 NC 0.06 0.06 0.04 0.07 0.04 0.05 0.04 0.03 ND - - - 0.01 <0.01 0.03 0.03 0.35 NE <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.39 1.14 NJ - - - - <0.01 <0.01 - 0.05 NV 0.10 0.15 0.09 0.19 0.15 0.49 - 0.01 W.23 2018 Updates

Table W.12. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 213,248) 211,198) 288,438) 222,563) 316,188) 259,697) 79,293) 235,755) NY 0.01 0.01 0.01 0.03 0.04 0.08 0.15 0.20 OH - 0.18 0.15 0.21 0.40 1.02 2.01 1.39 OK 0.30 0.38 0.65 0.78 0.73 0.88 0.49 0.20 OR 0.01 0.01 0.01 0.02 0.02 0.03 0.02 0.36 PA 0.11 0.12 0.12 0.11 0.10 0.09 0.09 0.60 SC 10.33 9.70 6.91 9.51 6.41 8.98 2.33 0.56 SD 0.06 0.10 0.10 0.12 0.11 0.09 0.05 0.08 TN 0.21 0.24 0.29 0.45 0.40 0.46 0.75 0.35 TX 0.19 0.27 0.23 0.26 0.55 1.88 0.42 0.41 UT 0.30 0.17 0.29 0.18 0.29 0.16 <0.01 - VA <0.01 0.05 0.03 0.01 0.06 0.19 0.01 <0.01 WA - - - - - <0.01 0.21 0.07 WI 0.04 0.05 0.06 0.19 0.47 2.07 3.78 49.97 WY ------0.38 2.73 No response 0.01 0.01 0.01 0.01 0.01 0.01 - - Race/Ethnicity Asian 4.47 4.52 4.51 4.52 4.51 4.49 3.09 3.79 Black/ African 14.44 14.45 14.43 14.44 14.42 14.36 11.87 9.00 American Hispanic/ 23.93 23.88 23.82 23.79 23.79 23.72 10.36 11.80 Latino Native 0.94 0.94 0.94 0.95 0.95 0.95 0.71 0.77 American Other 1.11 0.95 0.73 0.70 0.34 0.48 3.68 3.18 White 47.65 47.70 47.83 47.78 47.71 47.57 45.66 60.35 Unknown 7.46 7.56 7.73 7.82 8.28 8.43 24.63 11.10

W.24 2018 Updates

Table W.13. 2017 ACT Aspire English Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 2 1 1 1 1 1 1 1 406 3 1 1 1 1 1 1 1 407 5 2 1 1 2 1 1 1 408 8 3 1 2 2 2 2 2 409 12 5 2 3 3 2 2 2 410 16 7 3 4 4 3 3 3 411 22 9 5 5 5 4 4 4 412 27 13 7 7 6 5 6 5 413 33 16 9 9 7 6 7 6 414 39 21 11 11 9 8 9 7 415 45 25 15 13 11 10 11 9 416 51 31 18 16 13 12 13 10 417 57 36 22 19 15 14 15 12 418 62 42 26 23 17 16 17 14 419 67 48 31 27 20 19 20 16 420 72 53 36 31 23 22 23 18 421 76 59 41 35 27 26 26 20 422 80 65 47 40 30 29 29 23 423 84 70 52 44 34 33 32 26 424 87 75 58 49 38 37 36 28 425 90 79 63 54 42 41 39 31 426 92 83 68 58 46 45 42 34 427 94 87 73 62 51 49 46 37 428 96 90 77 67 55 53 49 40 429 97 92 81 71 59 58 53 43 430 98 95 84 74 64 62 56 46 431 99 96 88 78 68 66 60 49 432 99 97 90 81 72 70 63 53 433 99 98 92 84 76 74 66 56 434 99 99 94 87 79 77 69 59 435 100 99 96 89 82 80 73 62 436 99 97 91 85 83 76 66 437 99 98 93 88 86 79 69 438 100 99 94 91 88 81 72 W.25 2018 Updates

Table W.13. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 96 93 91 84 75 440 99 97 94 92 86 78 441 99 98 96 94 89 81 442 100 98 97 95 91 84 443 99 98 96 92 87 444 99 98 97 94 89 445 99 99 98 95 91 446 99 99 99 96 93 447 99 99 99 97 95 448 100 99 99 98 96 449 99 99 99 97 450 100 99 99 98 451 99 99 99 452 100 99 99 453 99 99 454 99 99 455 99 99 456 100 100

W.26 2018 Updates

Table W.14. 2017 ACT Aspire Mathematics Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 3 1 1 1 1 1 1 1 405 4 1 1 1 1 1 1 1 406 6 1 1 1 2 1 1 1 407 9 2 1 1 3 1 1 1 408 13 3 2 1 4 1 1 1 409 17 5 3 2 5 2 2 2 410 23 8 6 4 7 4 3 2 411 30 13 9 5 10 5 4 4 412 38 18 13 8 13 8 6 5 413 48 26 18 10 16 10 8 7 414 57 34 25 14 20 14 11 10 415 66 43 32 18 25 18 14 12 416 75 53 40 23 29 22 18 15 417 82 62 48 28 35 27 22 19 418 88 70 56 34 40 32 26 22 419 92 77 63 40 45 37 31 26 420 95 83 70 47 51 42 35 29 421 97 88 75 53 56 47 40 33 422 98 91 80 59 61 52 44 37 423 99 94 84 65 65 56 48 40 424 99 96 87 70 69 61 53 44 425 99 97 90 75 73 65 57 48 426 99 98 92 79 77 68 60 51 427 99 99 94 83 80 71 64 55 428 99 99 95 87 83 75 68 58 429 99 99 96 89 85 77 71 61 430 99 99 97 92 88 80 74 64 431 99 99 98 93 90 82 77 68 432 99 99 98 95 92 85 80 71 433 99 99 99 96 93 87 82 74 434 100 99 99 97 94 89 85 77 435 99 99 98 96 90 87 80 436 99 99 98 97 92 89 82 437 99 99 99 97 94 91 85 438 99 99 99 98 95 93 87 W.27 2018 Updates

Table W.14. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 99 96 94 90 440 100 99 99 99 97 96 92 441 99 99 99 98 97 94 442 99 99 99 99 98 95 443 99 99 99 99 98 97 444 99 99 99 99 99 98 445 99 99 99 99 99 99 446 100 99 99 99 99 99 447 99 99 99 99 99 448 99 99 99 99 99 449 99 99 99 99 99 450 99 99 99 99 99 451 100 99 99 99 99 452 99 99 99 99 453 100 99 99 99 454 99 99 99 455 99 99 99 456 100 99 99 457 99 99 458 99 99 459 99 99 460 100 100

W.28 2018 Updates

Table W.15. 2017 ACT Aspire Reading Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 4 2 1 1 1 1 1 1 405 8 4 2 1 1 1 1 1 406 13 6 3 2 1 1 1 1 407 19 9 5 4 2 1 2 2 408 26 13 7 6 4 2 3 3 409 32 18 10 8 6 4 4 4 410 38 23 14 11 8 5 6 6 411 44 28 19 14 11 7 9 8 412 50 34 24 18 14 9 11 11 413 55 39 29 22 17 12 15 13 414 61 45 34 26 21 15 18 16 415 66 51 40 30 25 18 21 19 416 71 57 45 34 29 21 25 22 417 76 62 51 39 33 24 29 25 418 81 68 56 44 38 28 33 28 419 86 73 61 49 42 32 36 31 420 89 78 66 53 48 36 40 34 421 93 83 71 58 53 40 44 38 422 95 87 76 64 59 45 48 41 423 97 90 80 69 64 50 52 45 424 98 93 84 73 70 55 56 48 425 99 95 88 78 76 60 60 52 426 99 97 91 83 81 66 64 56 427 99 98 94 86 85 71 69 61 428 99 99 96 90 89 76 73 65 429 100 99 97 93 93 81 77 70 430 99 98 95 95 85 81 75 431 100 99 97 97 89 85 80 432 99 98 98 92 89 85 433 99 99 99 95 92 89 434 100 99 99 97 95 92 435 99 99 98 97 95 436 100 99 99 98 97 437 99 99 99 99 438 100 99 99 99 W.29 2018 Updates

Table W.15. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 440 100 99 99 441 99 99 442 100 100

W.30 2018 Updates

Table W.16. 2017 ACT Aspire Science Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 3 1 1 1 1 1 1 1 405 6 3 1 1 1 1 1 1 406 9 4 2 2 2 1 1 1 407 13 6 3 3 3 1 1 1 408 18 9 4 4 4 2 2 2 409 23 12 6 5 5 3 2 2 410 28 15 8 7 7 5 3 3 411 33 19 11 9 10 6 5 5 412 37 23 14 12 12 8 7 6 413 42 27 17 15 15 11 9 8 414 47 32 21 18 19 13 11 10 415 52 36 25 21 22 16 14 13 416 57 42 29 25 26 19 17 15 417 61 47 34 29 29 22 20 18 418 66 52 39 33 33 26 24 21 419 71 58 44 37 37 30 27 24 420 75 63 50 42 41 33 31 28 421 80 69 56 47 45 37 35 31 422 84 74 62 52 49 41 39 34 423 88 79 68 57 53 46 43 37 424 91 83 73 62 57 50 47 41 425 94 87 78 68 62 54 51 44 426 96 90 83 73 66 59 55 47 427 98 93 87 77 70 63 58 51 428 99 95 91 82 74 68 62 54 429 99 96 93 86 79 72 65 57 430 99 98 95 89 82 76 69 60 431 99 98 97 92 86 80 72 63 432 99 99 98 94 89 84 76 67 433 100 99 99 96 92 87 79 70 434 99 99 97 95 90 82 73 435 99 99 98 97 92 85 77 436 100 99 99 98 94 88 80 437 99 99 99 96 90 83 438 100 99 99 97 92 86 W.31 2018 Updates

Table W.16. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 98 94 89 440 100 99 99 96 92 441 99 99 97 94 442 99 99 98 96 443 100 99 99 97 444 99 99 98 445 99 99 99 446 100 99 99 447 99 99 448 99 99 449 100 100

W.32 2018 Updates

Table W.17. 2017 ACT Aspire Composite Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 401 1 1 1 402 1 1 1 403 1 1 1 404 1 1 1 405 1 1 1 406 1 1 1 407 1 1 1 408 1 1 1 409 1 1 1 410 2 1 1 411 3 2 2 412 4 3 3 413 6 5 5 414 9 8 7 415 12 11 9 416 15 14 12 417 19 18 15 418 23 22 18 419 27 26 22 420 31 30 25 421 36 34 29 422 40 38 32 423 45 43 35 424 50 47 39 425 54 51 42 426 59 54 46 427 64 58 49 428 68 62 53 429 72 66 56 430 76 70 60 431 80 74 64 432 84 77 67 433 87 81 71 434 90 84 75 435 93 87 79 436 95 90 82 437 96 92 86 438 98 94 89 W.33

Note: Composite score is available for grade 8 and above. 2018 Updates

Table W.17. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 98 96 92 440 99 97 94 441 99 98 96 442 99 99 98 443 99 99 99 444 99 99 99 445 99 99 99 446 99 99 99 447 99 99 99 448 99 99 99 449 100 99 99 450 99 99 451 99 99 452 100 100

Note: Composite score is available for grade 8 and above.

W.34 2018 Updates

Table W.18. 2017 ACT Aspire ELA Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 1 1 1 1 1 1 1 1 406 1 1 1 1 1 1 1 1 407 1 1 1 1 1 1 1 1 408 2 1 1 1 1 1 1 1 409 5 1 1 1 1 1 1 1 410 8 3 2 2 2 1 1 1 411 12 4 3 3 3 2 2 2 412 17 7 5 4 4 3 4 3 413 23 11 8 6 6 4 5 5 414 30 15 11 8 9 6 7 6 415 37 20 14 11 11 8 9 8 416 44 26 19 14 14 11 12 10 417 52 32 24 18 18 14 15 13 418 59 39 29 22 22 17 18 15 419 66 47 35 26 26 21 21 18 420 72 54 41 31 30 25 25 21 421 78 62 48 36 35 30 29 24 422 83 69 54 41 41 35 33 27 423 87 76 61 47 46 40 37 30 424 91 82 67 53 52 45 41 34 425 94 87 73 59 58 51 46 37 426 96 91 78 65 64 57 50 41 427 98 94 83 70 69 62 55 45 428 99 96 87 76 75 68 59 50 429 99 98 91 80 80 73 64 54 430 99 99 93 85 84 78 69 59 431 99 99 96 89 88 83 74 64 432 99 99 97 92 91 87 78 69 433 99 99 98 94 94 90 82 74 434 99 99 99 96 96 93 86 79 435 100 99 99 97 97 95 90 84 436 100 99 98 98 97 93 88 437 99 99 99 98 95 91 438 99 99 99 99 97 94 439 100 99 99 99 98 96 440 99 99 99 99 98 441 99 99 99 99 99 W.35 2018 Updates

Table W.18. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 442 99 99 99 99 99 443 99 99 99 99 99 444 100 99 99 99 99 445 100 99 99 99 446 99 99 99 447 100 99 99 448 99 99 449 100 100

W.36 2018 Updates

Table W.19. 2017 ACT Aspire STEM Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 3 1 1 1 1 1 1 1 406 5 1 1 1 1 1 1 1 407 9 2 1 1 1 1 1 1 408 13 4 2 1 2 1 1 1 409 18 6 3 2 3 1 1 1 410 24 9 5 3 5 2 1 1 411 30 14 7 5 7 4 2 2 412 36 19 11 7 10 6 4 4 413 43 24 15 10 13 8 6 5 414 49 31 20 13 17 11 8 7 415 56 38 26 17 21 14 11 10 416 63 45 32 22 25 18 15 13 417 69 52 38 27 30 22 18 16 418 75 59 45 32 35 26 22 20 419 81 66 52 37 40 31 27 23 420 86 72 59 43 45 36 31 27 421 91 78 65 49 50 41 36 30 422 94 83 71 55 54 45 40 34 423 96 88 77 61 59 50 44 38 424 98 91 82 67 63 54 49 41 425 99 94 86 72 68 59 53 45 426 99 96 89 77 72 63 57 48 427 99 98 92 81 76 67 61 52 428 99 99 94 85 79 71 64 55 429 99 99 96 89 83 75 68 59 430 99 99 97 92 86 78 71 62 431 99 99 98 94 89 81 75 65 432 99 99 99 96 91 84 78 69 433 99 99 99 97 93 87 81 72 434 100 99 99 98 95 90 84 75 435 99 99 99 97 92 87 78 436 99 99 99 98 94 89 81 437 99 99 99 99 95 91 84 438 100 99 99 99 97 93 87 W.37 2018 Updates

Table W.19. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 98 95 90 440 99 99 99 98 96 92 441 99 99 99 99 98 94 442 100 99 99 99 98 96 443 99 99 99 99 97 444 99 99 99 99 98 445 99 99 99 99 99 446 100 99 99 99 99 447 99 99 99 99 448 100 99 99 99 449 99 99 99 450 99 99 99 451 100 99 99 452 99 99 453 99 99 454 99 99 455 100 100

W.38 2018 Updates

Table W.20. National Average Scale Scores Using 2017 Norm Sample

Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 417 414 413 415 417 415 4 420 416 415 418 420 417 5 423 418 418 420 422 419 6 425 421 420 422 424 422 7 427 421 420 422 424 422 8 428 423 423 424 425 425 424 9 429 425 423 426 426 426 425 10 431 427 424 427 427 428 427

Table W.21. Percentage of Students Who Met Subject Benchmark in the 2017 Norm Sample

Grade Subject area level English Mathematics Reading Science ELA STEM 3 72 61 40 39 41 19 4 69 57 43 42 38 22 5 73 52 39 44 39 18 6 73 60 47 48 41 19 7 77 44 42 42 42 17 8 75 39 50 41 43 16 9 61 36 44 34 45 16 10 63 32 39 37 46 19

W.39 2018 Updates

Section 2: Updating ELA and STEM Benchmarks The ACT Aspire ELA and STEM Benchmarks for grades 3–10 were updated in fall 2017 to better align with the ACT ELA and STEM Benchmarks. Full details on the methodology used to update the ELA and STEM Benchmarks are provided in an ACT technical brief (Allen & Fang, 2017), and the updated Benchmarks are provided in Table W.22 on the next page. In summary:

• The ELA and STEM Benchmarks for grades 8–10 are the scores associated with a 50% chance of meeting the respective ACT College Readiness Benchmark based on longitudinal data available through spring 2017. • The Benchmarks for grades 3–7 are approximately the same distance from the mean (in standard deviation units) as the grade 8 Benchmarks. Benchmarks for grades 3–7 are not linked directly to the ACT Benchmarks because 1) longitudinal data to the ACT test is not yet available for grades 3–7, and 2) the predictive power is weaker when the grades are further apart. • The updated STEM Benchmarks are substantially higher than the initial STEM Benchmarks. The STEM Benchmarks for grades 3–10 were initially set as the average of the Benchmarks for mathematics and science, but are now linked to the ACT STEM Benchmark, which reflects the greater difficulty of the math and science courses taken by students in STEM-related majors. • The updated ELA and STEM Benchmarks provide greater consistency across grade levels in percentage of students meeting the Benchmarks, and also agree more with average gain scores. • While two of the components of the ELA score (English and reading scores) are reported on a vertical scale, the writing score component is not. The ELA Benchmark still increases with grade level, with the exception of grade 6 to grade 7, where it is 426 for both grade levels. On average, students’ writing scores decrease by about 1.6 score points from grade 6 to grade 7. This contributes to a small average gain in ELA score of about 0.6 points from grade 6 to 7, and so having the same integer value of the ELA Benchmark from grade 6 to grade 7 is generally consistent with the small average gain.

W.40 2018 Updates

Table W.22. Updated ELA and STEM Readiness Benchmarks. Grade level ELA STEM 3 419 420 4 422 422 5 424 425 6 426 428 7 426 430 8 427 433 9 428 435 10 430 437 11 (ACT) 20 26

Reference

Allen, J., & Fang, Y. (2017). Updating ACT Aspire ELA and STEM Readiness Benchmarks for Grades 3–10. Iowa City, IA: ACT.

W.41 2018 Updates

Section 3: Reliability for Spring 2017 ACT Aspire Forms This section presents the reliability estimate of Cronbach’s alpha for ACT Aspire test forms administered in spring 2017. Ranges of reliability estimates for each grade and subject are shown in Table W.23.

Table W.23. Ranges of Cronbach’s Alpha of ACT Aspire Forms Administered in Spring 2017. Cronbach’s Alpha Grade Subject Online Paper 3 English 0.79–0.84 0.80–0.84 Mathematics 0.86–0.87 0.86–0.86 Reading 0.83–0.85 0.83–0.84 Science 0.85–0.87 0.86–0.86 4 English 0.83–0.87 0.84–0.86 Mathematics 0.82–0.87 0.83–0.87 Reading 0.82–0.85 0.81–0.83 Science 0.84–0.87 0.85–0.87 5 English 0.81–0.83 0.82–0.82 Mathematics 0.83–0.86 0.82–0.85 Reading 0.81–0.83 0.80–0.81 Science 0.85–0.87 0.86–0.87 6 English 0.80–0.86 0.83–0.84 Mathematics 0.80–0.86 0.79–0.84 Reading 0.83–0.84 0.82–0.84 Science 0.87–0.88 0.87–0.89 7 English 0.84–0.87 0.85–0.87 Mathematics 0.84–0.86 0.83–0.86 Reading 0.82–0.84 0.82–0.82 Science 0.88–0.90 0.88–0.89 8 English 0.83–0.87 0.86–0.87 Mathematics 0.85–0.88 0.86–0.88 Reading 0.80–0.85 0.81–0.83 Science 0.88–0.89 0.88–0.89 EHS* English 0.89–0.90 0.91–0.91 Mathematics 0.85–0.89 0.87–0.88 Reading 0.80–0.87 0.82–0.86 Science 0.90–0.90 0.89–0.90

* EHS = Early High School

W.42 2018 Updates

Section 4: Program Updates The ACT Aspire testing program is an effective and efficient measure of college and career readiness. The grade-level tests were designed to be given in a short amount of time to avoid the phenomenon of over-testing and allow for more valuable classroom learning experience. The program is also designed to allow for continuous improvement. The goal of ACT Aspire is to promote fluency in the College and Career Readiness Standards (CCRS), while optimizing the testing experience for students.

The following two categories of changes will be implemented in the spring 2018 operational administration:

1. Reducing the number of Constructed Response (CR) items in Mathematics, Reading and Science, replacing some CR items with Multiple Choice (MC) items. When replacing CR with MC items, item latency analysis is performed to ensure the test timing is controlled to avoid speededness. Secondly, items are added proportionally to reporting categories in order to collect sufficient and high quality evidence for content coverage. 2. English test: Introducing a mix of discrete items, paragraph-based items, and full essay-based items. This change has no effect on the total number of items or the scoring/point value of the items.

Reducing Constructed Response (CR) Items and Replacing with Multiple Choice (MC) Items The following is a summary of the CR item reductions described in the first category of changes:

Science Grades 3–7:

• Number of CR items decreases from 6 to 4 at each grade • Replace each 3-point CR with 3 MC/TE items • Number of dichotomously scored items (MC+TE) increases by 6

Grades 8 and EHS:

• Number of CR items decreases from 5 to 4 • Number of 3 point CRs decreases from 3 to 0 • Number of 2 point CRs increases from 2 to 4 • Number of dichotomously scored items (MC+TE) increases by 5

Summary: Every grade of Science will have 4 CR items.

W.43 2018 Updates

Reading Grades 3–EHS:

• Number of CR items decreases from 3 to 2 at all grades • Replace the 2-point CR with 1 MC (or TE) items • Number of dichotomously scored items (MC+TE) increases by 1 • The total number of points has gone down by 1

Summary: Every grade of Reading will have 2 CR items.

Mathematics Grades 6 & 7:

• Remove 1 CR from each grade, replacing each of these CR’s with 3 MC (or TE) items • Number of dichotomously scored items (SR+TE) increases by 3

Grades 8 & EHS:

• Remove 2 CR’s per grade, replacing each of these CR’s with 3 SR (or TE) items • Number of dichotomously scored items (SR+TE) increases by 6

Summary: Every grade of math will have 3 CR items.

Specification Tables The following tables show content and item type coverage for the changes made to the Science, Reading and Mathematics tests. They provide information as follows:

Science Table W.24 shows the increase in MC items and reduction of CR items by grade. Table W.25 shows the associated changes in point coverage by reporting category. Reading Table W.26 shows the increase in MC items and reduction of CR items by grade. Table W.27 shows the associated changes in point coverage by reporting category. Mathematics Table W.28 shows the increase in MC items and reduction of CR items by grade. Table W.29 shows the increase in points by reporting category for Mathematics MC items in grades 6–EHS, and a decrease in the Justification and Explanation (JE) category for these grades based on reducing by one CR item (equal to 4 points) Note: Grades 3 through 5 are not changed in spring 2018 and are included here for your reference.

W.44 2018 Updates

English Table W.30 shows the changes to the distribution of discrete, paragraph and essay- based items.

Table W.24. Science: Number of Items by Item Type Grades 3–5 Grades 6–7 Grades 8–EHS Item Type NEW OLD NEW OLD NEW OLD MC 25–26 22–23 29–30 22–23 29–30 23–24 TE 2–3 3–4 2–3 3–4 2–3 3–4 CR (2 pt) 4 4 4 4 4 2 CR (3 pt) 2 2 3 Total 32 32 36 32 36 32

MC–Multiple Choice; TE–Technology Enhanced; CR–Constructed Response. Paper and pencil tests do not have TE items, MC items are used in their place.

Table W.25. Science: Number of Points by Reporting Category Grades 3–5 Grades 6–7 Grade 8 Grade EHS Reporting Category NEW OLD NEW OLD NEW OLD NEW OLD IOD 18–22 14–19 20–22 19–20 18–20 17–19 16–18 17–19 SIN 7–9 8–11 6–10 9–11 8–10 9–11 10–12 9–11 EMI 7–9 9–13 10–12 9–12 12–14 11–12 12–14 11–12 Total 36 36 40 40 40 40 40 40

Abbreviation ACT Reporting Category IOD Interpretation of Data SIN Scientific Investigation EMI Evaluation of Models, Inferences and Experimental Results

Table W.26. Reading: Number of Items by Item Type Grades 3–7 Grades 8–EHS NEW OLD NEW OLD MC 19–21 18–20 19–21 20–21 TE 1–3 1–3 1–3 0–1 CR (2 pts) 1 1 CR (3 pts) 2 2 CR (4 pts) 2 2 Total 24 24 24 24

MC–Multiple Choice; TE–Technology Enhanced; CR–Constructed Response. Paper and pencil tests do not have TE items, MC items are used in their place.

W.45 2018 Updates

Table W.27. Reading: Number of Points by Reporting Category

Grades 3–7 Grades 8–EHS NEW OLD NEW OLD KID 13–17 14–18 13–18 17–18 C&S 6–10 7–11 6–11 7–11 IKI 4–5 4–6 5–6 4–6 Total 28 29 30 31

Abbreviation ACT Reporting Category KID Key Ideas and Details C&S Craft and Structure IKI Integration of Knowledge and Ideas

Table W.28. Mathematics: Number of Items by Item Type Grades 3–5 Grades 6–7 Grades 8–EHS Item Type NEW OLD NEW OLD MC 21–23 26–28 23–25 32–34 27–29 TE 4–6 5–7 5–7 5–7 5–7 CR 3 3 4 3 5 Total 30 36 34 42 38

MC–Multiple Choice; TE–Technology Enhanced; CR–Constructed Response. Paper and pencil tests do not have TE items, MC items are used in their place.

W.46 2018 Updates

Table W.29. Mathematics: Number of Points by Reporting Category

Grade 3 4 5 6 7 8 EHS Reporting Category NEW OLD NEW OLD NEW OLD NEW OLD Base 10 2–4 4–6 4–6 1–3 1–3 1–3 1–3 2–4 1–3 2–4 0–2 Fractions 3–5 4–6 4–9 1–3 1–3 1–3 1–3 2–4 1–3 2–4 0–2 NS 4–6 3–5 3–5 3–5 1–3 2–4 2–4 1–3 NQ 2–4 1–3 OAT 5–7 2–4 2–4 1–3 1–3 1–3 1–3 1–4 0–2 2–4 0–2 EE 5–7 3–5 5–7 3–5 7 5–7 2–4 2–4 RP 3–5 3–5 4–6 3–5 1–4 0–2 2–4 1–3 ALG 6–8 2–4 Functions 5–7 3–5 5–7 3–5 GEO 2–4 2–4 3–5 3–5 5–7 3–5 4–6 6–8 6–8 6–8 5–7 MD 3–5 2–4 2–4 1–3 1–3 0–2 0–2 0–2 0–2 0–2 0–2 SP 3–5 3–5 3–5 3–5 3–5 4–6 3–5 4–7 JE 12 12 12 12 16 12 16 12 20 12 20

Table W.30. English: Changes to stimulus type by grade Grades 3–5 Grades 6–8 EHS NEW OLD NEW OLD NEW OLD Discrete 14 13–15 15 13–15 14 13–15 Paragraph 7 0 10 0 12 0 Essay 10 16–18 10 20–22 24 35–37 Total 31 31 35 35 50 50

W.47 2018 Updates

Reporting for Writing Beginning with the fall 2017 administration, the score reports no longer provide an overall writing scale score. The writing reporting categories of “Ideas and Analysis,” “Development and Support,” “Organization,” and “Language Use and Conventions” continue to be reported on the Individual Student Report (ISR).

An overall writing scale score continues to be used, however, for the calculation of the ELA score. This writing scale score is obtained using the same method as in previous administrations of ACT Aspire. It is not reported and is used solely for the ELA score calculation, described in section 9.3 and section 2 of this update chapter.

W.48 2019 Updates

This chapter presents updated information and statistical analyses based on data collected from recent operational administrations. Specifically, Section 1 of this chapter presents the updated national spring norms in English, mathematics, reading, science, ELA, STEM, and Composite scores. Spring norms in the four subject tests are used for reporting students’ national percentile ranks in the score reports, regardless of whether students took the tests in fall or spring. Section 2 presents the fall norms based on data collected from fall testers only. The fall norms serve as an additional reference for fall testers. Section 3 presents ranges of raw score reliability estimates of standard forms administered in spring 2018. Section 4 presents the character limits for student responses on the ACT Aspire Writing Test.

Section 1: ACT Aspire Spring Norms ACT Aspire started with the goal of reporting three-year rolling user norms with equal weights to each student. Beginning with fall 2015 reporting, the norm data have been statistically weighted to more closely match a national distribution in terms of selected student demographics and achievement.

This section describes the development of ACT Aspire spring norms (used interchangeably with ACT Aspire norms hereafter) based on weighted data from tests administered through spring 2018; these spring norms are used for reporting starting from fall 2018. First, the inclusion rules for the samples are described, followed by a description of a weighting methodology designed to produce nationally representative norms. Then, the demographic information for the weighted samples, updated ACT Aspire spring norm tables, and summary statistics are presented.

X.1 2019 Updates

ACT Aspire Spring Norm Samples For grades 3–8, the spring norm samples included students who took ACT Aspire on-grade subject tests in spring 2016, spring 2017, or spring 2018 as consistent with the goal to include three years of rolling data.

For grades 9 and 10, the spring norm samples were restricted to students who took ACT Aspire in consecutive years. As described later, these longitudinal samples were used to anchor the grade 9 and grade 10 norm samples to the grade 8 and grade 9 score distributions, respectively. The grade 9 samples included students who took grade 8 ACT Aspire and grade 9 ACT Aspire subject tests approximately one year apart, with the grade 9 test taken during the 2013–2014, 2014–2015, 2015–2016, 2016–2017, or 2017–2018 academic years. Similarly, the grade 10 samples included students who took ACT Aspire in grade 9 and grade 10 approximately one year apart, with the grade 10 test taken during the same aforementioned academic years. Table X.1 provides the sample sizes by grade level and subject area.

Table X.1. Sample Sizes Used for Norm Establishment

Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 195,808 268,030 267,554 209,139 180,254 208,979 4 195,048 264,859 264,399 209,951 183,576 209,798 5 193,781 262,235 261,729 261,501 183,044 260,638 6 201,643 268,709 268,224 215,700 187,805 215,302 7 224,307 285,681 285,137 284,595 208,244 282,850 8 230,554 291,287 290,728 238,323 225,187 208,975 237,349 9 140,520 141,881 140,656 134,235 129,892 109,275 133,099 10 363,848 365,702 363,254 354,104 344,678 315,628 351,105 Note: Composite score is available for grade 8 and above.

Weighting methodology for ACT Aspire Spring Norms Students assessed with ACT Aspire are not representative of the national population of U.S. elementary and secondary school students. To support interpretations of nationally representative norms, weights were assigned to the ACT Aspire-tested samples so that the weighted samples are similar to the national population of U.S. elementary and secondary school students on school affiliation (public vs. private), and race/ethnicity and academic achievement among public school students.

X.2 2019 Updates

approach is that a district’s mean achievement level can be measured by its mean ACT Composite score,1 and that this measure reflects achievement in the lower grade levels. To examine this assumption, correlations between district mean ACT Composite score and district mean ACT Aspire scores in the lower grade levels were examined (Table X.2). Across subject areas and grade levels, district mean ACT Composite scores are highly correlated with district mean ACT Aspire score. The correlations in Table X.2 suggest that district mean ACT Composite score reflects achievement in lower grade levels.

Table X.2. Correlation of District Mean ACT Composite Scores with District Mean ACT Aspire Score Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 0.63 0.60 0.71 0.57 0.63 0.56 4 0.62 0.64 0.71 0.61 0.61 0.60 5 0.60 0.67 0.70 0.66 0.67 0.68 6 0.63 0.56 0.63 0.57 0.62 0.56 7 0.64 0.67 0.59 0.57 0.64 0.62 8 0.67 0.69 0.58 0.61 0.64 0.60 0.66 9 0.66 0.76 0.69 0.68 0.72 0.66 0.74 10 0.73 0.80 0.67 0.72 0.78 0.71 0.78 Note: Districts with at least 50 ACT Aspire-tested students are included.

1 When a district tests virtually all students with the ACT, as is often done with ACT state and district testing, a district’s mean ACT Composite score measures the mean achievement level of the district’s graduating students. 2 There is one norming sample for each grade level and subject area. 3 Estimated from Market Data Retrieval (MDR) data, the target population includes 90.5% public-school and 9.5% private-school enrollees. X.3 2019 Updates

(school FRL). Table X.3 shows the categorizations of these variables and relative frequencies in the public school target population.4 3. Identify all public school districts that administer the ACT test to all 11th grade students; we refer to ACT-tested students within these districts as the ACT state and district testing public school sample. Weight this sample to the public school target population on race/ethnicity and school FRL. Weights were determined by the ratio of the population and sample percentages of each combination of race/ ethnicity and school FRL: Population % (Race = x, FRL = y) weight (Race = x, FRL = y) = Sample % (Race = x, FRL = y) We refer We refer to this weighted sample as the ACT public school population. The ACT public school population was nearly identical to the public school target population with respect to relative frequencies of race/ethnicity and school FRL groups (Table X.3). 4. For each student in the ACT public school population, determine the mean ACT Composite score within their district. Categorize district mean ACT score5 and student race/ethnicity, and calculate the relative frequency of each combination of race/ethnicity and district mean ACT Composite score level (DMACT) within the ACT public school population. 5. For each ACT Aspire norm sample, determine public/private school affiliation and, for public school students, race/ethnicity and DMACT. For students enrolled in public schools, calculate the sample percentage for each combination of race/ ethnicity and DMACT. Weight the sample to the target population on public/private school affiliation and, for public school students, race/ethnicity and DMACT. Weights were determined by the ratio of the population and sample percentages:

Public weight (Race = x, DMACT = y) = Pop. % public 3 ACT Public Pop. % (Race = x, DMACT = y) Sample % public Public Sample % (Race = x, DMACT = y)

Private weight = Pop. % public Sample % private

For an example norm sample (grade 5 English), the relative frequencies of the weighting variables are provided in Table X.4 for the target population, the ACT public school population, the unweighted sample, and the weighted sample. The weighted sample was identical to the target population with respect to school affiliation, and the public school sample was identical to the ACT public school population on race/ ethnicity and district mean ACT Composite score. As shown in Table X.4, the weighted district mean ACT Composite score was higher than the unweighted district mean ACT Composite score, but the weighted mean ACT Aspire grade 5 English score was lower than the unweighted mean. This discrepancy shows that other weighting variables (i.e., race or school affiliation) likely affect the score distribution in a different direction.

4 The public school target population relative frequencies for race/ethnicity and school FRL were derived from the X.4 Common Core of Public School Universe Survey Data provided by the National Center for Education Statistics. 5 See Table X.3 for the categorization of district mean ACT Composite score. 2019 Updates

Table X.3. Target Public School Population and ACT Public School Population67 Population ACT state and district testing of public public school sample U.S. elementary and secondary ACT public school Variable students Unweighted population6 Race/ethnicity Asian 5.0% 3.0% 5.0% Black/African American 15.3% 14.5% 15.3% Hispanic 26.4% 12.3% 26.4% Native American 1.0% 0.9% 1.0% White 48.5% 53.8% 48.5% Unknown/Other7 3.8% 15.4% 3.8% FRL % of enrolled school 0–10% 6.4% 3.8% 6.4% 10–20% 8.5% 12.1% 8.5% 20–35% 14.4% 20.8% 14.4% 35–50% 16.8% 25.5% 16.8% 50–65% 16.6% 19.9% 16.6% 65–80% 14.1% 8.6% 14.1% 80–90% 9.5% 3.1% 9.5% 90–100% 11.0% 5.6% 11.0% Unknown 2.8% 0.7% 2.8% District mean ACT score <16.50 5.3% 9.7% 16.50–17.41 6.6% 10.1% 17.41–17.73 7.3% 10.1% 17.73–18.34 11.3% 10.6% 18.34–18.85 10.5% 9.9% 18.85–19.30 10.5% 9.6% 19.30–19.89 12.0% 9.9% 19.89–20.59 12.9% 10.0% 20.59–21.83 13.4% 10.0% >21.83 10.3% 10.0% Mean (SD) district mean ACT score 19.4 (2.4) 18.9 (2.8) Mean (SD) ACT Composite score 19.4 (5.4) 18.9 (5.4) Note: Shading represents variables used in weighting.

Table X.4. An Example of ACT Aspire Spring Norm Sample Weighting: Grade 5 English8

Population of ACT Aspire norm sample: U.S. elementary and ACT public school grade 5 English Variable secondary students population8 Unweighted Weighted Sex Female 48.6% 50.6% 49.0% 49.0% Male 51.4% 48.7% 50.4% 50.7% Unknown 0.7% 0.6% 0.3% School affiliation Public 90.5% 78.0% 90.5% Private 9.5% 22.0% 9.5% Public school frequencies Race/ethnicity Asian 5.0% 5.0% 1.5% 5.0% Black/African American 15.3% 15.3% 23.5% 15.3% Hispanic 26.4% 26.4% 11.3% 26.4% Native American 1.0% 1.0% 0.8% 1.0% White 48.5% 48.5% 56.7% 48.5% Unknown/Other 3.8% 3.8% 6.2% 3.8% FRL % of enrolled school 0–10% 6.4% 6.4% 0.9% 1.1% 10–20% 8.5% 8.5% 1.2% 1.5% 20–35% 14.4% 14.4% 8.6% 12.0% 35–50% 16.8% 16.8% 16.0% 16.4% 50–65% 16.6% 16.6% 25.4% 23.7% 65–80% 14.1% 14.1% 26.6% 24.1% 80–90% 9.5% 9.5% 7.3% 7.9% 90–100% 11.0% 11.0% 11.2% 9.7% Unknown 2.8% 2.8% 2.7% 3.6% District mean ACT score <16.50 9.9% 11.6% 9.9% 16.50–17.41 10.0% 8.1% 10.0% 17.41–17.73 10.1% 8.6% 10.1% 17.73–18.34 10.3% 11.8% 10.3% 18.34–18.85 9.6% 12.1% 9.6% 18.85–19.30 10.1% 12.3% 10.1% 19.30–19.89 10.0% 11.2% 10.0% 19.89–20.59 10.0% 12.9% 10.0% Note: Shading represents variables used in weighting.

8 ACT state and district testing public school sample weighted to national population of U.S. elementary and secondary students on race/ethnicity and school FRL.

X.6 2019 Updates

Table X.4. (continued)

Population of ACT Aspire norm sample: U.S. elementary and ACT public school grade 5 English Variable secondary students population8 Unweighted Weighted 20.59–21.83 10.0% 8.0% 10.0% >21.83 10.0% 3.4% 10.0% Mean (SD) district mean ACT score 18.9 (2.8) 18.7 (1.7) 18.9 (2.1) Mean (SD) ACT Aspire score 423.4 (7.0) 423.1 (7.0) Note: Shading represents variables used in weighting.

To illustrate the weighting process for grade 9, the relative frequencies of the weighting variables are provided for mathematics for the target population, the ACT public school population, the grade 8 weighted sample, the grade 9 unweighted sample, and the grade 9 weighted sample (Table X.5). The weighted sample is identical to the grade 8 weighted sample with respect to grade 8 ACT Aspire mathematics score categories. In turn, the grade 8 weighted sample is identical to the target population with respect to school affiliation, and the public school sample is identical to the ACT public school population on race/ethnicity and district mean ACT Composite score.

X.7 2019 Updates

Table X.5. An Example of ACT Aspire Spring Norm Sample Weighting: Grade 9 Mathematics9

ACT Aspire norm ACT Aspire norm sample: Population of U.S. ACT public sample: grade 8 grade 9 Mathematics elementary and school Mathematics, Variable secondary students population9 Weighted Unweighted Weighted Sex Female 48.6% 50.6% 49.0% 49.4% 49.2% Male 51.4% 48.7% 50.7% 50.3% 50.4% Unknown 0.7% 0.3% 0.4% 0.4% School affiliation Public 90.5% 90.5% 81.7% 84.6% Private 9.5% 9.5% 14.1% 12.3% Unknown 4.2% 3.1% Grade 8 ACT Aspire score range 400–403 0.1% 0.1% 0.1% 404–406 0.4% 0.4% 0.4% 407–409 1.9% 1.8% 1.9% 410–412 5.3% 4.7% 5.3% 413–415 9.9% 10.4% 9.9% 416–418 13.6% 13.7% 13.6% 419–421 14.6% 14.1% 14.6% 422–424 13.4% 14.3% 13.4% 425–427 11.2% 12.3% 11.2% 428–430 9.0% 8.8% 9.0% 431–433 7.2% 7.2% 7.2% 434–436 5.6% 5.3% 5.6% 437–439 4.0% 3.9% 4.0% 440–442 2.5% 2.2% 2.5% 443–445 1.1% 0.6% 1.1% 446–448 0.3% 0.2% 0.3% 449–451 0.0% 0.0% 0.0% 452–454 0.0% 0.0% 0.0% 455–456 0.0% 0.0% 0.0% Mean (SD) grade 8 423.4 (8.3) 423.2 (8.0) 423.4 (8.3) ACT Aspire score

9 ACT state and district testing public school sample weighted to national population of U.S. elementary and secondary students on race/ethnicity and school FRL.

X.8 2019 Updates

Table X.5. (continued) ACT Aspire norm ACT Aspire norm sample: Population of U.S. ACT public sample: grade 8 grade 9 Mathematics elementary and school Mathematics, Variable secondary students population9 Weighted Unweighted Weighted Public school frequencies Race/ethnicity Asian 5.0% 5.0% 5.0% 2.8% 2.9% Black/African 12.1% 12.1% American 15.3% 15.3% 15.3% Hispanic 26.4% 26.4% 26.4% 10.4% 10.4% Native American 1.0% 1.0% 1.0% 0.7% 0.7% White 48.5% 48.5% 48.5% 47.3% 47.3% Unknown/Other 3.8% 3.8% 3.8% 23.6% 23.6% FRL % of enrolled school 0–10% 6.4% 6.4% 1.5% 1.5% 1.5% 10–20% 8.5% 8.5% 5.4% 6.1% 6.3% 20–35% 14.4% 14.4% 13.8% 11.7% 11.8% 35–50% 16.8% 16.8% 20.2% 19.5% 19.5% 50–65% 16.6% 16.6% 25.7% 22.7% 22.6% 65–80% 14.1% 14.1% 17.8% 20.1% 20.0% 80–90% 9.5% 9.5% 4.5% 3.6% 3.6% 90–100% 11.0% 11.0% 9.0% 10.4% 10.4% Unknown 2.8% 2.8% 2.2% 4.2% 4.2% District mean ACT score <16.50 9.9% 9.9% 9.2% 9.2% 16.50–17.41 10.0% 10.0% 9.5% 9.4% 17.41–17.73 10.1% 10.1% 1.9% 1.9% 17.73–18.34 10.3% 10.3% 7.8% 7.7% 18.34–18.85 9.6% 9.6% 10.5% 10.5% 18.85–19.30 10.1% 10.1% 12.1% 12.0% 19.30–19.89 10.0% 10.0% 6.8% 6.8% 19.89–20.59 10.0% 10.0% 9.4% 9.4% 20.59–21.83 10.0% 10.0% 7.1% 7.1% >21.83 10.0% 10.0% 13.1% 13.3% Unknown 12.6% 12.7% Mean (SD) district 18.9 (2.8) 19.0 (2.2) 19.2 (2.2) 19.2 (2.2) mean ACT score Mean (SD) grade 9 424.7 (8.4) 424.8 (8.6) ACT Aspire score

X.9 2019 Updates

Figure X.1 summarizes the weighting procedure for the ACT Aspire norms. Tables X.6-X.12 present the demographic information on the weighted samples for each grade and subject test.

Spring Norm Tables and Summary Statistics

The spring norm table includes the cumulative percentages of students scoring at or below each scale score in the weighted norm sample from spring testers. A smoothing process was applied to the empirical cumulative percentages to reduce the sampling error and increase the precision of the norms. The spring norm tables are presented in Tables X.13-X.19 for each subject, respectively. The mean scale score values for students in each grade and subject test are given in Table X.20. These represent estimated mean scores if all students in the target population (all U.S. elementary and secondary school students) were to take the test. For the same target population, Table X.21 also shows the estimated percentages of students meeting ACT Readiness Benchmark in each grade and subject test.

School affiliation School affiliation (public or (public or private) = = private)

Race/ethnicity =Race/ethnicity = Race/ethnicity (public) (public) (public)

School FRL % = School FRL % (public) (public)

District mean = District mean ACT score ACT score (public) (public)

Grade 8 = Grade 8 score score distribution distribution

Grade 9 = Grade 9 score score distribution distribution

Grade 10 score distribution

Note: Variables used for weighting are included in boxes.

Figure X.1. Conceptual diagram of chain-linked weighting strategy X.10 2019 Updates

Table X.6. 2018 ACT Aspire Weighted English Spring Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 195,808) 195,048) 193,781) 201,643) 224,307) 230,554) 140,520) 363,848) Gender F 49.12 49.17 48.97 48.81 48.83 49.09 49.21 49.19 M 50.58 50.52 50.72 50.86 50.98 50.65 50.56 50.50 No response 0.30 0.32 0.30 0.33 0.19 0.26 0.23 0.31 State AL 23.41 23.41 22.96 22.41 23.38 23.57 1.03 0.37 AR 64.37 63.53 63.07 58.74 56.18 51.50 47.92 18.07 AS ------0.02 0.01 AZ 0.07 0.07 0.07 0.09 0.36 0.09 0.92 0.43 CA 0.83 0.89 1.63 4.45 4.24 3.29 1.23 2.40 CO 0.10 0.13 0.13 0.19 0.24 0.30 5.28 1.69 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.07 DC - - - 0.05 0.05 0.05 0.06 0.03 FL 0.15 0.16 0.18 0.29 0.31 0.31 0.82 2.40 GA 0.03 0.04 0.04 0.07 0.07 0.09 0.29 0.20 GU 0.17 0.12 0.11 0.20 0.14 0.14 4.10 1.39 HI - - - 0.02 0.11 2.73 1.46 1.33 IA 0.24 0.28 0.27 0.32 0.28 0.38 0.65 0.23 IL 3.43 3.39 3.33 3.20 3.12 2.98 2.91 2.80 IN - - - 0.23 0.13 0.33 1.44 0.27 KS 1.57 1.50 1.50 3.13 2.66 4.30 5.06 2.50 KY 0.06 0.06 0.08 0.09 0.23 0.23 0.84 0.22 LA 3.65 3.61 3.55 3.20 2.90 2.70 5.01 1.72 MA ------0.11 MD 0.03 0.03 0.03 0.04 0.04 0.05 0.17 0.15 ME ------MI 0.11 0.12 0.12 0.11 0.18 0.18 0.92 0.98 MN - - - 0.02 0.97 0.08 1.35 0.77 MO 0.03 0.03 0.03 0.04 0.10 0.08 1.75 0.87 MP ------1.41 0.48 MS 0.26 0.26 0.27 0.24 0.29 0.32 2.41 0.65 MT - - - 0.02 0.02 0.02 0.12 0.12 NC 0.04 0.04 0.04 0.04 0.03 0.04 0.05 0.06 ND <0.01 <0.01 <0.01 0.02 0.02 0.05 0.05 0.58 NE <0.01 <0.01 <0.01 0.01 <0.01 0.01 0.39 1.15 NH ------0.02 NJ - - - - 0.01 0.02 0.06 0.39 Note: - indicates no students tested. X.11 2019 Updates

Table X.6. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 195,808) 195,048) 193,781) 201,643) 224,307) 230,554) 140,520) 363,848) NM ------0.08 NV 0.08 0.08 0.08 0.08 0.09 0.08 0.53 0.19 NY <0.01 0.01 0.01 0.02 0.03 0.03 0.11 0.32 OH - 0.23 0.23 0.22 0.26 0.45 2.96 1.48 OK 0.22 0.61 1.00 0.99 1.01 1.03 0.81 0.31 OR <0.01 <0.01 <0.01 0.05 0.04 0.05 0.10 0.40 PA 0.14 0.15 0.14 0.12 0.12 0.11 0.14 0.56 SC 0.24 0.24 0.25 0.25 0.15 0.11 1.47 0.38 SD 0.07 0.09 0.10 0.09 0.14 0.08 0.05 0.06 TN 0.16 0.17 0.23 0.33 0.42 0.51 1.12 0.54 TX 0.11 0.42 0.13 0.18 0.61 1.95 1.01 0.50 UT 0.23 0.15 0.23 0.15 0.22 0.13 0.26 0.22 VA - - - - <0.01 0.01 0.06 0.24 VI ------0.05 0.01 WA ------0.05 WI 0.16 0.18 0.19 0.31 0.83 1.61 3.33 50.46 WV - - 0.01 - - 0.01 - - WY ------0.01 1.66 No response ------0.11 0.08 Race/Ethnicity Asian 4.63 4.63 4.65 4.66 4.65 4.64 2.74 3.48 Black/ African 14.29 14.28 14.25 14.25 14.23 14.22 12.27 9.20 American Hispanic/ 24.82 24.76 24.70 24.65 24.63 24.61 10.22 12.39 Latino Native 0.91 0.91 0.91 0.92 0.92 0.92 0.70 0.77 American Other 1.55 1.58 1.59 1.11 0.78 0.65 3.13 3.18 White 46.42 46.50 46.55 46.50 46.33 46.31 46.82 60.93 Unknown 7.38 7.34 7.34 7.93 8.47 8.65 24.12 10.05 Note: - indicates no students tested.

X.12 2019 Updates

Table X.7. 2018 ACT Aspire Weighted Mathematics Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n= (n = 268,030) 264,859) 262,235) 268,709) 285,681) 291,287) 141,881) 365,702) Gender F 49.13 49.21 48.94 48.82 48.76 48.99 49.24 48.94 M 50.61 50.51 50.79 50.90 51.05 50.75 50.40 50.74 No response 0.26 0.28 0.27 0.28 0.19 0.26 0.36 0.32 State AL 42.69 42.08 41.13 39.58 38.17 38.35 1.18 0.36 AR 45.94 45.91 46.01 43.71 43.52 39.39 47.35 18.14 AS ------0.02 0.01 AZ 0.07 0.07 0.07 0.09 0.36 0.09 0.80 0.40 CA 0.83 0.88 1.45 4.03 3.87 3.06 1.68 2.45 CO 0.10 0.13 0.13 0.18 0.24 0.29 5.27 1.69 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.06 DC - - - 0.04 0.03 0.03 0.10 0.03 FL 0.15 0.16 0.18 0.29 0.31 0.31 0.83 2.34 GA 0.03 0.04 0.04 0.07 0.07 0.09 0.29 0.21 GU 0.17 0.12 0.11 0.20 0.14 0.14 2.88 1.09 HI - - - 0.01 0.10 2.25 1.44 1.36 IA 0.15 0.18 0.17 0.22 0.20 0.31 0.64 0.29 IL 3.42 3.38 3.32 3.20 3.12 2.98 3.07 2.77 IN - - - 0.22 0.12 0.31 1.36 0.27 KS 1.12 1.10 1.23 2.10 2.05 3.43 5.01 2.57 KY 0.06 0.06 0.08 0.09 0.22 0.23 0.81 0.22 LA 3.42 3.39 3.34 3.01 2.66 2.51 4.98 1.66 MA ------0.11 MD 0.03 0.03 0.03 0.04 0.04 0.05 0.17 0.16 ME ------MI 0.11 0.12 0.12 0.11 0.18 0.18 0.93 0.95 MN - - - 0.02 0.60 0.08 1.36 0.75 MO 0.03 0.03 0.03 0.04 0.09 0.07 1.73 0.85 MP ------1.38 0.51 MS 0.26 0.25 0.26 0.24 0.28 0.31 2.37 0.66 MT - - - 0.02 0.02 0.02 0.12 0.12 NC 0.04 0.04 0.04 0.04 0.03 0.04 0.04 0.06 ND <0.01 <0.01 <0.01 0.01 0.01 0.04 0.05 0.56 NE <0.01 <0.01 <0.01 0.01 <0.01 0.01 0.53 1.14 NH ------0.02 NJ - - - - 0.01 0.02 0.06 0.39 Note: - indicates no students tested. X.13 2019 Updates

Table X.7. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n= (n = 268,030) 264,859) 262,235) 268,709) 285,681) 291,287) 141,881) 365,702) NM ------0.17 0.07 NV 0.08 0.08 0.08 0.08 0.09 0.08 0.87 0.58 NY <0.01 0.01 0.01 0.02 0.03 0.03 0.12 0.33 OH - 0.14 0.14 0.15 0.29 0.43 3.37 1.61 OK 0.20 0.52 0.78 0.79 0.82 0.87 0.80 0.29 OR <0.01 <0.01 <0.01 0.05 0.04 0.04 0.10 0.42 PA 0.14 0.15 0.14 0.12 0.12 0.11 0.14 0.59 SC 0.20 0.20 0.21 0.21 0.14 0.10 1.47 0.38 SD 0.07 0.09 0.10 0.09 0.14 0.08 0.05 0.06 TN 0.16 0.17 0.23 0.33 0.40 0.47 1.09 0.52 TX 0.12 0.32 0.14 0.19 0.59 1.89 1.09 0.48 UT 0.23 0.14 0.23 0.15 0.22 0.13 0.26 0.22 VA <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.07 0.23 VI ------0.05 0.01 WA ------0.05 WI 0.15 0.18 0.18 0.27 0.65 1.14 3.77 50.23 WV - - 0.01 - - 0.01 - - WY ------0.01 1.64 No response ------0.11 0.08 Race/Ethnicity Asian 4.63 4.63 4.66 4.66 4.65 4.64 2.85 3.49 Black/ African American 14.29 14.28 14.26 14.25 14.23 14.22 12.05 9.72 Hispanic/ Latino 24.82 24.76 24.70 24.65 24.63 24.61 10.37 12.78 Native American 0.91 0.91 0.91 0.92 0.92 0.92 0.70 0.80 Other 1.10 1.12 1.09 0.84 0.70 0.60 3.11 3.24 White 46.42 46.49 46.54 46.49 46.32 46.31 47.32 59.93 Unknown 7.83 7.80 7.85 8.21 8.55 8.70 23.61 10.05 Note: - indicates no students tested.

X.14 2019 Updates

Table X.8. 2018 ACT Aspire Weighted Reading Spring Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n= (n = 267,554) 264,399) 261,729) 268,224) 285,137) 290,728) 140,656) 363,254) Gender F 49.12 49.22 48.96 48.83 48.77 48.96 49.57 49.12 M 50.62 50.50 50.77 50.89 51.04 50.78 50.08 50.56 No response 0.26 0.28 0.28 0.28 0.19 0.26 0.35 0.31 State AL 42.59 41.99 41.04 39.50 38.06 38.25 1.20 0.36 AR 46.03 45.99 46.10 43.79 43.60 39.41 47.76 18.07 AS ------0.02 0.01 AZ 0.07 0.07 0.07 0.09 0.36 0.09 0.80 0.40 CA 0.83 0.88 1.45 4.03 3.86 3.06 1.69 2.45 CO 0.10 0.13 0.13 0.18 0.24 0.30 5.19 1.65 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.06 DC - - - 0.04 0.03 0.02 0.05 0.03 FL 0.15 0.16 0.18 0.29 0.31 0.31 0.84 2.35 GA 0.03 0.04 0.04 0.07 0.07 0.09 0.29 0.20 GU 0.17 0.12 0.11 0.20 0.14 0.14 2.83 1.05 HI - - - 0.01 0.10 2.25 1.42 1.35 IA 0.16 0.19 0.17 0.22 0.20 0.31 0.63 0.28 IL 3.42 3.38 3.32 3.20 3.13 2.99 3.07 2.79 IN - - - 0.22 0.13 0.32 1.36 0.28 KS 1.13 1.11 1.20 2.09 2.05 3.41 4.96 2.52 KY 0.06 0.06 0.08 0.09 0.23 0.23 0.77 0.22 LA 3.42 3.40 3.34 3.02 2.66 2.51 5.02 1.70 MA ------0.10 MD 0.03 0.03 0.03 0.04 0.04 0.05 0.18 0.16 ME ------MI 0.11 0.12 0.12 0.11 0.18 0.18 0.93 0.96 MN - - - 0.02 0.60 0.08 1.35 0.75 MO 0.03 0.03 0.03 0.04 0.09 0.07 1.77 0.86 MP ------1.39 0.50 MS 0.26 0.25 0.26 0.24 0.28 0.31 2.38 0.66 MT - - - 0.02 0.02 0.02 0.12 0.12 NC 0.04 0.04 0.04 0.04 0.03 0.04 0.05 0.07 ND <0.01 <0.01 <0.01 0.01 0.01 0.04 0.05 0.58 NE <0.01 <0.01 <0.01 0.01 <0.01 0.01 0.53 1.16 NH ------0.02 NJ - - - - 0.01 0.02 0.04 0.36 Note: - indicates no students tested. X.15 2019 Updates

Table X.8. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n= (n = 267,554) 264,399) 261,729) 268,224) 285,137) 290,728) 140,656) 363,254) NM ------0.12 NV 0.08 0.08 0.08 0.08 0.09 0.07 0.87 0.57 NY <0.01 0.01 0.01 0.02 0.03 0.03 0.12 0.14 OH - 0.15 0.14 0.14 0.32 0.48 3.46 1.65 OK 0.20 0.52 0.81 0.80 0.81 0.85 0.82 0.30 OR <0.01 <0.01 <0.01 0.05 0.04 0.04 0.10 0.43 PA 0.14 0.15 0.14 0.12 0.12 0.11 0.14 0.56 SC 0.20 0.20 0.21 0.21 0.14 0.11 1.39 0.38 SD 0.07 0.09 0.10 0.09 0.14 0.08 0.05 0.05 TN 0.16 0.17 0.23 0.33 0.40 0.47 1.14 0.53 TX 0.12 0.32 0.14 0.19 0.59 1.94 0.82 0.40 UT 0.23 0.14 0.23 0.15 0.22 0.13 0.26 0.22 VA <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.07 0.14 VI ------0.05 0.01 WA ------0.05 WI 0.15 0.18 0.18 0.27 0.66 1.15 3.76 50.64 WV - - 0.01 - - 0.01 - - WY ------0.01 1.66 No response ------0.09 0.08 Race/Ethnicity Asian 4.63 4.63 4.66 4.66 4.65 4.64 2.81 3.50 Black/ African American 14.29 14.28 14.26 14.25 14.23 14.22 12.05 9.26 Hispanic/ 24.70 24.65 24.63 24.61 10.35 12.47 Latino 24.82 24.76 Native 0.91 0.92 0.92 0.92 0.70 0.79 American 0.91 0.91 Other 1.10 1.12 1.11 0.84 0.71 0.60 3.12 3.20 White 46.42 46.49 46.54 46.49 46.33 46.31 47.49 60.98 Unknown 7.83 7.80 7.82 8.19 8.54 8.69 23.50 9.80 Note: - indicates no students tested.

X.16 2019 Updates

Table X.9. 2018 ACT Aspire Weighted Science Spring Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 209,139) 209,951) 261,501) 215,700) 284,595) 238,323) 134,235) 354,104) Gender F 49.08 49.15 48.95 48.91 48.75 49.16 49.20 48.93 M 50.63 50.54 50.78 50.76 51.06 50.58 50.41 50.75 No response 0.30 0.31 0.27 0.32 0.19 0.26 0.39 0.32 State AL 28.66 28.70 41.22 27.08 38.27 26.59 1.11 0.37 AR 59.46 58.59 45.95 54.72 43.45 48.88 50.03 18.65 AS ------0.02 0.01 AZ 0.07 0.07 0.07 0.09 0.36 0.09 0.85 0.41 CA 0.85 0.90 1.46 4.27 3.87 3.20 1.62 2.01 CO 0.11 0.13 0.13 0.19 0.25 0.30 5.55 1.70 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.02 DC - - - 0.03 0.01 0.08 0.07 0.02 FL 0.16 0.17 0.19 0.30 0.32 0.32 0.69 2.34 GA 0.03 0.04 0.04 0.07 0.07 0.09 0.30 0.19 GU ------HI - - - 0.01 0.10 2.69 1.52 1.37 IA 0.23 0.28 0.17 0.31 0.20 0.38 0.69 0.28 IL 3.54 3.46 3.40 3.30 3.20 3.07 3.21 2.85 IN - - - 0.23 0.12 0.33 1.41 0.28 KS 1.30 1.26 1.20 2.78 2.06 4.05 5.17 2.58 KY 0.06 0.07 0.08 0.09 0.23 0.24 0.89 0.23 LA 3.69 3.63 3.39 3.25 2.71 2.70 5.22 1.72 MA ------0.11 MD 0.01 0.01 0.01 0.01 0.02 0.01 0.10 0.11 ME ------MI 0.08 0.09 0.09 0.11 0.19 0.19 0.91 0.95 MN - - - 0.02 0.61 0.08 1.40 0.78 MO 0.03 0.03 0.03 0.04 0.09 0.08 1.83 0.87 MP ------1.41 0.48 MS 0.26 0.25 0.27 0.24 0.29 0.32 2.51 0.64 MT - - - 0.02 0.02 0.02 0.12 0.10 NC 0.04 0.04 0.04 0.04 0.04 0.04 0.05 0.07 ND - - - 0.02 0.01 0.05 0.05 0.59 NE <0.01 <0.01 <0.01 0.01 <0.01 0.01 0.56 1.17 NH ------0.02 NJ - - - - 0.01 0.01 0.06 0.40 Note: - indicates no students tested. X.17 2019 Updates

Table X.9. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 209,139) 209,951) 261,501) 215,700) 284,595) 238,323) 134,235) 354,104) NM ------0.08 NV 0.07 0.07 0.06 0.09 0.09 0.08 0.40 0.18 NY <0.01 0.01 0.01 0.02 0.03 0.03 0.13 0.31 OH - 0.23 0.14 0.21 0.32 0.61 3.38 1.58 OK 0.21 0.65 0.80 0.97 0.83 1.03 0.86 0.29 OR <0.01 <0.01 <0.01 0.05 0.04 0.05 0.08 0.40 PA 0.15 0.15 0.15 0.12 0.12 0.12 0.15 0.57 SC 0.23 0.22 0.21 0.25 0.12 0.11 1.54 0.38 SD 0.07 0.10 0.10 0.09 0.15 0.09 0.05 0.06 TN 0.17 0.17 0.24 0.34 0.41 0.52 1.15 0.53 TX 0.12 0.38 0.14 0.19 0.57 1.94 0.93 0.40 UT 0.24 0.15 0.23 0.15 0.22 0.14 0.26 0.22 VA <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.05 0.18 VI ------0.06 0.01 WA - 0.05 WI 0.14 0.16 0.16 0.28 0.61 1.45 3.34 51.65 WV - - 0.01 - - 0.01 - - WY ------0.01 1.67 No response ------0.11 0.09 Race/Ethnicity Asian 4.63 4.63 4.66 4.66 4.65 4.65 2.86 3.52 Black/ African 14.30 14.29 14.26 14.26 14.23 14.23 12.66 9.56 American Hispanic/ 24.85 24.78 24.72 24.67 24.65 24.63 10.76 12.51 Latino Native 0.91 0.91 0.91 0.92 0.92 0.92 0.73 0.80 American Other 1.33 1.35 1.09 0.92 0.71 0.64 3.21 3.23 White 46.47 46.53 46.58 46.58 46.39 46.38 48.94 61.91 Unknown 7.52 7.50 7.76 7.99 8.44 8.56 20.83 8.48 Note: - indicates no students tested.

X.18 2019 Updates

Table X.10. 2018 ACT Aspire Weighted Composite Spring Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = 225,187) 129,892) 344,678) Gender F 49.08 49.20 48.79 M 50.65 50.56 50.89 No response 0.27 0.23 0.32 State AL 23.07 1.08 0.36 AR 52.34 51.58 19.12 AS - 0.02 0.01 AZ 0.09 0.86 0.41 CA 3.27 1.33 1.98 CO 0.30 5.53 1.57 CT <0.01 - 0.02 DC 0.05 0.03 0.02 FL 0.32 0.69 2.34 GA 0.09 0.30 0.19 GU - - - HI 2.69 1.50 1.29 IA 0.38 0.69 0.24 IL 3.08 3.05 2.83 IN 0.33 1.31 0.24 KS 4.23 5.12 2.44 KY 0.24 0.79 0.23 LA 2.71 5.21 1.67 MA - - 0.01 MD 0.01 0.09 0.11 ME - - - MI 0.19 0.92 0.95 MN 0.09 1.39 0.75 MO 0.08 1.86 0.87 MP - 1.42 0.43 MS 0.32 2.51 0.63 MT 0.02 0.13 0.10 NC 0.04 0.05 0.07 ND 0.06 0.05 0.57 NE 0.01 0.41 1.18 NH - - 0.02 NJ 0.01 0.04 0.34 Note: - indicates no students tested. Composite score is available for grade 8 and above. X.19 2019 Updates

Table X.10. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = 225,187) 129,892) 344,678) NM - - 0.03 NV 0.07 0.39 0.18 NY 0.03 0.12 0.14 OH 0.35 2.96 1.49 OK 0.99 0.81 0.28 OR 0.05 0.08 0.38 PA 0.12 0.14 0.53 SC 0.11 1.47 0.38 SD 0.09 0.05 0.05 TN 0.52 1.13 0.53 TX 1.92 0.86 0.39 UT 0.14 0.27 0.21 VA 0.01 0.04 0.13 VI - 0.06 0.01 WA - 0.05 WI 1.57 3.37 52.37 WV 0.01 - - WY - 0.01 1.70 No response - 0.10 0.08 Race/Ethnicity Asian 4.65 2.88 3.52 Black/ African 14.23 12.95 9.48 American Hispanic/ 24.63 10.62 12.42 Latino Native 0.92 0.74 0.80 American Other 0.65 3.19 3.18 White 46.39 48.20 62.57 Unknown 8.53 20.43 8.03 Note: - indicates no students tested. Composite score is available for grade 8 and above.

X.20 2019 Updates

Table X.11. 2018 ACT Aspire Weighted ELA Spring Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 180,254) 183,576) 183,044) 187,805) 208,244) 208,975) 109,275) 315,628) Gender F 49.79 49.46 49.45 49.29 49.03 49.23 49.58 49.76 M 50.12 50.44 50.46 50.63 50.90 50.49 50.21 50.10 No response 0.09 0.10 0.09 0.09 0.07 0.29 0.21 0.15 State AL 18.70 18.64 18.27 18.71 19.38 19.83 1.05 0.37 AR 69.59 69.04 68.36 65.00 60.68 57.39 60.69 20.55 AS ------0.02 0.01 AZ 0.07 0.07 0.07 0.10 0.37 0.09 0.99 0.39 CA 0.85 0.89 1.62 4.44 4.29 3.32 1.50 2.40 CO 0.11 0.13 0.14 0.18 0.24 0.30 0.36 0.67 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - - DC - - - 0.07 0.06 0.05 0.06 0.03 FL 0.16 0.17 0.19 0.31 0.34 0.34 0.88 2.55 GA 0.03 0.04 0.05 0.07 0.07 0.10 0.36 0.21 GU ------HI - - - 0.02 0.16 1.34 0.45 0.69 IA 0.01 0.01 0.02 0.04 0.06 0.14 0.03 0.02 IL 3.61 3.55 3.51 3.42 3.40 3.29 2.92 2.18 IN - - - 0.25 0.17 0.37 1.58 0.28 KS 1.51 1.45 1.42 1.21 1.20 2.72 3.43 1.59 KY 0.06 0.07 0.08 0.10 0.23 0.25 0.98 0.19 LA 3.75 3.75 3.62 3.22 2.81 2.53 4.38 1.33 MA ------0.01 MD 0.01 0.01 0.01 0.01 0.02 0.01 0.10 0.12 ME ------MI 0.12 0.13 0.12 0.12 0.20 0.20 1.03 0.91 MN - - - 0.01 1.84 0.08 1.31 0.69 MO 0.03 0.03 0.03 0.05 0.12 0.09 2.05 0.60 MP ------1.69 0.50 MS 0.19 0.18 0.20 0.18 0.19 0.22 1.11 0.31 MT - - - 0.02 0.03 0.03 0.15 0.14 NC 0.05 0.04 0.04 0.04 0.04 0.04 0.05 0.07 ND <0.01 0.01 0.01 0.03 0.02 0.06 0.06 0.34 NE <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.03 NH ------0.02 NJ ------0.03 0.17 Note: - indicates no students tested. X.21 2019 Updates

Table X.11. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 180,254) 183,576) 183,044) 187,805) 208,244) 208,975) 109,275) 315,628) NM ------0.03 NV 0.09 0.08 0.08 0.09 0.10 0.08 0.43 0.17 NY <0.01 0.01 0.01 0.02 0.03 0.04 0.15 0.16 OH - 0.24 0.24 0.22 0.28 0.39 3.19 1.12 OK 0.17 0.46 0.89 0.87 0.94 0.93 0.59 0.30 OR <0.01 <0.01 <0.01 0.05 0.05 0.09 0.12 0.42 PA 0.15 0.15 0.15 0.13 0.13 0.13 0.17 0.43 SC 0.03 0.02 0.03 0.03 0.03 0.15 1.73 0.40 SD 0.07 0.10 0.10 0.09 0.15 0.09 0.02 0.05 TN 0.11 0.11 0.16 0.24 0.26 0.34 1.19 0.45 TX 0.11 0.26 0.14 0.19 0.55 1.94 0.97 0.41 UT 0.24 0.15 0.24 0.16 0.23 0.15 0.21 0.11 VA - - - - <0.01 0.01 0.05 0.04 VI ------0.07 0.01 WA ------0.06 WI 0.17 0.19 0.19 0.30 1.34 2.86 3.56 56.47 WV - - 0.01 - - 0.01 - - WY ------0.01 1.87 No response ------0.09 0.04 Race/Ethnicity Asian 4.63 4.63 4.66 4.67 4.66 4.66 2.68 3.30 Black/ African 14.28 14.28 14.25 14.27 14.25 14.24 14.56 9.31 American Hispanic/ 24.85 24.80 24.74 24.69 24.69 24.68 11.53 12.60 Latino Native 0.91 0.91 0.91 0.92 0.92 0.92 0.72 0.81 American Other 1.70 1.71 1.74 1.42 0.86 0.68 3.19 2.98 White 46.47 46.56 46.57 46.61 46.42 46.43 52.39 63.93 Unknown 7.15 7.10 7.12 7.42 8.20 8.39 14.93 7.07 Note: - indicates no students tested.

X.22 2019 Updates

Table X.12. 2018 ACT Aspire Weighted STEM Spring Norm Group Demographics Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 208,979) 209,798) 260,638) 215,302) 282,850) 237,349) 133,099) 351,105) Gender F 49.08 49.14 48.95 48.91 48.76 49.17 49.17 49.86 M 50.63 50.55 50.78 50.76 51.05 50.57 50.44 50.82 No response 0.30 0.31 0.27 0.33 0.19 0.27 0.39 0.32 State AL 28.65 28.69 41.11 27.07 38.12 26.63 1.10 0.36 AR 59.48 58.60 46.10 54.78 43.65 49.02 50.41 18.89 AS ------0.02 0.01 AZ 0.07 0.07 0.07 0.09 0.36 0.09 0.85 0.41 CA 0.85 0.90 1.45 4.26 3.86 3.19 1.62 2.01 CO 0.11 0.13 0.13 0.19 0.25 0.30 5.55 1.66 CT <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 - 0.02 DC - - - 0.03 0.01 0.08 0.07 0.02 FL 0.16 0.17 0.19 0.30 0.32 0.32 0.69 2.32 GA 0.03 0.04 0.04 0.07 0.07 0.09 0.30 0.19 GU ------HI - - - 0.01 0.10 2.67 1.51 1.38 IA 0.23 0.28 0.17 0.31 0.20 0.38 0.69 0.28 IL 3.54 3.47 3.41 3.31 3.20 3.07 3.20 2.84 IN - - - 0.23 0.12 0.33 1.36 0.26 KS 1.30 1.25 1.20 2.77 2.05 4.02 5.14 2.56 KY 0.06 0.07 0.08 0.10 0.23 0.24 0.86 0.23 LA 3.69 3.63 3.39 3.25 2.70 2.70 5.15 1.68 MA ------0.11 MD 0.01 0.01 0.01 0.01 0.02 0.01 0.10 0.11 ME ------MI 0.08 0.09 0.09 0.11 0.19 0.19 0.91 0.95 MN - - - 0.02 0.61 0.08 1.41 0.76 MO 0.03 0.03 0.03 0.04 0.09 0.08 1.82 0.87 MP ------1.41 0.47 MS 0.26 0.25 0.27 0.24 0.29 0.32 2.49 0.64 MT - - - 0.02 0.02 0.02 0.12 0.10 NC 0.04 0.04 0.04 0.04 0.04 0.04 0.05 0.07 ND - - - 0.02 0.01 0.05 0.05 0.58 NE <0.01 <0.01 <0.01 0.01 <0.01 0.01 0.56 1.17 NH ------0.02 NJ - - - - 0.01 0.01 0.06 0.39 X.23 2019 Updates

Table X.12. (continued) Grade (%) 3 4 5 6 7 8 9 10 (n = (n = (n = (n = (n = (n = (n = (n = 208,979) 209,798) 260,638) 215,302) 282,850) 237,349) 133,099) 351,105) NM ------0.04 NV 0.07 0.07 0.06 0.09 0.09 0.08 0.39 0.19 NY <0.01 0.01 0.01 0.02 0.03 0.03 0.12 0.30 OH - 0.23 0.14 0.21 0.29 0.53 3.26 1.57 OK 0.21 0.65 0.77 0.94 0.82 1.02 0.83 0.28 OR <0.01 <0.01 <0.01 0.05 0.04 0.05 0.08 0.39 PA 0.15 0.15 0.14 0.12 0.12 0.12 0.14 0.56 SC 0.23 0.22 0.21 0.25 0.12 0.11 1.54 0.38 SD 0.07 0.10 0.10 0.09 0.15 0.09 0.05 0.06 TN 0.17 0.17 0.24 0.34 0.41 0.52 1.13 0.53 TX 0.12 0.38 0.14 0.19 0.57 1.91 0.93 0.39 UT 0.24 0.15 0.23 0.15 0.22 0.14 0.27 0.21 VA <0.01 <0.01 <0.01 <0.01 <0.01 0.01 0.05 0.18 VI ------0.06 0.01 WA ------0.05 WI 0.14 0.16 0.16 0.28 0.62 1.45 3.35 51.78 WV - - 0.01 - - 0.01 - - WY ------0.01 1.68 No response ------0.11 0.08 Race/Ethnicity Asian 4.63 4.63 4.66 4.66 4.65 4.65 2.90 3.52 Black/ African 14.30 14.29 14.26 14.26 14.23 14.23 12.74 9.71 American Hispanic/ 24.85 24.78 24.72 24.67 24.65 24.63 10.77 12.60 Latino Native 0.91 0.91 0.91 0.92 0.92 0.92 0.73 0.81 American Other 1.33 1.35 1.10 0.93 0.72 0.64 3.20 3.23 White 46.47 46.54 46.59 46.57 46.39 46.38 48.92 61.79 Unknown 7.52 7.49 7.75 7.99 8.44 8.55 20.75 8.33

X.24 2019 Updates

Table X.13. 2018 ACT Aspire English Spring Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 1 1 1 1 1 1 1 1 406 3 1 1 1 1 1 1 1 407 5 2 1 1 1 2 1 1 408 8 3 1 2 2 2 2 2 409 11 5 2 2 3 3 3 2 410 15 7 3 3 3 4 3 3 411 20 10 5 5 4 5 4 4 412 26 13 6 6 5 6 6 5 413 31 17 9 8 7 7 7 6 414 37 21 11 10 8 9 9 7 415 43 26 14 13 10 10 11 9 416 49 31 18 16 12 12 13 10 417 54 36 22 19 14 15 15 12 418 60 41 26 22 17 17 18 14 419 65 47 31 26 19 20 20 16 420 70 52 36 30 22 23 23 18 421 74 58 41 35 25 26 26 21 422 79 63 47 39 29 29 29 23 423 82 69 52 43 33 33 33 26 424 86 74 58 48 37 36 36 29 425 89 78 63 52 41 40 39 32 426 92 82 68 57 45 44 43 34 427 94 86 73 61 49 49 46 37 428 95 89 78 65 54 53 49 40 429 97 92 82 69 58 57 53 44 430 98 94 85 73 63 61 56 47 431 99 96 88 77 67 65 60 50 432 99 97 91 80 71 69 63 53 433 99 98 93 83 75 73 66 56 434 99 99 95 86 79 77 70 60 435 100 99 96 88 82 80 73 63 436 99 97 90 85 83 76 66 437 99 98 92 88 86 78 69 438 100 99 94 90 88 81 72 X.25 2019 Updates

Table X.13. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 95 92 91 84 75 440 99 97 94 92 86 78 441 99 97 96 94 88 81 442 100 98 97 95 90 84 443 99 98 96 92 86 444 99 98 97 94 89 445 99 99 98 95 91 446 99 99 98 96 93 447 99 99 99 97 94 448 100 99 99 98 96 449 99 99 99 97 450 100 99 99 98 451 99 99 99 452 100 99 99 453 99 99 454 99 99 455 99 99 456 100 100

X.26 2019 Updates

Table X.14. 2018 ACT Aspire Mathematics Spring Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 3 1 1 1 1 1 1 1 405 4 1 1 1 1 1 1 1 406 6 1 1 1 2 1 1 1 407 9 2 1 1 2 1 1 1 408 13 3 2 1 3 1 1 1 409 18 5 4 2 5 2 2 2 410 24 8 6 3 7 4 2 2 411 31 13 9 5 9 5 4 4 412 39 19 14 7 12 8 6 5 413 48 26 19 10 15 10 8 7 414 58 34 25 13 19 14 11 10 415 67 43 33 18 24 18 14 12 416 76 53 41 23 29 22 18 15 417 83 62 49 28 34 26 22 18 418 89 70 57 34 39 31 26 22 419 93 78 64 40 44 36 30 25 420 96 84 70 47 50 41 35 29 421 98 88 76 53 55 46 39 33 422 99 92 81 59 60 50 44 36 423 99 95 85 65 64 55 48 40 424 99 96 88 71 68 59 52 44 425 99 98 91 76 72 63 56 47 426 99 99 93 80 76 67 60 51 427 99 99 94 84 79 70 64 54 428 99 99 96 87 82 74 67 58 429 99 99 97 90 85 77 71 61 430 99 99 97 92 87 79 74 64 431 99 99 98 94 89 82 77 67 432 99 99 98 96 91 84 80 71 433 99 99 99 97 93 87 82 74 434 100 99 99 98 94 89 85 77 435 99 99 98 96 90 87 80 436 99 99 99 97 92 89 82 437 99 99 99 98 94 91 85 438 99 99 99 98 95 93 87 X.27 2019 Updates

Table X.14. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 99 96 94 90 440 100 99 99 99 97 96 92 441 99 99 99 98 97 93 442 99 99 99 99 98 95 443 99 99 99 99 98 96 444 99 99 99 99 99 97 445 99 99 99 99 99 98 446 100 99 99 99 99 99 447 99 99 99 99 99 448 99 99 99 99 99 449 99 99 99 99 99 450 99 99 99 99 99 451 100 99 99 99 99 452 99 99 99 99 453 100 99 99 99 454 99 99 99 455 99 99 99 456 100 99 99 457 99 99 458 99 99 459 99 99 460 100 100

X.28 2019 Updates

Table X.15. 2018 ACT Aspire Reading Spring Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 3 2 1 1 1 1 1 1 405 7 4 2 1 1 1 1 1 406 13 6 3 2 1 1 1 1 407 19 10 5 4 3 2 2 2 408 26 13 8 6 4 3 3 3 409 32 18 11 8 6 4 5 5 410 38 23 15 11 9 6 7 6 411 44 28 19 15 12 8 9 9 412 50 33 24 19 15 10 12 11 413 55 39 29 23 18 13 15 14 414 60 44 35 27 22 15 19 17 415 66 50 40 31 25 18 22 20 416 71 56 46 36 29 21 26 23 417 76 62 51 40 34 25 30 26 418 81 67 56 45 38 28 34 29 419 86 73 62 50 43 32 37 32 420 90 78 67 54 48 36 41 36 421 93 82 72 59 54 41 45 39 422 95 86 76 64 59 45 49 43 423 97 90 81 69 65 50 53 46 424 98 93 85 74 71 55 57 50 425 99 95 88 79 76 61 61 54 426 99 96 91 83 82 66 65 58 427 99 98 94 87 86 71 69 62 428 99 99 96 90 90 76 73 67 429 100 99 97 93 93 81 78 71 430 99 98 96 95 86 82 76 431 100 99 97 97 89 86 81 432 99 98 98 92 89 85 433 99 99 99 95 92 89 434 100 99 99 97 95 93 435 99 99 98 97 95 436 100 99 99 98 97 437 99 99 99 99 438 100 99 99 99 X.29 2019 Updates

Table X.15. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 440 100 99 99 441 99 99 442 100 100

X.30 2019 Updates

Table X.16. 2018 ACT Aspire Science Spring Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 4 2 1 1 1 1 1 1 405 6 3 1 1 1 1 1 1 406 10 5 2 2 2 1 1 1 407 14 7 3 3 2 1 1 1 408 19 9 4 4 4 2 2 2 409 23 12 6 6 5 4 2 2 410 28 15 9 8 7 5 3 3 411 33 19 11 10 10 7 5 5 412 38 23 14 13 12 9 7 6 413 43 27 18 15 15 11 9 8 414 47 32 22 18 18 14 11 10 415 52 36 26 22 22 17 14 13 416 56 41 30 25 25 20 17 15 417 61 47 35 29 29 23 20 18 418 66 52 40 33 32 26 24 21 419 70 58 45 38 36 30 28 25 420 75 63 50 42 40 34 31 28 421 79 68 56 47 44 37 35 31 422 84 74 62 52 48 41 39 35 423 87 78 68 58 53 46 43 38 424 91 83 73 63 57 50 47 41 425 94 87 78 68 61 54 51 45 426 96 90 83 73 66 59 55 48 427 98 93 87 78 70 63 59 51 428 99 95 91 82 75 68 62 54 429 99 96 93 86 79 72 66 58 430 99 97 95 89 83 76 69 61 431 99 98 97 92 86 80 73 64 432 99 99 98 94 90 84 76 68 433 100 99 99 96 92 87 79 71 434 99 99 97 95 90 82 74 435 99 99 98 96 92 85 77 436 100 99 99 98 94 88 81 437 99 99 99 96 90 84 438 100 99 99 97 92 87 X.31 2019 Updates

Table X.16. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 98 94 90 440 100 99 99 96 92 441 99 99 97 94 442 99 99 98 96 443 100 99 99 97 444 99 99 98 445 99 99 99 446 100 99 99 447 99 99 448 99 99 449 100 100

X.32 2019 Updates

Table X.17. 2018 ACT Aspire Composite Spring Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 401 1 1 1 402 1 1 1 403 1 1 1 404 1 1 1 405 1 1 1 406 1 1 1 407 1 1 1 408 1 1 1 409 1 1 1 410 2 1 1 411 3 2 2 412 5 4 3 413 7 6 5 414 9 8 7 415 12 11 10 416 15 15 13 417 19 19 16 418 23 23 19 419 27 27 22 420 31 31 26 421 36 35 29 422 40 39 33 423 45 43 36 424 49 47 40 425 54 51 43 426 59 55 46 427 63 59 50 428 68 63 53 429 72 66 57 430 76 70 61 431 80 74 64 432 84 77 68 433 87 81 72 434 90 84 76 435 93 87 79 436 95 90 83 437 96 92 86 438 98 95 89 X.33 Note: Composite score is available for grade 8 and above. 2019 Updates

Table X.17. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 98 96 92 440 99 97 94 441 99 98 96 442 99 99 98 443 99 99 99 444 99 99 99 445 99 99 99 446 99 99 99 447 99 99 99 448 99 99 99 449 100 99 99 450 99 99 451 99 99 452 100 100 Note: Composite score is available for grade 8 and above.

X.34 2019 Updates

Table X.18. 2018 ACT Aspire ELA Spring Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 1 1 1 1 1 1 1 1 406 1 1 1 1 1 1 1 1 407 1 1 1 1 1 1 1 1 408 3 1 1 1 1 1 1 1 409 5 2 1 1 1 1 1 1 410 8 3 2 2 2 2 2 1 411 12 5 3 3 3 2 2 2 412 17 8 5 4 4 4 4 3 413 23 11 7 5 6 5 5 4 414 29 16 10 8 8 7 7 6 415 36 20 14 10 11 9 9 8 416 43 26 18 13 14 11 12 10 417 50 32 23 16 17 14 15 12 418 57 38 28 20 21 17 18 15 419 64 45 34 24 25 21 21 17 420 70 52 40 28 29 25 25 20 421 76 60 46 33 34 29 28 23 422 81 67 53 38 39 33 32 26 423 86 73 59 44 45 38 36 30 424 90 80 65 50 50 44 41 33 425 93 85 71 55 56 49 45 37 426 95 89 77 61 62 55 50 41 427 97 93 82 67 68 61 54 45 428 98 96 86 73 73 67 59 50 429 99 97 90 78 78 72 64 54 430 99 98 93 83 83 77 69 59 431 99 99 95 87 87 82 74 64 432 99 99 97 90 90 86 79 70 433 99 99 98 93 93 90 83 75 434 99 99 99 95 95 93 87 80 435 100 99 99 97 97 95 90 84 436 100 99 98 98 97 93 88 437 99 99 99 98 96 92 438 99 99 99 99 97 95 439 100 99 99 99 98 97 440 99 99 99 99 98 441 99 99 99 99 99 X.35 2019 Updates

Table X.18. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 442 99 99 99 99 99 443 99 99 99 99 99 444 100 99 99 99 99 445 100 99 99 99 446 99 99 99 447 100 99 99 448 99 99 449 100 100

X.36 2019 Updates

Table X.19. 2018 ACT Aspire STEM Spring Norms: Percent of Students at or Below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 2 1 1 1 1 1 1 1 405 3 1 1 1 1 1 1 1 406 6 1 1 1 1 1 1 1 407 9 2 1 1 1 1 1 1 408 14 4 2 1 2 1 1 1 409 19 7 3 2 3 1 1 1 410 24 10 5 3 5 2 1 1 411 30 14 8 5 7 4 2 2 412 36 19 11 7 10 6 4 4 413 43 25 15 10 13 8 6 5 414 49 31 20 14 16 11 8 7 415 56 38 26 18 20 14 11 10 416 62 45 32 22 25 18 15 13 417 69 52 39 27 29 22 19 16 418 75 59 46 32 34 26 23 20 419 81 65 52 38 39 31 27 23 420 86 72 59 43 44 35 31 27 421 90 78 65 49 49 40 36 31 422 94 83 71 55 53 45 40 34 423 96 88 77 61 58 49 44 38 424 98 91 82 66 62 54 49 42 425 99 94 86 72 67 58 53 45 426 99 96 89 77 71 62 57 49 427 99 98 92 81 75 66 61 52 428 99 99 94 85 79 70 64 55 429 99 99 96 89 82 74 68 59 430 99 99 97 92 85 78 71 62 431 99 99 98 94 88 81 75 65 432 99 99 99 96 91 84 78 69 433 99 99 99 97 93 87 81 72 434 100 99 99 98 95 90 84 75 435 99 99 99 97 92 87 79 436 99 99 99 98 94 89 82 437 99 99 99 99 95 91 85 438 100 99 99 99 97 93 88 X.37 2019 Updates

Table X.19. (continued)

X.38 2019 Updates

Table X.20. National Average Scale Scores Using 2018 Spring Norm Sample

Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 417 413 413 415 418 415 4 420 416 416 418 420 417 5 423 418 417 420 422 419 6 425 421 419 422 424 422 7 427 421 420 422 424 422 8 428 423 423 424 425 425 424 9 429 425 423 425 426 426 425 10 431 427 424 427 427 428 427 Note: Composite score is available for grade 8 and above.

Table X.21. Percentage of Students Who Met Subject Benchmark in the 2018 Spring Norm Sample

Grade Subject area level English Mathematics Reading Science ELA STEM 3 74 61 40 39 43 19 4 69 57 44 42 40 22 5 74 51 38 44 41 18 6 74 60 46 48 45 19 7 78 45 41 43 44 18 8 74 41 50 41 45 16 9 61 36 43 34 46 16 10 63 33 38 36 46 18 Note: Benchmark is not available for ACT Aspire Composite Score.

X.39 2019 Updates

Section 2: ACT Aspire Fall Norms ACT Aspire Fall Norm Samples ACT Aspire spring norms are based on data from spring testers and used in reporting for both fall and spring administrations. Norms based on data from fall testers are presented here to serve as an additional reference to facilitate score interpretation. One thing to note is that the weighting methodology used in generating spring norms was not implemented in generating fall norms. Rather, unweighted data from fall testers were used. For grades 3–10, the fall norm samples included students who took ACT Aspire on-grade subject tests in fall 2015, fall 2016, or fall 2017 which is consistent with the goal of including three years of rolling data for ACT Aspire norms. Table X.22 provides the sample sizes by grade level and subject area.

Table X.22. Sample Sizes Used for Fall Norm Establishment

Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 7,911 8,186 8,089 7,515 5,186 7,494 4 15,992 16,564 16,469 15,633 6,018 15,601 5 12,677 13,333 13,230 12,261 6,491 12,214 6 28,074 28,617 28,308 28,135 13,889 27,323 7 45,230 46,553 46,386 45,419 29,329 45,185 8 108,092 113,211 113,010 107,397 102,706 73,003 106,267 9 122,515 130,854 121,778 125,819 114,038 82,727 125,012 10 164,732 166,743 163,030 160,664 153,260 118,057 159,083

Fall Norm Tables and Summary Statistics The fall norm table includes the cumulative percentages of students scoring at or below each scale score in the unweighted norm sample from fall testers. The fall norm tables are presented in Tables X.23–X.29. The mean scale score values for students in each grade and subject test are given in Table X.30. Table X.31 shows the estimated percentages of students meeting ACT Readiness Benchmark in each grade and subject test.

X.40 2019 Updates

Table X.23. 2018 ACT Aspire English Fall Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 2 1 1 1 1 1 1 1 406 4 2 1 1 1 1 1 1 407 6 2 1 1 2 1 2 1 408 9 4 1 2 2 2 2 1 409 14 6 2 3 3 3 3 2 410 20 9 3 4 4 3 4 3 411 26 11 4 6 5 5 5 4 412 32 14 6 7 5 6 6 5 413 39 18 8 9 8 8 7 6 414 45 21 11 11 10 8 10 8 415 52 26 14 14 10 10 12 10 416 58 30 18 18 13 12 14 12 417 63 37 22 21 15 15 17 15 418 69 42 27 26 18 17 19 17 419 74 50 32 29 20 19 22 20 420 79 55 38 32 23 22 23 21 421 82 60 43 37 27 27 26 24 422 87 64 46 42 29 29 30 26 423 90 70 52 47 34 33 32 29 424 92 75 58 52 39 38 36 32 425 94 81 64 55 42 40 40 36 426 96 84 70 59 47 45 43 39 427 97 87 72 63 53 49 46 41 428 98 91 77 68 56 55 50 45 429 99 94 82 74 58 58 53 48 430 99 95 85 76 64 64 57 52 431 99 97 88 81 69 68 61 56 432 99 97 92 85 74 73 65 60 433 99 99 93 86 76 74 68 63 434 99 99 95 89 81 78 72 67 435 100 99 97 91 85 80 75 70 436 99 97 93 87 84 78 73 437 99 98 93 90 88 81 75 438 100 99 96 91 88 83 78 X.41 2019 Updates

Table X.23. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 96 94 92 85 81 440 99 97 95 93 87 83 441 99 98 97 95 88 84 442 100 99 97 96 91 87 443 99 98 98 93 89 444 99 99 98 95 92 445 99 99 99 96 94 446 99 99 99 97 95 447 99 99 99 97 96 448 100 99 99 98 97 449 99 99 99 98 450 100 99 99 98 451 99 99 99 452 100 99 99 453 99 99 454 99 99 455 99 99 456 100 100

X.42 2019 Updates

Table X.24. 2018 ACT Aspire Mathematics Fall Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 2 1 1 1 1 1 1 1 403 3 1 1 1 1 1 1 1 404 5 1 1 1 1 1 1 1 405 8 1 1 1 2 1 1 1 406 12 1 1 1 2 1 1 1 407 17 3 3 2 4 2 1 1 408 23 4 4 3 5 2 1 1 409 32 6 6 3 6 4 2 2 410 40 11 10 5 8 5 3 3 411 50 17 14 8 11 7 5 4 412 59 25 19 10 12 9 7 6 413 70 34 26 14 16 12 8 7 414 79 46 32 18 22 16 11 9 415 86 54 41 25 25 21 14 12 416 92 64 52 33 32 26 18 16 417 96 73 60 36 39 30 22 19 418 98 80 66 43 42 35 27 24 419 99 87 75 51 48 40 30 26 420 99 90 79 58 55 44 34 30 421 99 93 84 62 57 49 39 35 422 99 96 88 69 63 55 42 38 423 99 98 90 74 68 59 47 43 424 99 99 92 79 72 63 51 47 425 99 99 94 83 76 67 56 51 426 99 99 96 86 80 71 59 54 427 99 99 97 90 83 75 63 58 428 99 99 98 92 85 78 66 61 429 99 99 98 94 87 80 70 64 430 99 99 99 96 89 82 73 68 431 99 99 99 97 92 84 77 71 432 99 99 99 97 93 86 79 74 433 99 99 99 98 95 88 82 77 434 100 99 99 99 96 90 85 80 435 99 99 99 97 92 87 83 436 99 99 99 98 93 89 86 437 99 99 99 99 94 91 88 438 99 99 99 99 96 93 90 X.43 2019 Updates

Table X.24. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 99 97 95 92 440 100 99 99 99 97 96 94 441 99 99 99 98 97 96 442 99 99 99 99 98 97 443 99 99 99 99 99 98 444 99 99 99 99 99 99 445 99 99 99 99 99 99 446 100 99 99 99 99 99 447 99 99 99 99 99 448 99 99 99 99 99 449 99 99 99 99 99 450 99 99 99 99 99 451 100 99 99 99 99 452 99 99 99 99 453 100 99 99 99 454 99 99 99 455 99 99 99 456 100 99 99 457 99 99 458 99 99 459 99 99 460 100 100

X.44 2019 Updates

Table X.25. 2018 ACT Aspire Reading Fall Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 2 1 1 1 1 1 1 1 404 5 4 1 1 1 1 1 1 405 11 5 2 2 1 1 1 1 406 18 8 4 3 2 2 1 1 407 28 10 6 5 3 2 2 2 408 34 14 8 8 5 4 3 3 409 41 19 13 11 8 5 5 4 410 48 24 17 15 11 7 7 6 411 55 29 21 19 15 10 11 9 412 60 34 25 22 17 13 13 11 413 66 40 30 26 21 16 16 14 414 72 50 35 31 26 19 18 16 415 77 52 41 36 30 22 22 20 416 82 57 47 40 34 26 25 23 417 86 63 53 45 36 29 29 26 418 90 69 60 50 41 33 33 30 419 93 74 65 56 47 36 37 33 420 95 80 72 58 52 40 41 37 421 97 85 75 62 57 44 46 41 422 99 89 78 68 62 49 49 45 423 99 91 84 74 67 54 54 49 424 99 93 85 78 72 59 58 54 425 99 96 89 80 77 64 62 58 426 99 98 93 85 82 69 68 63 427 99 98 96 89 86 74 69 64 428 99 99 96 93 90 80 73 69 429 100 99 98 96 94 84 79 75 430 99 99 96 96 88 81 79 431 100 99 98 97 89 85 81 432 99 99 98 93 89 86 433 99 99 99 96 93 91 434 100 99 99 98 95 94 435 99 99 98 98 97 436 100 99 99 99 99 437 99 99 99 99 438 100 99 99 99 X.45 2019 Updates

Table X.25. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 440 100 99 99 441 99 99 442 100 100

X.46 2019 Updates

Table X.26. 2018 ACT Aspire Science Fall Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 3 1 1 1 1 1 1 1 404 6 2 1 1 1 1 1 1 405 9 3 1 1 1 1 1 1 406 14 6 2 2 2 2 1 1 407 19 8 3 3 3 2 2 1 408 26 11 5 5 5 4 2 2 409 32 13 7 7 7 6 3 3 410 36 17 10 10 10 7 4 3 411 41 22 13 13 13 9 6 5 412 51 26 16 16 16 12 7 5 413 56 30 20 19 19 16 10 9 414 61 36 23 22 20 17 11 9 415 65 41 28 26 23 21 16 13 416 69 46 32 30 27 24 20 17 417 73 51 37 32 32 27 23 20 418 77 57 42 36 36 31 27 23 419 81 62 47 40 39 35 32 28 420 85 66 54 44 43 38 34 30 421 88 71 59 50 47 42 38 34 422 92 78 66 54 51 46 42 38 423 94 82 70 59 54 51 46 42 424 96 85 75 65 58 55 50 45 425 98 90 80 70 63 59 55 50 426 99 92 85 76 66 62 59 53 427 99 94 89 81 71 67 63 57 428 99 96 91 84 76 72 66 60 429 99 97 94 88 79 75 70 64 430 99 98 96 89 82 79 73 67 431 99 99 97 92 86 82 76 71 432 99 99 99 95 89 85 79 74 433 100 99 99 96 91 88 82 77 434 99 99 97 95 90 84 79 435 99 99 98 97 93 87 83 436 100 99 99 98 95 90 85 437 99 99 99 95 92 88 438 100 99 99 97 94 91 X.47 2019 Updates

Table X.26. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 98 96 94 440 100 99 99 97 96 441 99 99 98 97 442 99 99 99 98 443 100 99 99 99 444 99 99 99 445 99 99 99 446 100 99 99 447 99 99 448 99 99 449 100 100

X.48 2019 Updates

Table X.27. 2018 ACT Aspire Composite Fall Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 401 1 1 1 402 1 1 1 403 1 1 1 404 1 1 1 405 1 1 1 406 1 1 1 407 1 1 1 408 1 1 1 409 1 1 1 410 3 1 1 411 4 2 2 412 6 4 3 413 9 6 5 414 12 9 7 415 15 12 10 416 18 15 13 417 22 19 16 418 26 22 19 419 30 26 23 420 34 30 27 421 39 34 30 422 44 38 34 423 48 43 38 424 53 47 43 425 58 51 47 426 63 56 51 427 67 60 55 428 72 64 59 429 76 68 63 430 80 72 67 431 83 76 71 432 86 79 74 433 89 83 78 434 91 86 81 435 94 88 84 436 95 91 87 Note: Composite score is available for grade 8 and above. X.49 2019 Updates

Table X.27. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 437 97 93 90 438 98 95 93 439 99 97 95 440 99 98 96 441 99 99 98 442 99 99 99 443 99 99 99 444 99 99 99 445 99 99 99 446 99 99 99 447 99 99 99 448 99 99 99 449 100 99 99 450 99 99 451 99 99 452 100 100 Note: Composite score is available for grade 8 and above.

X.50 2019 Updates

Table X.28. 2018 ACT Aspire ELA Fall Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 403 1 1 1 1 1 1 1 1 404 1 1 1 1 1 1 1 1 405 1 1 1 1 1 1 1 1 406 1 1 1 1 1 1 1 1 407 1 1 1 1 1 1 1 1 408 3 1 1 1 1 1 1 1 409 6 1 1 1 1 1 1 1 410 11 3 2 2 2 2 2 1 411 17 5 3 3 4 3 3 2 412 23 8 5 5 6 5 5 3 413 31 12 7 7 8 6 7 5 414 38 16 11 10 11 8 9 7 415 46 22 14 13 14 11 12 9 416 55 28 18 17 18 14 14 11 417 63 35 24 21 21 17 18 14 418 70 43 30 26 26 21 21 18 419 77 52 36 30 30 25 25 21 420 83 60 43 36 35 30 29 25 421 88 67 51 42 41 35 33 29 422 92 74 58 48 46 40 38 33 423 94 81 65 54 52 45 42 37 424 96 87 71 61 58 51 47 42 425 98 91 77 67 64 57 52 46 426 99 95 82 73 69 62 57 51 427 99 97 87 78 75 68 62 56 428 99 98 91 83 80 74 67 61 429 99 99 93 87 84 78 71 66 430 99 99 96 90 88 83 76 70 431 99 99 97 93 91 87 80 75 432 99 99 99 95 94 90 84 79 433 99 99 99 97 96 93 87 83 434 99 99 99 98 97 95 90 87 435 100 99 99 99 98 97 93 90 436 100 99 99 99 98 95 93 437 99 99 99 99 97 95 438 99 99 99 99 98 97 439 100 99 99 99 99 98 440 99 99 99 99 99 441 99 99 99 99 99 X.51 2019 Updates

Table X.28. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 442 99 99 99 99 99 443 99 99 99 99 99 444 100 99 99 99 99 445 100 99 99 99 446 99 99 99 447 100 99 99 448 99 99 449 100 100

X.52 2019 Updates

Table X.29. 2018 ACT Aspire STEM Fall Norms: Percent of Students at or below Each Scale Score

Scale Grade Score 3 4 5 6 7 8 9 10 400 1 1 1 1 1 1 1 1 401 1 1 1 1 1 1 1 1 402 1 1 1 1 1 1 1 1 403 1 1 1 1 1 1 1 1 404 3 1 1 1 1 1 1 1 405 6 1 1 1 1 1 1 1 406 10 1 1 1 1 1 1 1 407 16 3 1 1 2 1 1 1 408 22 5 2 2 3 1 1 1 409 29 8 4 3 4 2 1 1 410 37 12 7 4 6 4 2 1 411 45 17 11 7 8 6 3 2 412 52 23 15 10 12 8 4 4 413 60 30 19 14 15 11 7 5 414 66 37 24 18 19 15 9 8 415 72 44 31 23 23 18 12 10 416 78 51 37 27 28 22 16 14 417 83 58 44 32 32 27 20 17 418 88 65 51 38 37 31 24 21 419 92 72 59 43 42 35 28 25 420 95 78 66 49 47 40 33 29 421 97 84 72 55 52 45 37 33 422 99 88 78 61 56 49 41 37 423 99 92 83 67 61 54 46 41 424 99 94 87 72 65 58 50 45 425 99 96 90 78 70 63 54 49 426 99 98 92 82 74 67 58 53 427 99 99 95 86 78 71 62 57 428 99 99 96 89 81 75 66 61 429 99 99 97 92 84 78 70 64 430 99 99 98 94 87 81 73 68 431 99 99 99 96 90 84 77 71 432 99 99 99 97 92 86 80 74 433 99 99 99 98 94 89 83 78 434 100 99 99 99 96 91 86 81 435 99 99 99 97 93 88 84 436 99 99 99 98 94 90 86 437 99 99 99 99 96 92 89 438 100 99 99 99 97 94 91 X.53 2019 Updates

Table X.29. (continued)

Scale Grade Score 3 4 5 6 7 8 9 10 439 99 99 99 98 96 93 440 99 99 99 98 97 95 441 99 99 99 99 98 97 442 100 99 99 99 99 98 443 99 99 99 99 99 444 99 99 99 99 99 445 99 99 99 99 99 446 100 99 99 99 99 447 99 99 99 99 448 100 99 99 99 449 99 99 99 450 99 99 99 451 100 99 99 452 99 99 453 99 99 454 99 99 455 100 100

X.54 2019 Updates

Table X.30. National Average Scale Scores Using 2018 Fall Norm Sample

Grade Subject area level English Mathematics Reading Science Composite ELA STEM 3 416 411 411 413 416 412 4 420 415 415 417 419 416 5 423 417 417 420 421 418 6 424 420 419 421 423 421 7 427 420 420 422 423 421 8 427 423 422 423 424 424 423 9 428 425 422 425 425 425 425 10 430 426 423 426 426 426 426 Note: Composite score is available for grade 8 and above.

Table X.31. Percentage of Students Who Met Subject Benchmark in the 2018 Fall Norm Sample

Grade Subject area level English Mathematics Reading Science ELA STEM 3 68 41 28 27 30 8 4 70 46 43 38 33 16 5 73 40 35 41 35 13 6 71 49 42 46 33 14 7 77 43 38 42 36 16 8 73 37 46 38 38 14 9 60 37 42 30 38 14 10 59 29 36 29 34 14 Note: Benchmark is not available for ACT Aspire Composite Score.

X.55 2019 Updates

Section 3: Reliability for Spring 2018 ACT Aspire Standard Forms This section presents the estimates of coefficient alpha for ACT Aspire standard test forms administered in spring 2018. Ranges of reliability estimates for each grade and subject are shown in Table X.32.

Table X.32. Ranges of Coefficient Alpha of ACT Aspire Standard Forms Administered in Spring 2018 Coefficient Alpha Grade Subject Online Paper 3 English 0.83–0.87 0.83–0.86 Mathematics 0.83–0.87 0.84–0.88 Reading 0.84–0.87 0.83–0.87 Science 0.86–0.90 0.85–0.90 4 English 0.82–0.87 0.81–0.86 Mathematics 0.82–0.87 0.84–0.87 Reading 0.84–0.87 0.82–0.86 Science 0.86–0.90 0.85–0.88 5 English 0.82–0.87 0.81–0.88 Mathematics 0.83–0.87 0.82–0.87 Reading 0.82–0.86 0.82–0.86 Science 0.86–0.89 0.85–0.88 6 English 0.84–0.86 0.84–0.86 Mathematics 0.85–0.87 0.85–0.88 Reading 0.84–0.86 0.84–0.85 Science 0.88–0.90 0.86–0.89 7 English 0.85–0.88 0.85–0.86 Mathematics 0.86–0.87 0.87–0.88 Reading 0.84–0.87 0.82–0.86 Science 0.88–0.91 0.86–0.90 8 English 0.85–0.88 0.85–0.87 Mathematics 0.87–0.88 0.86–0.88 Reading 0.81–0.86 0.80–0.83 Science 0.89–0.91 0.88–0.89 EHS English 0.90–0.91 0.91–0.92 Mathematics 0.88–0.90 0.88–0.90 Reading 0.82–0.87 0.82–0.87 Science 0.90–0.91 0.89–0.91 Note. EHS = Early High School; Forms with sample size less than 200 are not included.

X.56 2019 Updates

Section 4: Character Limits for Student Responses on the ACT Aspire Writing Test Table X.33. Character Limits for Student Responses on the ACT Aspire Writing Test

Grade level Character limit 3 5,000 4 5,000 5 5,000 6 8,000 7 8,000 8 10,000 Early High School 10,000

X.57