THE PRICE OF EXCELLENCE: STUDENT LEARNING OUTCOMES FROM ALTERNATIVE COMPENSATION MODELS THAT FEATURE SUBSTANTIAL STARTING SALARIES FOR TEACHERS

By

KENNETH SAVAGE

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION

UNIVERSITY OF FLORIDA

2019

© 2019 Kenneth A. Savage

To my beloved Beca in honor of our shared passion for social justice, as well as my departed friend and colleague, Dr. Brian Dassler

ACKNOWLEDGMENTS

I could never have imagined when I was embarking on this doctoral journey just what a life changing experience it would be. When I started this journey, I was working as an assistant principal at a high school. Years later, I married one of the other members of our doctoral cohort, we have two beautiful children, and I have now been a leader at all levels; elementary, middle, high, district, and state level leadership. I have successfully helped influence turnaround for a number of underperforming schools and I have had the privilege of serving as the State of Florida Principal of the year for 2018.

More significantly, I have had the opportunity to experience how temporary this life journey can be as I have had to bury numerous students, teachers, inspiring colleagues, and my own extended family. It is that fleeting nature of our lives that most invigorates me to seize each and every day with a purpose and to serve to the best of my ability.

I pursued this doctorate not just for my own edification, but as an instrument to enhance my ability to serve the greater good. For as I have been given the privilege of my life and experiences, I owe far more than I can ever hope to repay. In that spirit, I first want to thank my committee chair, Dr. R. Craig Wood for his willingness to lend his brilliant mind and sage experience to not just myself, but our entire cohort through our classes and his individual counsel. His message wasn’t always easily received, but its truth was undeniable. His candor and urging to create ‘time on task’ was the type of internal mantra that eventually helped me find the light at the end of the very long tunnel. I also wish to thank my wife, family, and close friends for their unrelenting belief in me and their unconditional love and support. All that I am, I owe to that love- perhaps one day, I can help influence the realization of a world where everyone is afforded that same grace.

4

TABLE OF CONTENTS

page

ACKNOWLEDGMENTS ...... 4

LIST OF TABLES ...... 7

LIST OF FIGURES ...... 9

LIST OF ABBREVIATIONS ...... 10

ABSTRACT ...... 11

CHAPTER

1 INTRODUCTION TO THE STUDY ...... 13

Population ...... 21 Statement of Purpose ...... 22 Research Question ...... 22 Research Hypothesis ...... 22 Assumptions ...... 22 Limitations ...... 23 Summary ...... 24

2 LITERATURE REVIEW ...... 25

The Early History of American Teacher Compensation ...... 25 Emerging Concerns with the Single Salary Schedule ...... 28 Alternative Teacher Compensation Reform ...... 30 International Context for Alternative Compensation ...... 33 Differentiated Teacher Compensation Methods ...... 35 Pay-for-Performance ...... 35 Knowledge- and Skills-based Pay ...... 39 Career Ladder Programs ...... 41 Market-Based Pay ...... 43 Summary ...... 47

3 RESEARCH DESIGN ...... 49

Research Question ...... 49 Research Design ...... 49 Florida Learning Gain Value ...... 51 New York Median Student Growth Percentile Value ...... 52 Research Population...... 53 Florida Sample ...... 54 New York Sample ...... 57

5

Data Sources ...... 58 Data Analysis ...... 58 Summary ...... 60

4 PRESENTATION OF RESULTS ...... 61

Introduction ...... 61 Descriptive Statistics Results ...... 61 Florida Sample ...... 61 New York Samples ...... 63 2015-2016 School Year ...... 64 2016-2017 School Year ...... 65 Normality Test Results ...... 67 Florida Sample ...... 67 New York Samples ...... 71 2015-2016 School Year ...... 71 2016-2017 School Year ...... 75 Statistical Hypothesis Test Results ...... 81 Florida Sample ...... 81 New York Sample ...... 82 Summary ...... 83

5 CONCLUSIONS ...... 85

Summary of Findings: ...... 85 Recommendations for Future Policy ...... 87 Recommendations for Future Research ...... 89

LIST OF REFERENCES ...... 92

BIOGRAPHICAL SKETCH ...... 99

6

LIST OF TABLES

Table page

3-1 School Grade History for Florida treatment school 1999-2018 within Florida State Accountability System beginning with most recent year ...... 55

4-1 Descriptive Statistics for Student Demographics of Florida treatment school 2017-2018 School Year ...... 62

4-2 Descriptive Statistics for Student Demographics Florida selected peer sample for comparative analysis 2017-2018 School Year ...... 63

4-3 Descriptive Statistics for Student Demographics of New York treatment school 2015-2016 School Year ...... 64

4-4 Descriptive Statistics for Student Demographics New York selected peer sample for comparative analysis 2015-2016 School Year ...... 65

4-5 Descriptive Statistics for Student Demographics of New York treatment school 2016-2017 School Year ...... 66

4-6 Descriptive Statistics for Student Demographics New York selected peer sample for comparative analysis 2016-2017 School Year ...... 67

4-7 Descriptive Statistics for Distribution of English Language Arts Learning Gains Florida selected peer sample for comparative analysis 2017-2018 School Year ...... 67

4-8 Descriptive Statistics for Distribution of Mathematics Learning Gains Florida selected peer sample for comparative analysis 2017-2018 School Year ...... 69

4-9 Shapiro-Wilk Normality Test of English Language Arts and Mathematics Learning Gains Florida selected peer sample for comparative analysis 2017- 2018 School Year ...... 70

4-10 Descriptive Statistics for Distribution of English Language Arts Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2015-2016 School Year...... 71

4-11 Descriptive Statistics for Distribution of Mathematics Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2015-2016 School Year ...... 73

4-12 Shapiro-Wilk Normality Test of English Language Arts and Mathematics Median Student Growth Percentile Values for New York selected peer sample for comparative analysis 2015-2016 School Year ...... 75

7

4-13 Descriptive Statistics for Distribution of English Language Arts Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2016-2017 School Year...... 75

4-14 Descriptive Statistics for Distribution of Mathematics Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2016-2017 School Year ...... 78

4-15 Shapiro-Wilk Normality Test of English Language Arts and Mathematics Median Student Growth Percentile Values for New York selected peer sample for comparative analysis 2016-2017 School Year ...... 80

4-16 One-Sample T-Test results for Florida treatment school compared with peer sample 2017-2018 ...... 81

4-17 One-Sample T-Test results for New York treatment school compared with peer sample 2015-2016 ...... 82

4-18 One-Sample T-Test results for New York treatment school compared with peer sample 2016-2017 ...... 83

8

LIST OF FIGURES

Figure page

4-1 Normal Q-Q Plot of 17-18 English Language Arts Learning Gains for Florida Peer Group of Schools ...... 68

4-2 Detrended Normal Q-Q Plot of 17-18 English Language Arts Learning Gains for Florida Peer Group of Schools ...... 68

4-3 Normal Q-Q Plot of 17-18 Mathematics Gains for Florida Peer Group of Schools ...... 69

4-4 Detrended Normal Q-Q Plot of 17-18 Mathematics Gains for Florida Peer Group of Schools ...... 70

4-5 Normal Q-Q Plot of 15-16 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools ...... 72

4-6 Detrended Normal Q-Q Plot of 15-16 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools ...... 72

4-7 Normal Q-Q Plot of 15-16 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools ...... 74

4-8 Detrended Normal Q-Q Plot of 15-16 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools ...... 74

4-9 Normal Q-Q Plot of 16-17 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools ...... 76

4-10 Detrended Normal Q-Q Plot of 16-17 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools ...... 77

4-11 Normal Q-Q Plot of 16-17 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools ...... 79

4-12 Detrended Normal Q-Q Plot of 16-17 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools ...... 80

9

LIST OF ABBREVIATIONS

‘At-Risk’ School A school that is predominantly comprised of students are either likely to drop out of school prior to graduation or fail to achieve basic levels of proficiency in mathematics and reading subjects1

‘Hard-to-Staff’ Schools that have difficulty in attracting and retaining experienced School or well-prepared new teachers most often located in high poverty inner-city or rural areas2

Learning Gain A term signifying that a student demonstrates successful performance of a defined annual growth threshold from one year to the next year as specified by criteria outlined within the Florida Department of Education accountability system3

Substantial The term substantial is being utilized to otherwise qualify a Compensation compensation level as being significantly above typically observed compensation levels within the majority of school systems relative to the local economies. For the purposes of this study, this level will be qualified by approximately one-and-one-half times the mean or median salary reported or greater

1 Kaufman, Phillip, and Denise Bradbury. August 1992. Characteristics of At-Risk Students in NELS:88. Contractor Report, U.S. Department of Education, Office of Educational Research and Improvement, Berkeley, CA: National Center for Education Statistics. Accessed September 19, 2018. https://nces.ed.gov/pubs92/92042.pdf.

2 Southeast Center for Teaching Quality. n.d. Recruiting Quality Teachers to Hard-to-Staff Schools. James B Hunt, Jr. Institute for Educational Leadership and Policy. Accessed September 27, 2018. http://www.nea.org/assets/docs/HE/mf_HuntHardtoStaff.pdf.

3 Florida Department of Education. June 2018. "2017-2018 Guide to Calculating School and District Grades." Technical Assistance Report, Bureau of Accountability Reporting; Divsion of Accountability, Research, and Measurement. Accessed December 13, 2018.

10

Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Education

THE PRICE OF EXCELLENCE: STUDENT LEARNING OUTCOMES FROM ALTERNATIVE COMPENSATION MODELS THAT FEATURE SUBSTANTIAL STARTING SALARIES FOR TEACHERS

By

Kenneth Savage

August 2019

Chair: R. Craig Wood Major: Educational Leadership

Teacher compensation is one of the most divisive issues in public educational policy as all stakeholder groups ponder the cost effectiveness of increasing teacher pay.

What this issue presupposes is that our American educational system tends to underpay teachers and thusly the results of our schools reflect this lagged commitment to appropriate funding. However, critics suggest that as long as we continue to pay our worst teachers and best teachers at comparable levels, the issue is not necessarily that there is a lack of funding as much there is a lack of purposeful strategic spending to differentiate priorities within the teacher labor group as a whole. The current dominant method of pay is the single-salary schedule which is characterized as a byproduct of a labor union-influenced field that insulates our teacher workforce from substantial salary reform to protect the unexceptional majority of teachers at the expense of attracting and retaining our most valuable teachers while potentially acquiescing mediocrity for our students.

The purpose of this study was to evaluate to what extent a substantial increase in teacher compensation levels may produce a positive statistically significant impact on

11

student learning. This term of substantial compensation is not currently present in the research literature but seeks to qualify an opportunity for isolated cases of considerable deviation from existing models of compensation with extraordinary bonuses or salary levels that represent at least 150 percent of a median educator salary within a local community labor market. Two treatment schools; one new implementation in Florida and one multi-year implementation in New York, were studied and the results in student learning were compared to peer groups of schools. Both descriptive and inferential statistics were employed. A series of One-Sample T-Tests were utilized to detect statistical significance of the annual student growth measures compared from treatment schools among their respective peer schools. Results indicated the Florida treatment school was unable to reject the null hypothesis, but the New York treatment school demonstrated statistical significance over both years of results with a large practical significance indicating a potentially compelling opportunity that warrants additional study.

12

CHAPTER 1 INTRODUCTION TO THE STUDY

It is a widely accepted finding that teachers constitute among the most crucial variables in student learning.1 Moreover, our American society purports to value and prioritize having quality schools, yet teacher compensation is a consistently lagging labor market with teachers’ wages being significantly outpaced by other occupations with comparable term-length college graduates.2 This visible and consistent trend in diminished earning potential “almost certainly affects who does and who does not think about teaching as an occupation.”3 Furthermore, this pattern has existed for a significant period of time with an observable decline that is evident as far back as the late 1950s up through the present day.4 The specific cause of this decline is difficult to discern with legitimate precision, but a leading researcher in the economics of educational policy contends that at least a good portion of this can be attributed to the predominant pay structure in education known as the single-salary schedule.5

Hanushek characterizes the current system as tending “to underpay effective teachers while overpaying their ineffective peers, again avoiding use of any monetary incentives

1 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

2 National Education Association. 2005. "Status of the American Public School Teacher, 2005-2006." National Occupational Employment and Wage Estimates. U.S. Department of Labor, Bureau of Labor Statistics, May.

3 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

4 Ibid.

5 Ibid.

13

to improve the teaching force. We are thus left to hope that the goodwill and strong drive of effective teachers is sufficient to keep them in the classroom.”6

The question of what constitutes an effective versus ineffective teacher bears careful consideration, but for the purpose of this study, it will be intentionally restricted to the issues of compensation juxtaposed with teacher quality only as it emphasizes producing statistically significant positive outcomes in student learning. It is well documented that when compared to the role of the teacher, “no other aspect of schools

– spending, leadership, curriculum, etc. – is nearly as important in determining student achievement.”7 Even within a school the variance in student outcomes from teacher to teacher can be particularly disparate where some have demonstrated over one and a half years of growth in student achievement in a single academic year, while contrasted with other teachers having equivalent students being found to net only a half year of growth.8 Two students beginning at the same level of performance can achieve vastly different outcomes within just a year’s worth of instruction that can be predominantly attributed to the quality of the teacher.9 No other trait of a school approaches this singular impact.10

6 Ibid.

7 Ibid.

8 Hanushek, E. A., and S. G. Rivkin. 2006. "Teacher Quality." In Handbook of the Economics of Education, edited by E. A. Hanushek and F. Welch, 1051-1078. Amsterdam: North Holland.

9 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

10 Ibid.

14

While it is clear that a teacher has a substantial impact on student learning, discerning the most critical systemic professional attributes that may be targeted in traditional or alternative compensation models bears closer scrutiny. “Extensive research has found little that consistently distinguishes among teachers in their classroom effectiveness. Most well-documented has been the finding that master’s degrees bear no consistent relationship to student achievement.”11 However, there are some scholarly conclusions relating to years of experience that may relate to this topic.

First among these findings is that aside from the initial first three years of teaching, the remaining total years of experience appears to have no consistent relationship to student learning.12 Furthermore, “there is little evidence that conventional teacher certification, source of teacher training, or salary level are systematically related to the amount of learning that goes on in the classroom.”13

Certainly there is greater depth in any discussion of teacher quality, but as it pertains to associations with teacher compensation, the logical aim is to ensure that all schools are staffed with sufficiently remunerated personnel to produce high-quality growth outcomes in students.14 Salaries serve as a potential incentive “to attract and retain the teacher force that we need as a country.”15 Yet, our current traditional

11 Ibid.

12 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

13 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

14 Ibid.

15 Ibid.

15

configuration of the educational labor market may be exacerbating a continued impairment of this worthy intention.16 Among the most noticeable teacher salary characteristics is how intensely they have experienced comparative decline from a longitudinal perspective.17 One method to corroborate this claim is to study gender- specific teacher salaries over time. In 1940, the average male teacher’s salary was marginally above the 50th percentile when contrasted with the incomes of other male college graduates at that time, whereas the average female salary was approximately the 70th percentile compared to other college-educated females.18 Over the next several decades both salary levels dropped sharply for men to the bottom third of earned salary distribution and by the 1990s, female teacher salary levels were close to that male position, too.19 One important additional consideration is that the ranks of those interested in pursuing teaching as a career were often representing the lower strata of college graduates, indicating that most of the best students were not sufficiently attracted to consider teaching as a profession.20

The critical dilemma from this labor trend can best be expressed by Hanushek’s question: “How could we simultaneously place high value on teachers and their role in

16 Ibid.

17 Ibid.

18 Hanushek, E. A., and S. G. Rivkin. 2006. "Teacher Quality." In Handbook of the Economics of Education, edited by E. A. Hanushek and F. Welch, 1051-1078. Amsterdam: North Holland.

19 Ibid.

20 Bacolod, M. P. 2007. "Do Alternative Opportunities Matter? The Role of Female Labor Markets in the Decline of Teacher Quality." Review of Economics and Statistics 89 (4): 737-751.

16

society and yet let their pay slip so badly against others in the economy?”21 The most prevalent approach to teacher compensation, the single-salary model that pays teachers according to experience and degree-held nearly undoubtedly suppresses any significant potential to change teacher salaries.22 With most factors utilized to regulate compensation being disconnected from effectiveness, this critical lever acts as a restraint on overall rises in salary.23 Lastly, there is ambiguity in assessing the transferrable competitive value of a teacher’s training relative to a rival labor opportunity because “the skills needed to be an effective teacher are not necessarily those needed to be successful elsewhere in the economy.”24

It is generally accepted that a compensation level for a worker corresponds to the economic value of that position.25 “While the salaries of some professional athletes or celebrities may keep us from accepting this as a universal truth, it is nonetheless the case that the overall compensation pattern across the economy seems reasonable to most people.”26 Economists infer this configuration of earnings and production of workers as an expected consequence of a viable economy.27 Logically, if one business

21 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

22 Ibid.

23 Hanushek, Eric A. 2003. "The Failure of Input-Based Schooling Policies." Economic Journal 113 (485): F64-F98.

24 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

25 Ibid.

26 Ibid.

27 Ibid.

17

does not compensate an employee a wage that matches value in terms of productivity, a contending company would.28 However, if the employee were paid significantly beyond the value contributed, the firm would not be as competitive with other companies and would risk going out of business.29

Educational labor markets are distinctly different in that salary is established predominantly through collective bargaining efforts with individual school districts.30

While they are not subject to competitive shutdown in the same manner that for-profit businesses are, they are still subject to political forces in a similar manner to other public organizations.31 Due to the political aspects of educational salary, there are many potentially conflicting values to balance in any discussion of appropriate pay.32 Whether it be ideas such as fairness, competitive economic position, or focusing on overall quality of schools as the primary driver, there is no debating the significant value that high quality education can provide.33 The value of high quality education directly relates to completion of higher education, professional attainment, lifespan earnings, overall wellbeing, as well as tremendous societal benefits.34 From an economic standpoint,

28 Ibid.

29 Ibid.

30 Koppich, Julia E., and Jessica Rigby. March 27, 2009. Alternative Teacher Compensation: A Primer. ERIC Document Reproduction Service No. ED510159, Berkeley, CA: Policy Analysis for California Education, PACE.

31 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

32 Ibid.

33 Ibid.

34 Ibid.

18

policies that “enhance teacher pay can be justified if they improve the quality of schools, because both individuals and society gain.”35 Consequently, any policy derived from a benefit-cost structure in which increases from enhanced progress can be evaluated against the costs of any alteration should be convenient to assess.36 Subsequently, any observed lack of improvement would confer no value and if that included additional expense, it would yield costs exceeding benefits.37 From this framing using an economic lens, the implications of various salary-related perspectives in a direct contrast to the predominantly single-salary schedule are examined.

Alternative compensation models can be classified into three categories: payment for input, payment for output, and combinations of those two.38 Payment for input systems are often referred to as market-based systems and would include any compensation plan that incentivizes defined attributes of the applicant including potential features such as advanced degrees, qualified or specialized experience, or recruitment incentives that are not explicitly tied to any type of performance outcome on the part of the employee. A payment for output system, otherwise known as pay-for- performance or merit-pay system is designed to incentivize what the employee is able to produce in terms of valuable outcomes. The combination systems borrow features from both input and output systems. However, this study was focused on a particularly narrow subsection within the combination alternative compensation models: an

35 Ibid.

36 Ibid.

37 Ibid.

38 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

19

exceptionally high salary level for teachers with the underlying aim that those positions can only be maintained as long as the teacher is able to produce desired success for the target population of students.

The intention within this inquiry was to focus on schools or systems where there was a clear labor market advantage afforded to teachers with pay levels that vastly exceed typical compensation approaches and that due to that competitive advantage, only the most effective teachers would be able to retain this substantial compensation.

The rationale for this thinking is that most bonuses are insufficient to impact a prospective college student’s decision to choose education as a professional course of study or to sufficiently attract any eligible candidate to choose education as a long term, viable career.39 While there is no shortage of alternative compensation being deployed in educational systems, extraordinarily few systems result in teachers being paid above even the 50th percentile of college-educated workers.40 However, there are a few examples of schools or systems that have attempted to strategically pursue this approach either for all teachers or an intentionally selected group of the teaching force.

The longitudinal student learning results of these exclusive schools and systems are able to be examined for the compensation strategy’s apparent effectiveness and considered as a unique contribution to any examination of potential compensation approaches that may warrant further study.

39 Jupp, Brad. 2011. "What are Teachers in It For?" In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 155-163. Cambridge, MA: Harvard Education Press.

40 National Education Association. 2005. Rankings and Estimates: Rankings of the States 2004 and Estimates of School Statistics 2005. June. https://files.eric.ed.gov/fulltext/ED490871.pdf.

20

Population

The treatment population examined for this study consists of two middle schools where the entire faculty of each school received a level of compensation that qualifies as substantial for the purposes of this treatment. One school was a public charter school in the State of New York and the other school was a traditional public school in the State of Florida. Both of these schools served high poverty, high minority populations and this substantial compensation approach was being utilized as a strategy to significantly enhance the level of learning beyond what would be otherwise typical for schools that serve similar populations of students. The primary reason these two schools were chosen for this study was that due to the extremely limited number of schools that satisfy the threshold to qualify as substantial compensation, these schools were among some of the only schools in the entire United States that offered this substantial compensation approach. Additionally, the entire faculties received this substantial compensation and thusly the level of measurement for the treatment could be aggregated school-wide data as opposed to tracking individual teachers with individual classroom performance. All of the student performance data for these schools was provided by each state’s Department of Education. The school populations were described with socio-economic measures, racial composition, and comparable student achievement levels all information collected and reported by each state’s Department of

Education. The respective comparison peer samples of schools were selected from each state utilizing the equivalent socio-economic measurement thresholds, comparable racial composition, and student achievement levels of the two treatment schools in order to find appropriate comparison samples of schools for this study.

21

Statement of Purpose

The purpose of this study was to evaluate to what extent a substantial increase in teacher compensation levels may produce a positive statistically significant impact on student learning.

Research Question

This study answered the following research question:

 Was there a positive statistically-significant relationship in student learning growth at schools that utilized a front-loaded, substantial compensation model when contrasted with typical compensation schools that served a similar demographic population?

Research Hypothesis

This study accepted the following as a null hypothesis:

 There was not a statistically significant difference in student learning growth at schools that utilize a front-loaded, substantial compensation model when contrasted with a traditional compensation model when comparing with similar school demographics characteristics (student enrollment, percentage of students on free and reduced lunch).

Assumptions

The following assumptions were made regarding this study:

 Student-level data were collected and measured without error

 Student-school linkages relating student achievement information reflect eligible students who were taught by teachers receiving the substantial compensation described in the treatment

 Student growth calculations were determined by each State accountability measurement system and were presumed to be suitable growth metrics as they were deployed statewide for all public schools as common industry practice utilized for detecting effective changes in aggregate student learning

 Student enrollment characteristics within the schools selected were expected to be relatively similar to other schools with comparable demographic characteristics

22

 This study examined intact groups of students as a convenience sample based upon the extremely limited number of schools implementing conditions aligned with the treatment

Limitations

Due to a very small sample size of schools that fit the profile of a substantial, front-loaded salary compensation model, one major limitation was the potential lack of generalizability due to that extremely small sample. With such a limited number of schools, there may be other factors that contribute to a school’s success such as inadvertent enrollment impact due to the exclusivity and publicizing of the experimental school. Any family-targeted enrollment or student selectivity-related issues that may result in a potentially more motivated and higher achieving student than is otherwise represented in the demographic sampling characteristics employed within this study may not have been reasonably detected within this study’s available student performance data. Furthermore, due to the observed small samples in each region and a lack of comparable metric between regions, that further restricted the comparative analysis. Additionally, in the Florida treatment school, the compensation scheme was deployed for only a single year whereas the New York treatment school had implemented the compensation strategy for multiple years. This created a specific limitation for the Florida sample not being able to be examined across multiple years but this was not an issue for the New York sample. Since the samples involved were quite small, generalizability was acknowledged to be limited, but any positive statistically significant relationships could suggest additional research and continued emphasis on this approach.

23

Summary

The impact of the substantially higher teacher salary is already being implemented at a small sample of schools and the student performance data are compiling with each year. These schools have typically received considerable media attention and have stimulated broader policy conversation around the significance of this type of compensation approach specifically as it relates to variables such as teacher shortages in high-need schools, turnaround schools, high poverty schools, and resource-limiting factors such as class size. If the most effective teachers were given a significant salary level; one that greatly exceeds the conventional bonuses that are offered as incentives and approaches the doubling of a teacher’s typical initial salary level as a starting point, then the impact of that approach on student learning may warrant empirical examination.

This study compared a small sample of schools that feature this substantial compensation as the core strategy to enhance teacher quality and increase effectiveness of student learning. Research pertaining to this type of compensation approach was reviewed and summarized. Student learning results in English language arts and mathematics were compiled for treatment schools along with demographically comparable schools utilizing socio-economic measurements, racial composition, and similar student achievement levels. The results were analyzed and compared for a limited generalization based on this small sample.

24

CHAPTER 2 LITERATURE REVIEW

The Early History of American Teacher Compensation

As the American economy transitioned from an agrarian to industrialized base at the turn of the twentieth century, the identity of public education shifted as well.1 This reconceptualization emphasized an approach to develop quality citizens, unify the general public, and diminish poverty as well as criminal behavior.2 With this evolving focus, buttressed with enhanced professionalism in the teaching field, compensation approaches transitioned from the grade-based method into the single-salary schedule.3

“The grade-based compensation model paid teachers according to the level of schooling taught, and many of these models rewarded teachers based on annual performance reviews completed by school administration.”4 This early version of merit pay usually resulted in equity issues between gender and race with white males receiving substantially more pay than all other groups of teachers.5 Additionally, the difference between secondary teachers and elementary, primarily female teachers was significant as well.6 By the year 1918, Nearly one-half of school districts featured

1 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

2 Ibid.

3 Murnane, Richard J, and David Cohen. 1986. "Merit Pay and the Evaluation Problem: Why Most Merit Pay Plans Fail and Few Survive." Harvard Education Review 56 (1): 1-17.

4 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

5 Adkins, Gary A. 1983. Pros and Cons and Current Status of Merit Pay in the Public Schools. Eric Resources Information Center No. ED238-162, Radford, VA: Virginia Association of Teacher Educators.

6 English, Fenwick. 1992. "History and Critical Issues of Educational Compensation System." In Teacher Compensation and Motivation, edited by Larry Frase, 3-26. Lancaster, PA: Technomic Publishing.

25

comparable merit pay approaches within their respective grade-based compensation systems.7

During the women’s right movement emphasis on equivalent wages for comparable work, school districts began developing and implementing more equal teacher compensation approaches.8 “In 1921 Denver and De Moines introduced the single salary schedule, which has underpinned teacher pay practices… according to two criteria thought to be most central to teacher productivity- years of service and degree held.”9 While this served to create a model that was equitable irrespective of gender, race, or grade level of students taught, it also marked the end of merit pay approaches at the time.10 As a very predictable pattern of compensation, the single salary schedule alleviated tension from school district salary negotiations, which was valued due to a problematic disposition around labor-relations in American society during this time period.11

This new, single-salary schedule approach was not ubiquitously accepted, with opponents demanding a compensation approach that results in a scientific return on investment for public funds from a socio-economic value as opposed to any sentimentality for the workers.12 This rival attitude was largely influenced by Frederick

7 Ibid.

8 Davis, Hazel. 1943. "Teachers' Salaries." Review of Educational Research 13 (3): 276-284.

9 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

10 Ibid.

11 Ibid.

12 Moehlman, Arthur B. 1927. Public School Finance. New York: Rand-McNally.

26

Taylor’s scientific management theory.13 These concerns about treating all teachers as if they were equal in terms of performance was viewed with scrutiny with one

Superintendent at the time, Aaron Sheeley, equated that the single salary schedule allocated “a premium to mediocrity, if not to positive ignorance and incompetency.”14

However, as World War II ended, the school population substantially increased and more teachers were needed, so this standardized, predictable method of the single- salary schedule offered a stable and secure recruitment structure.15 This blooming new teaching profession appeared to be a genuine favoring of egalitarianism over competition and the salary structure affirmed that notion.16 Furthermore, quality of the teaching force was being assessed via experience and additional training as a proxy for expertise with more experienced teachers thusly being deemed as deserving of greater value from a performance standpoint without any research to validate that type of thinking.17 Nevertheless, by 1950 the single salary schedule was implemented by 97 percent of all American public schools and has remained the prevailing approach ever since.18

13 Mitchell, D. J. B., D. Lewin, and E. E. Lawler. 1990. "Alternative Pay Systems, Firm Performance, and Productivity." In Paying for Productivity: A Look at the Evidence, edited by Alan S Blinder. Brookings.

14 Odden, Allan, and Carolyn Kelly. 2002. Paying Teachers for What They Know and Do: New and Smarter Compensation Strategies to Improve Schools. Thousand Oaks, CA: Corwin Press.

15 Koppich, Julia E., and Jessica Rigby. March 27, 2009. Alternative Teacher Compensation: A Primer. ERIC Document Reproduction Service No. ED510159, Berkeley, CA: Policy Analysis for California Education, PACE.

16 Lipsky, D B, and S B Bacharach. 1983. "The Single Salary Schedule vs. Merit Pay: An Examination of the Debate." Collective Bargaining Quarterly 11 (4): 1-11.

17 Kerchner, C T, J Koppich, and J Weeres. 1997. United Mind Workers: Unions and Teaching in the Knowledge Society. San Francisco: Jossey-Bass.

18 Protsik, Jean. 1995. History of Teacher Pay and Incentive Reform. Washington: Educational Resources Information Center.

27

With the beginning of collective bargaining in the 1960s and 1970s, the unions representing teachers- affiliates of the National Education Association (NEA) and the

American Federation of Teachers (AFT) accepted the single salary schedule as the compensation procedure that presented teachers with a structure of equitable, objective, and predictable salary distribution.19 Teachers could rely on years of service and overall college credit earned to predict not just one year, but the entirety of their career.20 However, this is not to suggest that there have not been reform efforts even with the prevailing implementation of the single salary schedule. “Efforts to reform teacher compensation policies have emerged in virtually every decade since the

1950s.”21 While there are considerable varieties of alternative compensation models, most can be broadly classified into approximately four categories: pay-for-performance, knowledge and skills-based pay, career ladder programs, and market-based pay.22

These four categories of alternative compensation approaches are explored in subsequent sections.

Emerging Concerns with the Single Salary Schedule

As an approach that compensates teachers exclusively on the foundation of years served and college credit earned has the virtue of simplicity and ostensibly equity,

19 Kerchner, C T, J Koppich, and J Weeres. 1997. United Mind Workers: Unions and Teaching in the Knowledge Society. San Francisco: Jossey-Bass.

20 Koppich, Julia E., and Jessica Rigby. March 27, 2009. Alternative Teacher Compensation: A Primer. ERIC Document Reproduction Service No. ED510159, Berkeley, CA: Policy Analysis for California Education, PACE.

21 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

22 Ibid.

28

but it yields neither professionally competitive nor market sensitive earnings.23 With minimal or modest incentives and no priority to distinguish exemplary teaching, it does not address the reality that some teaching assignments are more difficult than others and some subject area vacancies are more challenging to fill.24 These persistent concerns speak to the core efficacy issue with the single salary system: the architecture of the salary system and the challenge that it should reflect what matters most for improving student achievement.25 Policy examination and reform efforts focus on the basic principles that undergird teacher salaries.26 Whether these evolving methods of compensation are represented under any specific reform initiative, they are all focused on modifying, overhauling, or replacing the single salary schedule.27 While the single salary schedule attempts to feature input values in the form of supplementary compensation for additional graduate level work and years of service, teaching experience and advanced degrees held account for only a very small percentage of the explainable benefits attributed to a teacher’s impact on student learning.28 Additionally,

23 Koppich, Julia E., and Jessica Rigby. March 27, 2009. Alternative Teacher Compensation: A Primer. ERIC Document Reproduction Service No. ED510159, Berkeley, CA: Policy Analysis for California Education, PACE.

24 Ibid.

25 Loeb, S, and M Reininger. 2004. Public policy and teacher labor markets: What We Know and Why It Matters. Lansing, MI: The Education Policy Center at Michigan State University.

26 Koppich, Julia E., and Jessica Rigby. March 27, 2009. Alternative Teacher Compensation: A Primer. ERIC Document Reproduction Service No. ED510159, Berkeley, CA: Policy Analysis for California Education, PACE.

27 Ibid.

28 Goldhaber, D. 2006. Teacher Pay Reforms: The Political Implications of Recent Research. Washington, D.C.: The Center for American Progress.

29

the single salary schedule tethers teachers to a consistently slow salary increase over the full span of the career.29

Alternative Teacher Compensation Reform

With increasing student enrollment since the 1980s, the consistency of school districts being able to attract and retain high performing teachers is a challenge that continues to persist in spite of consistent national supply from colleges of education.30 It is observed that a portion of schools appear to have challenges with recruitment and retention while some schools do not experience this issue.31 Ingersoll estimates that roughly 50 percent of teacher turnover can be classified as ‘teacher migration’ where teachers transition employment from one school to another school within a district or state,32 signifying that some schools are less desirable than other schools, with one of the most significant factors being the demographic characteristics of a school’s student population.33 When a school’s percentage of enrolled minority student exceeds 20 percent, irrespective of location, these schools have a higher proportion of

29 Koppich, J. 2008. Toward a More Comprehensive Model of Teacher Pay. Paper prepared for National Center on Performance Incentives’ Conference: Performance Incentives—Their Growing Impact on American K-12 Education, NCPI Working Paper 2008-06, Nashville: Vanderbilt University.

30 Ingersoll, Richard. 2003. Is There Really a Teacher Shortage? The Consortium for Policy Research in Education and The Center for the Study of Teaching and Policy. Accessed November 2, 2018. http://repository.upenn.edu/gse_pubs/133.

31 Ibid

32 Ingersoll, Richard. 2001. Teacher Turnover, Teacher Shortages, and the Organization of Schools. Seattle: Center of the Study of Teaching and Policy, University of Washington.

33 Hassel, B. C. 2002. Better Pay for Better Teaching: Making Teacher Compensation Pay Off in the Age of Accountability. Washington, DC: Progressive Policy Institute, 21st Century Schools Project.

30

disadvantaged students and lagging academic performance that may denote these as

‘hard-to-staff’ schools.34

When schools are under-performing, this lagging student performance is exacerbated by lowered staff expectations for student ability, inadequate curriculum, and diminished parental engagement.35 These under-performing schools disproportionately hire less-qualified, more inexperienced, and insufficiently trained teachers in addition to often having larger class sizes and lower levels of student access to advanced coursework.36 Teachers often have less motivation to remain in lower performing schools with typically high levels of employee dissatisfaction and where working conditions are generally described as poor quality.37 Many of the teachers characteristically leave the school once they are able to secure a more desirable location or exit the educational field altogether.38 Factors that are cited in teacher exit include low salary, insufficient administrative support, student misconduct, and a lack of teacher input into decision-making.39

34 Lashway, Larry. 2003. Finding Leaders for Hard-to-Staff Schools. ERIC Digest 173, December. Accessed October 11, 2018. https://scholarsbank.uoregon.edu/xmlui/bitstream/handle/1794/3234/digest173.pdf;sequence=3.

35 Cohen, M., and A. Ginsburg. 2001. School Improvement Report: Executive Order on Actions for Turning Around Low-Performing Schools. Washington, DC: Department of Education.

36 Berry, B, L Turchi, and D Johnson. 2003. The Impact of High-Stakes Accountability on Teachers' Professional Development: Evidence from the South. Chapel Hill, NC: Southeast Center for Teaching Quality.

37 Ansell, S, and M McCabe. 2003. Off Target. Accessed October 26, 2018. http://counts.cdweck.org/sreports/qc03/templates/article.cfm7slugM.

38 Wise, A. 2004. "Teaching Teams." Education Weekly. Accessed October 25, 2018. https://www.edweek.org/ew/articles/2004/09/29/05wise.h24.html.

39 Ingersoll, Richard. 2001. Teacher Turnover, Teacher Shortages, and the Organization of Schools. Seattle: Center of the Study of Teaching and Policy, University of Washington.

31

The most challenging schools can be improved with continuity of a teaching force as compounded professional learning and experience improve the faculty performance.40 Yet, often these challenging schools are some of the most negatively impacted when it comes to teacher retention41 and enhancing the combined social capital to develop leadership among the faculty.42 These considerably high rates of staff turnover negatively impact student performance as well as the whole school community.43 With staffing constituting such a critical variable for these precarious schools, various forms of differentiated compensation plans have emerged as alternatives to address these challenges.44

With increased levels of accountability being focused on schools, this additional pressure has intensified the challenge of recruitment and retention for hard-to-staff schools.45 This type of issue is most prevalent in low-income urban and remote rural schools often negatively impacted by lower community tax bases; especially when

40 Hassel, B. C. 2002. Better Pay for Better Teaching: Making Teacher Compensation Pay Off in the Age of Accountability. Washington, DC: Progressive Policy Institute, 21st Century Schools Project.

41 Day, D V. 2000. "Leadership Development: A Review in Context." Leadership Quarterly 11 (4): 581- 613.

42 Baptiste, I. 2001. "Educating Lone Wolves: Pedagogical Implications of Human Capital Theory." Adult Education Quarterly 51 (3): 184-202.

43 Ingersoll, Richard. 2001. Teacher Turnover, Teacher Shortages, and the Organization of Schools. Seattle: Center of the Study of Teaching and Policy, University of Washington.

44 Kotterman, P. 2000. "The Teacher Quality Continuum: Recruitment, Preparation, and Retention in an Era of Teacher Shortages." Teachers: Supply and demand in an age of rising standards. Amherst, MA: National Evaluation Systems. 105-120. Accessed October 27, 2018. https://images.pearsonassessments.com/images/NES_Publications/2000_07Kotterman_430_1.pdf.

45 Ansell, Susan E, and Melissa McCabe. 2003. Off Target. Accessed October 26, 2018. http://counts.cdweck.org/sreports/qc03/templates/article.cfm7slugM.

32

contrasting competitive teacher salary with their suburban counterparts.46 These imbalances are more noteworthy as it pertains to significant differences between specific subjects such as areas of math, science, and special education.47 Teachers migrating away from schools serving disadvantaged populations contribute to that critical subject area shortage issue.48 The distribution of these subjects varies considerably, but some estimate that there are four times as many under-qualified math and science teachers residing in hard-to-staff schools as compared with typical schools.49 Wise suggests that this negative impact on student achievement may be diminished with strategic financial incentives and other approaches to improve teacher recruitment and retention.50

International Context for Alternative Compensation

While education in the United States of America has existed in some form since the birth of our nation, in a global context, education has existed far longer in other parts of the world. It is not surprising then that each country’s approach to not only education but to teacher compensation varies as well.51 Beginning with information from the

Organization for Economic Cooperation and Development (OECD), Dolton and

46 Glennie, E., C. Coble, and M. Allen. 2004. Teacher Perceptions of the Work Environment in Hard-to- Staff Schools. Denver, CO: Education Commission of the States.

47 Certo, J., and J. E. Fox. 2002. "Retaining quality teachers." High School Journal 86 (1): 57-76.

48 Hirsch, E. 2001. Teacher Recruitment: Staffing Classrooms With Quality Teachers. ERIC Document Reproduction Service No. ED453199, Denver, CO: State Higher Education Executive Officers.

49 Esch, C. E., C. M. Change-Ross, R. Guha, D. C. Humphrey, P. M. Shields, and J. D. Tiffany-Morales. 2005. The Status of the Teaching Profession 2005. Santa Cruz, CA: Center for the Future of Teaching and Learning.

50 Wise, A. 2004. "Teaching Teams." Education Weekly. Accessed October 25, 2018. https://www.edweek.org/ew/articles/2004/09/29/05wise.h24.html.

51 Ripley, Amanda. 2013. The Smartest Kids in the World. New York: Simon & Schuster, Inc.

33

Marcenaro-Gutierrez conducted an analysis of teacher salary level and Program for

International Student Assessment PISA (PISA) scores, resulting in a clear statistical association between higher relative pay for teachers and higher student performance across countries.52 Two potential explanations offered by the researchers for this correlation was that higher pay attracted better graduates and thus increased competitiveness in the hiring pool as well as that increased pay level relative to other professions raised the national status of the profession and thusly incentivized better students to consider pursuing the field.53 With increased United States public school spending while largely retaining the traditional single salary schedule, student achievement has not substantially increased; whereas other countries that have implemented compensation reform have in many cases but not exclusively observed improved student performance.54 While there is no clear consensus in an international approach to teacher compensation and comparing schools in other countries is a challenge due to significantly different operating contexts,55 it is evident that improvements in educational performance appear as a key factor contributing to economic growth in contemporary decades for all OECD countries.56

52 Dolton, Peter, and Oscar Marcenaro-Gutierrez. 2011. "If You Pay Peanuts, Do You Get Monkeys? A Cross-country Analysis of Teacher Pay and Pupil Performance." Economic Policy 26 (65): 5-55. Accessed October 21, 2018. http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0327.2010.00257.x/full.

53 Ibid.

54 Lavy, V. 2007. "Using Performance-Based Pay to Improve the Quality of Teachers." The Future of Children 17: 87-109.

55 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

56 Dolton, Peter, and Oscar Marcenaro-Gutierrez. 2011. "If You Pay Peanuts, Do You Get Monkeys? A Cross-country Analysis of Teacher Pay and Pupil Performance." Economic Policy 26 (65): 5-55. Accessed October 21, 2018. http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0327.2010.00257.x/full.

34

Differentiated Teacher Compensation Methods

Consistent with the impetus of teacher compensation reform, there are some broad categories to characterize these unique methods to differentiate pay for teachers.

These include the following: Pay-for-performance, extra work for extra pay, knowledge- and skills-based pay, career ladder programs, and market-based pay.57 Within these classifications, there are unique examples of how schools, districts, state legislatures, or state departments of education have attempted to develop policies or approaches that fit inside each of these categories.

Pay-for-Performance

The pay-for-performance programs have some of the most diverse and varied applications of this concept58 while also serving as among the most controversial of methods.59 This category includes such concepts as Merit Pay and is most commonly associated to a teacher receiving additional compensation based on a teacher’s ability to raise student performance as measured on standardized tests.60 However, this is not the exclusive purpose of pay-for-performance schemes. The critical variance in these approaches centers on whose performance should regulate bonus reward entitlement, which performance indicators will be considered, whether the standard will be criterion- based or relative within a population, and business rules around distribution eligibility

57 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

58 Ibid.

59 Koppich, Julia E., and Jessica Rigby. March 27, 2009. Alternative Teacher Compensation: A Primer. ERIC Document Reproduction Service No. ED510159, Berkeley, CA: Policy Analysis for California Education, PACE.

60 Ibid.

35

and procedures.61 Thusly, establishing a comprehensive research base is essential in evaluating the potential of any pay-for-performance design model as well as interpreting each potentially unique approach.62

Those merit pay systems that were viewed more favorably and lasted over a prolonged period of time had the following common characteristics: (1) high teacher morale, (2) performance pay was not punitive toward less effective teachers, (3) communities served had greater levels of affluence, (4) public schools were regarded positively, (5) teacher salaries were already typically characterized as higher base pay prior to the implementation of additional performance pay, and (6) teacher evaluation approaches were considered effective measurement processes within those school systems.63 However, the research literature pertaining to evaluating the effectiveness of performance pay systems “is surprisingly thin considering the number of schools, districts, and states that have adopted teacher compensation reforms.”64 Numerous rigorous compensation reform evaluations pertaining to performance pay have been conducted abroad and generally confirm a typically positive influence on student learning.65 However, the nature of that positive impact requires additional scrutiny as the

61 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

62 Ibid.

63 Murnane, Richard J, and David Cohen. 1986. "Merit Pay and the Evaluation Problem: Why Most Merit Pay Plans Fail and Few Survive." Harvard Education Review 56 (1): 1-17.

64 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

65 Ibid.

36

learning may not persist over time and effects may result more from varying operational context from what is typically encountered in American public schools.66

One such pay for performance scheme that was thoroughly studied was the

Project on Incentives in Teaching (POINT) a study conducted over three years in the

Metropolitan Nashville School System from 2006 to 2009, in which middle school math teachers voluntarily joined in a controlled experiment to gauge the impact of financial rewards on teachers whose students showed statistically significant positive gains on annual standardized tests.67 The intention of the experiment was to examine the concept that rewarding teachers for superior scores would cause scores to increase, but it was up to participating teachers to choose what they desired to do to increase student performance: partake in additional professional learning, receive coaching, increase teacher collaboration, or engage in additional reflective practice.68 Thusly, POINT was fixated on the conception that the substantial issue in American education is the lack of suitable incentives, and that improving the incentive configuration would, in and of itself, create an appropriate intervention that upgraded student performance, but the results of the study did not ultimately support that hypothesis.69

There are numerous well-established critiques of pay-for-performance schemes to be found in the research literature. First among them is not wanting to undermine the

66 Ibid.

67 Springer, Matthew G, Dale Ballou, Laura Hamilton, Vi-Nhuan Le, J. R. Lockwood, Daniel F. McCaffrey, Matthew Pepper, and Brian M. Stetcher. 2012. Final Report: Experimental Evidence from the Project on Incentives in Teaching. Nashville, TN: National Center on Performance Incentives at Vanderbilt University.

68 Ibid.

69 Ibid.

37

traditionally cooperative and collaborative nature of teaching that would most likely occur in an individual performance reward system.70 Additionally, another critical issue is the nature of the educational field lacking optimally reliable, valid, and fair measurement systems upon which to endow performance bonuses.71 Inventive metrics such as the Value-Added Model (VAM) which attempt to take into consideration varying student characteristics in order to generate more equitable expectations for teacher performance still rely predominantly on state standardized assessments that align to only one-third of teaching positions.72 Furthermore, teachers are expected to assist students in a vast array of activities and reaching developmental milestones that are simply are not captured within a state or federal accountability program.73 Lastly, even in the cases where short-term benefits of pay-for-performance are observed, there is concern that this type of approach will result in “crowding out of intrinsic motivation over time may reduce effort, self-esteem, and originality to the point of negatively affecting teacher and school productivity.”74

70 Murnane, Richard J, and David Cohen. 1986. "Merit Pay and the Evaluation Problem: Why Most Merit Pay Plans Fail and Few Survive." Harvard Education Review 56 (1): 1-17.

71 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

72 Ibid.

73 Prince, Cynthia D. 2009. The Other 69 Percent: Fairly Rewarding the Performance of Teachers of Non- Test Subjects. Office of Elementary and Secondary Education, U.S. Department of Education, Washington: The Center for Educator Compensation Reform.

74 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

38

Knowledge- and Skills-based Pay

When teachers received differentiated compensation based on the skills they are able to demonstrate or the substantiation of knowledge they have acquired as opposed to simply the job title they hold, this payment strategy is collectively referred to as knowledge- and skills-based compensation.75 This type of approach is also cited as

“competency pay” and is largely designed to incentivize teachers on the basis of pursuing professional growth.76 This method focuses on skills that can be classified by depth, breadth, or vertical categories where depth refers to the increasing level of complexity of a particular skill, whereas breadth represents a variety of skills to be captured, and vertical is designated for those competencies most related to leadership and management viewed through an organizational context.77 Within an empirical review of seven knowledge- and skills-based pay systems, Milanowski appraised that utilizing additional compensation for the purpose of acquiring enhanced pedagogical skill would serve as an adequate incentive.78 However, the additional critical finding from Milanowski’s analysis was that although the incentive would be sufficient to attract teachers to this acquisition of skills, this would not necessarily be sufficient to improve student learning results.79

75 Odden, Allan, and Carolyn Kelly. 2002. Paying Teachers for What They Know and Do: New and smarter compensation strategies to improve schools. 2nd Edition. Thousand Oaks, CA: Corwin Press.

76 Stronge, J. H., C. R. Gareis, and C. A. Little. 2006. Teacher Pay and Teacher Quality: Attracting, Developing and Retaining the Best Teachers. Thousand Oaks, CA: Corwin Press.

77 Ibid.

78 Milanowski, A. 2003. "The Varieties of Knowledge and Skill-Based Pay Design: A Comparison of Seven New Pay Systems for K-12 Teachers." Education Policy Analysis Archives 11 (4). Accessed October 22, 2018. https://epaa.asu.edu/ojs/article/view/232.

79 Ibid.

39

When the educational knowledge- and skills-based pay is contrasted with similar payment systems in the private sector, the significant costs associated to the professional learning can be partially if not completely offset by increased profits; this is not the case in a publicly-funded enterprise such as schools systems where increased students learning does not typically result in increased overall funding for any given district.80 However, attempts have been made to create a visible linkage between increased competence of teachers through skill-acquisition should result in increased student mastery of educational standards while honoring the associated taxpayer interest where increased taxes should result in increased student achievement.81 Yet, one additional challenge within the knowledge- and skills-based compensation system is capturing an accurate measurement of the supposedly acquired skills through an evaluation system.82 Having an evaluation system that accurately and equitably generates meaningful records of which teachers are able to implement learned skills continues to draw scrutiny and generally lack the support of most teacher unions due to this lack of precision.83 Milanowski concluded that due to the multifaceted issues ranging from appropriate evaluation, equitable treatment of teachers, and valid improvement of student learning, tremendous trust among unions and district leadership

80 Firestone, W. A. 1994. "Redesigning Teacher Salary Systems for Educational Reform." American Educational Research Journal 549-574. Accessed October 24, 2018. http://www.aera.net/Publications/Journals/JSTOR-Online-Archives.

81 Stronge, J. H., C. R. Gareis, and C. A. Little. 2006. Teacher Pay and Teacher Quality: Attracting, Developing and Retaining the Best Teachers. Thousand Oaks, CA: Corwin Press.

82 Milanowski, A. 2003. "The Varieties of Knowledge and Skill-Based Pay Design: A Comparison of Seven New Pay Systems for K-12 Teachers." Education Policy Analysis Archives 11 (4). Accessed October 22, 2018. https://epaa.asu.edu/ojs/article/view/232.

83 Odden, Allan, and Carolyn Kelly. 2002. Paying Teachers for What They Know and Do: New and smarter compensation strategies to improve schools. 2nd Edition. Thousand Oaks, CA: Corwin Press.

40

must be established for any knowledge- and skills-based compensation system to succeed.84 However, even with an empirical review of seven successful implementations of this approach, the variance in student achievement that is attributable to teacher skills as assessed within an evaluation system accounted for a minimal range of 9 percent to 16 percent with correlations from 0.3 to 0.4.85

Career Ladder Programs

Kelly and Odden characterize career ladders by labeling it as a prospect to rework the flat career configuration of education by providing teachers career opportunities outside the typical classroom.86 In the 1980s as this type of compensation reform began appearing at a larger scale, career ladders were envisioned to recognize the finest teachers and reward them with leadership roles that focus on such specialized tasks as delivering professional development or writing curriculum which could occur either completely outside the classroom role or above and beyond that role with additional payment.87 However, there are numerous issues that are raised with this approach: a perceived lack of equity in the selection of teachers88 for these roles as well as depleting what is perceived as an already thin talent pool of classroom teachers by removing the best teachers from classroom roles who in turn “have not been very

84 Milanowski, A. 2003. "The Varieties of Knowledge and Skill-Based Pay Design: A Comparison of Seven New Pay Systems for K-12 Teachers." Education Policy Analysis Archives 11 (4). Accessed October 22, 2018. https://epaa.asu.edu/ojs/article/view/232.

85 Ibid.

86 Odden, Allan, and Carolyn Kelly. 2002. Paying Teachers for What They Know and Do: New and smarter compensation strategies to improve schools. 2nd Edition. Thousand Oaks, CA: Corwin Press.

87 Ibid.

88 Firestone, W. A. 1994. "Redesigning Teacher Salary Systems for Educational Reform." American Educational Research Journal 549-574. Accessed October 24, 2018. http://www.aera.net/Publications/Journals/JSTOR-Online-Archives.

41

successful in growing more effective teachers.”89 While the positive aspects of the career ladder are designed to manifest in career advancement opportunities intended to boost retention and a variation in duties and tasks projected to neutralize educational career stagnation,90 this still does not address the nature of a teacher’s output being hard-to-measure in a consistent, valid, and impartial fashion and thusly justify adding that same added-value back to those consequent classroom vacancies.91 Additional challenges experienced in career ladder designs have included an insufficient level of teacher involvement in system design, lack of long-term organizational commitment to funding, and teacher union opposition.92

An additional type of pay that can add to the otherwise flat pay structure is when teachers are offered additional compensation for extra time or duties that they willingly accept that can be referred to as “extra work for extra pay.”93 This additional compensation can appear either as additional hours of pay due to a longer teacher workday or is included as a salary supplement in a standard district pay scale to include such example duties as mentoring new teachers, designing curriculum, or supervisory

89 Podgursky, Michael, and Matthew Springer. 2011. "Teacher Compensation Systems in the United States K-12 Public School System." National Tax Journal 64 (1): 1-28. Accessed October 27, 2018. https://faculty.missouri.edu/~podgurskym/doc/articles/18teachercompensationsystemsart2011-36.pdf.

90 Cresap, McCormick, and Paget, Inc. 1984. Teacher Incentives: A Tool for Effective Management. Washington: National Association of Secondary School Principals.

91 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

92 Odden, Allan, and Carolyn Kelly. 2002. Paying Teachers for What They Know and Do: New and smarter compensation strategies to improve schools. 2nd Edition. Thousand Oaks, CA: Corwin Press.

93 Björk, L. G. 2004. Interim report: Differentiated compensation research project. Frankfort, KY: Kentucky Department of Education.

42

duties above and beyond an otherwise regularly expected duties and results in this supplemental pay.94

Market-Based Pay

Market-based compensation seeks to address four broad types of strategies within education through targeted pay tactics: the overall lack of quality in teacher supply within a supply-demand context, recruitment or retention award pay, hard-to-staff subjects, and hard-to-staff schools.95 While these four types of strategies are all classified as market-based pay programs, they have a philosophical basis in the concept of the free market compensation approach as being a potential answer for the educational labor challenge. Beginning with the larger issue of a teacher labor shortage or at least a perceived lack of quality talent, the market-pay response begins with adjusting teacher salaries to match current earnings prior to the observed decline in relative professional compensation levels.96 However, there are numerous assumptions and critical policy considerations that undergird this otherwise seemingly simple premise.

The first assumption is that if teaching salaries were as competitive at present as they once were, similarly talented people would be attracted to the profession.97 A

94 Painter, Suzanne, Thomas Haladyna, and Sally Hurwitz. 2007. "Attracting Beginning Teachers: The Incentives and Organizational Characteristics that Matter." Planning and Changing 38 (1-2): 108-127.

95 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

96 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

97 Ibid.

43

second assumption is that the newly attracted recruits would have talent that is better than those currently employed and that schools would choose and retain those teachers from this expanded applicant pool.98 Milanowski concluded that attracting large numbers of college students outside an expressed interest in education would necessitate substantially higher starting salaries than the average starting teacher salary in the

Midwestern state where his research was conducted.99 However, if setting teacher salaries at a market rate in an open economy were the suggested course of policy action, it is not well established within professional literature which occupations would provide an appropriate comparison since most private comparisons have salaries structured to distinct positional productivity and accordingly greater discrepancies in wages to reflect larger employment risks.100

The second type of market-based pay strategy is recruitment or retention award pay. In these market-based pay schemes, rewards or bonuses are offered to attract educators to a school, district, or state and to encourage continued years of service.101

These rewards or bonuses can vary considerably in terms of amount, conditions under which the bonus is received, and overall structure with payments fitting into varying models such as signing bonuses, targeted salary escalations, housing inducements,

98 Ibid.

99 Milanowski, A. 2003. "The Varieties of Knowledge and Skill-Based Pay Design: A Comparison of Seven New Pay Systems for K-12 Teachers." Education Policy Analysis Archives 11 (4). Accessed October 22, 2018. https://epaa.asu.edu/ojs/article/view/232.

100 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

101 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

44

tuition assistance, or tax incentives along with numerous other methods of financial benefit.102 However, the substantial nature of the additional compensation and the permanence or temporary nature of the reward pay are critical features that must be considered as all bonuses are not created equally.103 One of the challenges in reviewing the research on rewards and bonus systems within education is that rarely does the size of the reward receive scrutiny for significance as many bonus programs offer only a slight monetary amount in contrast with the basic architecture of the traditional salary structure remaining virtually unchanged.104

The third and fourth types of market-based pay strategies are specific forms of recruitment and retention bonus systems focused on hard-to-staff schools or distinct subject areas experiencing a teacher shortage.105 The primary intention in these market-based reform strategies are intended to positively offset the inequitable dispersal of higher quality and more experienced teachers among schools serving diverse socio-economic student populations in addition to addressing teacher shortages in subjects such as special education, science, and mathematics.106 While there is considerable discussion of market-based incentive programs in the research literature,

102 Prince, C. D. 2003. Higher Pay in Hard-To-Staff Schools. Lanham, MD: University Press of America, Inc.

103 Odden, A,, and M. Wallace. 2004. "Experimenting with Teacher Compensation." The School Administrator (61): 24-28. Accessed October 29, 2018. http://www.wilsonweb.com.

104 Koppich, J. 2008. Toward a More Comprehensive Model of Teacher Pay. Paper prepared for National Center on Performance Incentives’ Conference: Performance Incentives—Their Growing Impact on American K-12 Education, NCPI Working Paper 2008-06, Nashville: Vanderbilt University.

105 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

106 Prince, C. D. 2003. Higher Pay in Hard-To-Staff Schools. Lanham, MD: University Press of America, Inc.

45

there is a negligible amount of professional empirical analysis regarding the effectiveness of such market-based incentive approaches in spite of numerous large- scale implementations of these strategies.107 In an example of a smaller bonus program, a North Carolina district found a positive association between an annual

$1,800 bonus for teachers in high-poverty schools being certified in mathematics, sciences, or special education in reducing mean turnover rates by 17 percent.108 In an example of a typical larger bonus program, a specific Massachusetts signing bonus program offered $20,000 to entice high quality individuals into teaching that may not otherwise have selected to work in education with a preliminary payment of $5,000, and the remainder of $15,000 to be awarded over the next four years, but ultimately the initiative was less effective at recruitment and retention than the system’s existing alternative certification programs.109 These examples are offered to illuminate what a typical bonus approach resembles but offer little in generalizing what the overall value of these type of market-based approaches may be. It is clear that the size of the bonus itself is a critical aspect for any analysis of teacher recruitment, retention, and

107 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

108 Clotfelter, Charles, Elizabeth Glennie, Helen Ladd, and Jacob Vigdor. 2008. "Would Higher Salaries Keep Teachers in High-Poverty Schools? Evidence from a Policy Intervention in North Carolina." Journal of Public Economics 92: 1352-1370.

109 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

46

subsequent effectiveness in producing improved student learning results and yet there are few studies that have conclusively addressed this complete question.110

Summary

The history of teacher compensation in American public education is rich and varied and yet there is no clear method that has been established as the most empirically sound approach to producing the highest levels of student achievement.111

However, it is clear that the single-salary schedule has served as the most consistent defining approach within United States educational systems and yet there is compelling evidence to suggest that it has no positive effect on student learning and possibly even acts as a restraining feature on improving outcomes for students.112 With each of the most common alternative compensation reform strategies, the limitations of those methods have resulted in insufficient organizational momentum for any widespread deviation from the traditional, yet ineffective single-salary approach.113 However, there is minimal research that focuses exclusively on a substantially large compensation enhancement, possibly due to the isolated cases that would fit this type of criteria for

110 Prince, C. D. 2003. Higher Pay in Hard-To-Staff Schools. Lanham, MD: University Press of America, Inc.

111 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

112 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

113 Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

47

scholarly study.114 For that reason, this concept of substantial teacher compensation with some promising support from international studies and isolated cases within the

United States is suitable for further review and analysis from academia.

114 Hanushek, E. A., and S. G. Rivkin. 2006. "Teacher Quality." In Handbook of the Economics of Education, edited by E. A. Hanushek and F. Welch, 1051-1078. Amsterdam: North Holland.

48

CHAPTER 3 RESEARCH DESIGN

The purpose of this study was to evaluate to what extent a substantial increase in teacher compensation levels would produce a positive statistically significant impact on student learning as measured on state standardized tests utilizing the growth methodologies set forth by each state’s accountability structure.

The question addressed in this research was:

Research Question

 Was there a positive statistically-significant relationship in student learning growth at schools that utilized a front-loaded, substantial compensation model when contrasted with typical compensation schools that served a similar demographic population?

Research Design

This study utilized an ex post facto, quasi-experimental design to examine differences in the performance of students in schools where the entire faculty received a form of compensation that could be characterized as a front-loaded, substantial salary enhancement far beyond what is typically offered in most bonus compensation schemes found within the field of education as contrasted with schools serving a demographically similar population of students that did not utilize such a compensation scheme. Due to the very limited number of schools within the United States that offer compensation that matches the criteria set forth in this study to qualify as substantial, the methodology had to support this extraordinary exclusivity. Furthermore, this design was suitable because data analyzed were historical achievement results that were readily available to the public and were frequently utilized for evaluation and strategic decisions regarding performance at school, district, and state levels of educational leadership. Ex post facto research is employed as a research design after the independent variables of interest

49

within the treatment have already occurred and when the observation of the dependent variables is available. In this study, the set of dependent variables of student growth in literacy and mathematics were analyzed at a school-wide level of measurement due to the school-wide deployment of the target treatment since performance between teachers within the school was not germane to the research question. The treatment independent variable was represented as a dichotomous option with the school’s compensation approach deployed being either a substantial, front-loaded compensation scheme or a typical compensation approach.

The dependent variable calculations of student growth at the student level of measurement were not conducted by the researcher as they were compiled as summary data values provided from the department of education for each state’s public school accountability reporting system. For the purposes of this study, each of the two selected states featured school-wide student growth values in both literacy and mathematics as a significant summary statistic within each respective accountability reporting systems. This acted as a critical mechanism for the public, school, district, and state leaders to assess a school’s annual effectiveness in school-wide student growth as a potentially superior consideration to level of academic achievement which may be a conflated measurement most closely related to a student’s prior level of achievement and thusly may distort performance based on the student population being served as opposed to the value-added from effective instruction.1 The student growth metrics were designed to serve as a clearer association to the student learning added in a singular

1 Popham, James W. 2000. Modern Educational Measurement: Practical Guidelines for Educational Leaders. 3rd ed. Boston: Allyn and Bacon.

50

year’s time2 and thusly a preferred measure in precisely the type of analysis being conducted within this study. Due to the sophistication and widespread use of the student growth metrics as both a public policy instrument as well as an industry standard with widespread use in the field of education, these school aggregate values were observed as meaningful school summary characteristics. All statistical tests to evaluate the significance of the student learning growth in both literacy and mathematics at the school-wide level of measurement were conducted by the researcher.

Florida Learning Gain Value

This study utilized a One-Sample T-Test to determine the statistical significance of the observed school value when compared to the sample mean of the selected peer group of schools. For the purposes of this study, the school learning gain values for

English Language Arts and Mathematics were utilized. This school-wide value was produced by the Florida Department of Education as a measure within the Florida school accountability system. The school-wide learning gain calculation was an aggregate of individual student learning gains that were derived from a student’s prior year score being directly compared to the current year score.3 At the foundation of this individual student gain value were the two distinct state standardized tests; each of which produced a student score that was utilized for this calculation. So at the heart of that learning gain calculation was the validity and reliability of the state standardized

2 Ibid.

3 Florida Department of Education. June 2018. "2017-2018 Guide to Calculating School and District Grades." Technical Assistance Report, Bureau of Accountability Reporting; Divsion of Accountability, Research, and Measurement. Accessed December 13, 2018. http://www.fldoe.org/core/fileparse.php/18534/urlt/SchoolGradesCalcGuide18.pdf.

51

tests upon which any subsequent analysis relied. This study did not attempt to prove or disprove the assumption that these measures were in fact valid and reliable. Instead, the study acknowledged that the summary values used were based on psychometric and empirically-accepted conventions and standards that have been used consistently on a practical basis within the field of education. The Florida Department of Education recorded considerable technical documentation to demonstrate the reliability, content validity, internal structure validity, form validity, and test fairness of implemented state assessments.4 These statistical elements were tested and the results were made publicly available in Florida Department of Education published technical assistance manuals on this basis of inquiry.5

New York Median Student Growth Percentile Value

In spite of the differences between the educational policies in Florida and New

York, there were similarities in the way that each state prioritized the inclusion of a student growth measure although these calculations were performed very differently.

Since this study utilized a One-Sample T-Test to determine the statistical significance of the observed school value when compared to the sample mean of the selected peer group of schools, the New York school median student growth value for English

Language Arts and Mathematics was necessary for this analysis. This school-wide value was produced by the New York Department of Education as a measure within the state accountability system. The student growth calculations were utilized in school,

4 Florida Department of Education. n.d. "FSA Technical Report: Volume 4." Evidence of Reliability and Validity, Tallahassee, FL, 1-92. Accessed December 21, 2018. https://fsassessments.org/assets/documents/V4_FSA_Technical_Report_Year_2016- 2017_FINAL_508.pdf.

5 Ibid.

52

principal, and teacher evaluation systems as varying aggregates of individual student growth scores that were derived from a student’s prior multiple year test history and student demographics to include ELL status, disability status, and poverty status all tethered to the current year score.6 This approach first utilized a linear regression model to account for measurement variance in identified predictor variables in addition to the outcome variable in order to yield unbiased estimates of the model coefficients.7 These coefficients were then used to generate a predicted score upon which a student growth percentile was based.8 Lastly, these measures were also based upon state standardized tests that were thoroughly vetted for reliability and validity utilizing industry standard best practices for quality control of standardized assessments.9

Research Population

The two middle schools that served as treatment schools were both characterized as having served urban residential student populations with pupil demographics that would have otherwise allowed them to be characterized as ‘hard-to- staff’ schools10 and the entire faculty of each school received a level of compensation that qualified as substantial for the purposes of this treatment. One school was a public charter school in the State of New York and the other school was a traditional public

6 Education Analytics, Inc. November 2017. Growth Model for Educator Evaluation 2016/17 Technical Report. Prepared for the New York State Education Department, NY: New York State Department of Education. Accessed December 18, 2018. http://www.nysed.gov/common/nysed/files/programs/state- growth-measures-toolkits/2016-17-technical-report-for-growth-measures.pdf.

7 Ibid.

8 Ibid.

9 Ibid.

10 Prince, C. D. 2003. Higher pay in hard-to-staff schools. Lanham, MD: University Press of America, Inc.

53

school in the State of Florida. Both of these middle schools served high poverty, high minority populations and this substantial compensation approach was utilized as a strategy to significantly enhance the level of learning beyond what would be otherwise typical for schools that served similar populations of students. The primary reason these two schools were chosen for this study was that due to the extremely limited number of schools that satisfied the threshold to qualify as substantial compensation, these schools were among some of the only schools in the entire United States that met the criteria for a sufficient level of substantial compensation as defined in this study.

Additionally, the entire faculties received this substantial compensation and thusly the level of measurement for the treatment was able to be aggregated as school-wide data as opposed to having tracked individual teachers with individual classroom performance. All of the student performance data for these schools was provided by each state’s Department of Education. The non-treatment peer samples of schools were comprised of demographically similar schools utilizing thresholds derived from the intact convenience sample of each treatment school. This was done separately for each sample studied and utilized each respective state’s state-defined socio-economic measurements, racial composition, and achievement levels collected strictly within the same assessment year to secure the most valid comparison.

Florida Sample

In the Florida treatment middle school, the student performance for the 2017-

2018 school year represented the first year that the substantial compensation strategy was installed. The reason the school district selected the school to exclusively receive the substantial differentiated compensation strategy was due to the consistent history of the school’s underperformance within the state accountability system. Having applied

54

the most closely similar demographic measurement thresholds from the treatment school, a group of 8 peer schools matched the treatment school population to form a comparison sample when controlled for socio-economic status, racial composition, and student achievement levels. In the 2015-2016 school year, the treatment school was funded and operated without any significant enhanced resources or strategies and earned a grade of an “F” in the State of Florida letter grade school accountability system. This triggered a “turnaround” effort where specific strategies were deployed with an intention of raising overall school performance to an increased performance rating. Consequently, the school’s performance did improve to the grade of a “C” in

2016-2017 without any substantial compensation approach being deployed, although many significant additional resources and strategies were implemented at the school.

Table 3-1. School Grade History for Florida treatment school 1999-2018 within Florida State Accountability System beginning with most recent year School School Year Grade Year Grade 2018 C 2008 C 2017 C 2007 D 2016 F 2006 C 2015 D 2005 C 2014 F 2004 C 2013 D 2003 C 2012 D 2002 C 2011 D 2001 D 2010 C 2000 D 2009 C 1999 D

In an attempt to not only achieve but to also sustain success, the school district secured a grant for a pilot program that featured a $20,000 bonus provided to every teacher in the school without being concretely tethered to any student performance

55

output metrics. The average teacher salary in the District where the school was located was listed as $47,19811 whereas teachers at this school received their same annual pay based on their district salary scale and the additional bonus supplemented that amount by an additional $20,000. While this served as a market-based recruitment and retention incentive, the principal of the school also had the opportunity to transfer out any teachers at any time during the three year pilot as dictated in the District’s

Memorandum of Understanding negotiated with the collective bargaining unit.12 In this manner, the substantial bonus addressed recruitment, retention, and by manner of principal judgement, performance with students. The challenge as it pertained to analyzing this sample population was that in the 2016-2017, the school’s substantial improvement may have resulted in a statistical ceiling effect13 that could have had an impact in the interpretation of the subsequent 2017-2018 results. The possibility of a diminishing return was therefore an issue to consider when the results of the statistical significance testing were interpreted. Since there was only one year of student performance data with the experimental treatment deployed, this was an observed limitation of the Florida sample.

11 Florida Department of Education. n.d. Average Salaries for Teachers 2017-2018 Final Survey 3 Data. Tallahassee, FL. Accessed December 12, 2018. http://www.fldoe.org/accountability/data-sys/edu-info- accountability-services/pk-12-public-school-data-pubs-reports/staff.stml.

12 Florida Department of Education. 2017. Project Application: Schools of Hope - Whole School Transformation Model. Tallahassee, FL, August 15. Accessed December 15, 2018. http://www.fldoe.org/core/fileparse.php/5673/urlt/orange-memorial.pdf.

13 Popham, James W. 2000. Modern Educational Measurement: Practical Guidelines for Educational Leaders. 3rd ed. Boston: Allyn and Bacon.

56

New York Sample

The New York treatment middle school presented a more prolonged implementation of the experimental treatment. The school first opened its inaugural cohort of 5th grade students in the 2009-2010 school year with its first full middle school class having graduated in the 2012-2013 school year. The students at the treatment school predominantly resided in an urban area within New York City with the neighborhood being largely identified as majority Latino with two-thirds of households identified as having spoken Spanish in the home and approximately 47 percent of families identified as a single-parent female as the head of household.14 49 percent of families were reported as foreign-born and over 90 percent were eligible for a government subsidized lunch.15 The experimental approach of a substantial compensation strategy for the school’s faculty was the featured distinguishing characteristic of the school’s innovative approach to improving student outcomes.

Teacher salaries started at a level of $125,000 annually with weekly professional development salary incentive add-ons and schoolwide performance bonuses. In comparative context, the median salary for New York City district teachers in the same geographical area as the treatment school during the 2012-2013 school year was

$75,092.16 Having applied the demographic measurement thresholds from the intact convenience sample treatment school to generate a corresponding comparison, a group

14 Furgeson, Joshua, Moira McCullough, Clare Wolfendale, and Brian Gill. 2014. The Equity Project Charter School: Impacts on Student Achievement. Final Report submitted to The Bill & Melinda Gates Foundation, Cambridge, MA: Mathematica Policy Research.

15 Ibid.

16 New York State Education Department. n.d. Salary Percentiles for Classroom Teachers 2012-2013. New York City. Accessed December 12, 2018. http://www.p12.nysed.gov/irs/pmf/2013/2013_Stat-14.pdf.

57

of 34 peer schools were identified for the peer comparison sample in 2015-2016 and 18 peer schools were identified for the peer comparison sample in 2016-2017 when controlled for socio-economic status, racial composition, and student achievement levels.

Data Sources

The study utilized publicly available and readily accessible data from the Florida

Department of Education and the New York Department of Education through each respective agency’s website. The files included student performance data aggregated by school as well as descriptive demographic information about the school populations in separate files. Each file was separated by year and the following years were collected from each state: 2015-2016, 2016-2017, and 2017-2018. Schools were already matched utilizing a unique identifier for each school site that distinguished each school by its representative district and an individual school number. Both Florida and New

York used separate alpha-numeric conventions for this unique identifier and each school number was maintained in all electronic files for both the treatment schools as well as for every school within the peer sample for validation and verification. Aggregate teacher salary information was also readily available; organized by district and was available as a contemporaneous measure as well as a historical measure.

Data Analysis

Due to the small number of schools with the observed substantial compensation approach, a number of statistical tests were applied to gain the maximum level of insight from an otherwise limited and isolated sample of student performance. The initial formation of the peer samples relied on descriptive statistics and normality tests to ensure that not only were peer schools comparable, but that the group provided a

58

robust and meaningful comparison through which to interpret the treatment school results. The descriptive tests performed included the mean, standard deviation, and range characteristics with sample thresholds based on critical demographic proportions within each school. By controlling for common ratios of students who were identified as having learned English as a second language, students with disabilities, and students who were identified as economically disadvantaged, this ensured that school samples analyzed were more accurately assessed for the effect of the school’s impact as compared with conflating or confounding effects that may occur with resource potential in a more advantaged socio-economic sample of students.17

For hypothesis testing, the following series of tests were conducted each with a specific line of inquiry to address the research question. Initially, the Shapiro-Wilk test was used to assess the normality of distribution for the peer school groups that was utilized to measure the significance of the treatment schools. Due to the small sample sizes, the Shapiro-Wilk test was the ideal test for normality.18 Secondly, a series of one- sample t-tests were conducted for each school-wide growth summary value compared for significance to the sample of schools selected as a peer group having served comparable student demographics. While the sample of schools involved were small,

17 Popham, James W. 2000. Modern Educational Measurement: Practical Guidelines for Educational Leaders. 3rd ed. Boston: Allyn and Bacon.

18 Razali, Nornadiah Mohd, and Yap Bee Wah. 2011. "Power Comparisons of Shapiro-Wilk, Kolmogorov- Smirnov, Lilliefors and Anderson-Darling Tests." Journal of Statistical Modeling and Analytics 2 (1): 21-33. Accessed December 12, 2018. https://www.researchgate.net/profile/Bee_Yap/publication/267205556_Power_Comparisons_of_Shapiro- Wilk_Kolmogorov-Smirnov_Lilliefors_and_Anderson- Darling_Tests/links/5477245b0cf29afed61446e1/Power-Comparisons-of-Shapiro-Wilk-Kolmogorov- Smirnov-Lilliefors-an.

59

the one-sample t-test was still able to provide sufficient power for such an analysis.19 In the case of the Florida treatment school, the substantial compensation scheme was only deployed for a single year, so this analysis was only performed for that year.

However, the New York treatment school had multiple years of data well into implementation of the substantial compensation scheme. This provided an additional opportunity for significance to be assessed over multiple years. In all hypothesis testing, the p-values were set at the .05 significance level. In the event that any significance was detected within the one-sample t-tests, the statistical power of that observation was also measured for effect size utilizing Cohen’s d calculation.

Summary

Chapter 3 delineated the design and statistical analyses that were executed in the study. The design of the study was intended to assess whether there was an apparent relationship between a substantial compensation approach being deployed in isolated experimental school cases and the consequent impact in observed student learning. The treatment school student populations were briefly discussed and contextual conditions distinguished. Sources for data utilized in the study were provided and the analytical tests employed were identified and expounded upon for any associated rationale. Results of these analyses were discussed in Chapter 4.

19 de Winter, J.C.F. 2013. "Using the Student's t-test With Extremely Small Sample Sizes." Practical Assessment, Research & Evaluation 18 (10): 1-12. Accessed November 19, 2018. https://pareonline.net/pdf/v18n10.pdf.

60

CHAPTER 4 PRESENTATION OF RESULTS

Introduction

The intent of this study was to assess whether a substantial teacher compensation enhancement produced a statistically significant impact on student learning as measured on state standardized tests utilizing the growth methodologies set forth by each state’s accountability structure. When such a relationship was detected, the degree of significance and magnitude of that effect was also investigated.

Descriptive statistics were employed to gain insight into the treatment schools as well as respective peer samples. Tests for normality were applied. A series of one-sample t- tests were performed for hypothesis testing and the effect sizes were analyzed for statistical power.

Descriptive Statistics Results

Florida Sample

The Florida treatment middle school that employed the substantial compensation bonus served a predominantly at-risk student population in the 2017-2018 school year.

With an overall school literacy rate of 26 percent as measured by the English Language

Arts Florida Standards Assessment, this represented a school where the majority of students were performing below grade level. It was critical that the comparison sample of peer schools had a comparable level of student achievement as well as other demographic characteristics that could have impacted the legitimacy of the comparison.

Furthermore, the nature of the learning gains calculation was observed to be clearly impacted by additional student demographic considerations such as the prior achievement level of the student, a student having acquired English as a second

61

language, and a student’s documented disabled status.1 Lastly, the role of socio- economic factors in impacting student performance have been well-documented challenges in the field of education. Thusly, it was vital to ensure that any selected peer school for the purposes of evaluating this treatment also accounted for a comparable demographic proportion of students with documented economically disadvantaged status.

Table 4-1. Descriptive Statistics for Student Demographics of Florida treatment school 2017-2018 School Year Statistic Florida Treatment School Total School Enrollment 793 Percent of English Language Arts Achievement 26% Percent of Mathematics Achievement 28% Percent English Language Learners 11.0% Percent Students with Disabilities 11.8% Percent Economically Disadvantaged Students 79.1% Percent Asian 0.0% Percent Black 90.1% Percent Hispanic 8.9% Percent Two or More Races **.* Percent White **.* **.* is denoted when the Florida Department of Education suppresses data when the total number of students in a group is fewer than 10 in order to protect the privacy of individual students

The demographically matched peer sample of schools utilized for the hypothesis testing consisted of eight non-charter, public middle schools that were most closely comprised of the same level of literacy achievement and demographic characteristics for comparative analysis. While racial considerations were not expressly controlled, the

1 Florida Department of Education. June 2018. "2017-2018 Guide to Calculating School and District Grades." Technical Assistance Report, Bureau of Accountability Reporting; Division of Accountability, Research, and Measurement. Accessed December 13, 2018. http://www.fldoe.org/core/fileparse.php/18534/urlt/SchoolGradesCalcGuide18.pdf

62

school populations still demonstrated considerable similarity with the primary observable feature of being largely a non-white student enrollment.

Table 4-2. Descriptive Statistics for Student Demographics Florida selected peer sample for comparative analysis 2017-2018 School Year Statistic for N Schools Minimum Maximum Mean SD Total School Enrollment 307 997 549.125 211.7554 Percent of English Language Arts Achievement 22% 29% 25.625% 2.5951% Percent of Mathematics Achievement 18% 42% 29.250% 6.9417% Percent English Language Learners 0.0% 22.2% 11.038% 6.8817% Percent Students with Disabilities 8.9% 25.5% 16.350% 5.4472% Percent Economically Disadvantaged Students 75.3% 93.2% 85.725% 5.5301% Percent Asian 1.3% 1.3% 1.315% 0.0000% Percent Black 46.3% 93.5% 70.899% 16.2958% Percent Hispanic 9.3% 50.7% 26.897% 16.1060% Percent Two or More Races 1.1% 4.2% 2.843% 1.1489% Percent White 2.2% 7.0% 3.912% 1.9591% a. N=8 Peer Group Schools

New York Samples

The New York treatment school exemplified the substantial salary enhancement as the primary approach to increased quality of the overall enterprise and this was purposefully deployed within an otherwise hard-to-staff environment. Furthermore, this experimental model of compensation was deployed since the school’s inception in

2009. For that reason, there was a significant longitudinal opportunity to study multiple years of school performance data. However, in order to have a comparable quality of learning for the purposes of this study, only student performance data since 2015 were utilized since this study was a multi-state study and wanted to reflect the most current

63

and ambitious instructional standards being assessed as opposed to standards prior to the implementation of the Common Core Standards.

2015-2016 School Year

By the 2015-2016 school year the New York treatment middle school had deployed the experimental payment structure and served an urban, ‘at-risk’ student population as the key demographic to test the effectiveness of their approach for a number of years. With 22.3 percent of the student population having qualified as English

Language Learners (ELL) and 21.3 percent of the student enrollment identified as students with disabilities, these characteristics along with the 79.1 percent school economic need index demonstrated a substantial at-risk student demographic population that was characteristic of the neighborhood in which the school was located as a high poverty area.

Table 4-3. Descriptive Statistics for Student Demographics of New York treatment school 2015-2016 School Year Statistic New York Treatment School Total School Enrollment 479 Percent English Language Learners 22.3% Percent Students with Disabilities 21.3% Economic Need Index 79.1% Percent in Temp Housing 10.2% Percent HRA Eligible 64.1% Percent Asian 0.2% Percent Black 5.6% Percent Hispanic 92.5% Percent White 1.0%

The demographically matched peer sample of schools utilized for the hypothesis testing consisted of 34 public middle schools that were most closely comprised of the same level of demographic characteristics for comparative analysis. With proportions of students having acquired English as a second language, students identified with a

64

learning disability, and three different economic indicators to substantiate a level of poverty, these summary values demonstrated the significant level of educational challenge within the sample of schools. While racial considerations were not expressly controlled, the school populations still demonstrated similarity with the primary observable feature of being largely a non-white student enrollment as evidenced by the sample mean of 4.144% (SD = 5.1371%).

Table 4-4. Descriptive Statistics for Student Demographics New York selected peer sample for comparative analysis 2015-2016 School Year Statistic for N Schools Minimum Maximum Mean SD Total School Enrollment 166 1291 573.65 328.612 Percent English Language Learners 17.6% 27.3% 22.035% 3.1256% Percent Students with Disabilities 16.6% 26.3% 21.168% 2.9891% Economic Need Index 74.8% 83.9% 80.376% 2.7735% Percent in Temp Housing 3.1% 26.3% 15.021% 5.3887% Percent HRA Eligible 58.9% 77.1% 65.065% 3.5544% Percent Asian 0.0% 37.4% 6.612% 9.5112% Percent Black 0.4% 63.8% 17.368% 15.5453% Percent Hispanic 29.8% 97.7% 70.706% 18.4292% Percent White 0.5% 20.2% 4.144% 5.1373% a. N=34 Peer Group Schools

2016-2017 School Year

In the 2016-2017 school year the New York treatment middle school continued to deploy the experimental payment structure and continued to serve an urban, ‘at-risk’ student population as the key demographic to test the effectiveness of the compensation approach. With 24.1 percent of the student population qualifying as

English Language Learners (ELL) and 22.8 percent of the student enrollment identified as students with disabilities, these characteristics along with the 77.9 percent school economic need index continued to qualify this enrollment as a substantial ‘at-risk’

65

student demographic population that was characteristic of the neighborhood in which the school was located as a high poverty area.

Table 4-5. Descriptive Statistics for Student Demographics of New York treatment school 2016-2017 School Year Statistic New York Treatment School Total School Enrollment 598 Percent English Language Learners 24.1% Percent Students with Disabilities 22.8% Economic Need Index 77.9% Percent in Temp Housing 13.0% Percent HRA Eligible 60.7% Percent Asian 0.4% Percent Black 5.3% Percent Hispanic 93.4% Percent White 0.4%

The demographically matched peer sample of schools utilized for the hypothesis testing consisted of eighteen public middle schools that were narrowly comprised of the equivalent level of demographic characteristics for comparative analysis. With continued use of proportions of students acquiring English as a second language, students identified with a learning disability, and three different economic indicators to substantiate a level of poverty, these summary values demonstrated the significant level of educational challenge within the sample of schools. While racial considerations were not explicitly controlled, the school populations still demonstrated similarity with the primary observed feature of being largely a non-white student enrollment as evidenced by the sample mean of 4.839% (SD = 5.6227%). The overall enrollment of the treatment school of 598 was slightly larger but still within one standard deviation of the peer sample mean of 517.11 (SD = 268.232).

66

Table 4-6. Descriptive Statistics for Student Demographics New York selected peer sample for comparative analysis 2016-2017 School Year Statistic for N Schools Minimum Maximum Mean SD Total School Enrollment 109 1217 517.11 268.232 Percent English Language Learners 20.1% 29.9% 24.367% 3.0199% Percent Students with Disabilities 19.9% 28.4% 24.239% 2.7339% Economic Need Index 74.9% 83.7% 80.200% 2.4959% Percent in Temp Housing 11.0% 19.7% 15.794% 2.6939% Percent HRA Eligible 54.6% 70.6% 63.644% 4.1035% Percent Asian 0.0% 33.0% 6.617% 9.0602% Percent Black 1.5% 65.1% 17.500% 15.3407% Percent Hispanic 23.9% 93.6% 69.939% 20.7118% Percent White 0.4% 18.4% 4.839% 5.6227% a. N=18 Peer Group Schools

Normality Test Results

Florida Sample

In order to assign the correct statistical hypothesis test, the first step was to assess the shape and normality of both dependent variables for the peer sample.

Table 4-7. Descriptive Statistics for Distribution of English Language Arts Learning Gains Florida selected peer sample for comparative analysis 2017-2018 School Year English Language Arts Learning Gains for N Standard Schools Statistic Error Mean 38.38 1.149 95% Confidence Interval for Mean (Lower Bound) 35.66 95% Confidence Interval for Mean (Upper Bound) 41.09 5% Trimmed Mean 38.53 Median 39.00 Variance 10.554 Standard Deviation 3.249 Minimum 32 Maximum 42 Range 10 Interquartile Range 5 Skewness -0.981 0.752 Kurtosis 1.347 1.481 a. N=8 Peer Group Schools

67

Figure 4-1. Normal Q-Q Plot of 17-18 English Language Arts Learning Gains for Florida Peer Group of Schools

Figure 4-2. Detrended Normal Q-Q Plot of 17-18 English Language Arts Learning Gains for Florida Peer Group of Schools

68

Table 4-8. Descriptive Statistics for Distribution of Mathematics Learning Gains Florida selected peer sample for comparative analysis 2017-2018 School Year Standard Mathematics Learning Gains for N Schools Statistic Error Mean 40.75 3.452 95% Confidence Interval for Mean (Lower Bound) 32.59 95% Confidence Interval for Mean (Upper Bound) 48.91 5% Trimmed Mean 40.28 Median 40.00 Variance 95.357 Standard Deviation 9.765 Minimum 29 Maximum 61 Range 32 Interquartile Range 11 Skewness 1.250 0.752 Kurtosis 2.350 1.481 a. N=8 Peer Group Schools

Figure 4-3. Normal Q-Q Plot of 17-18 Mathematics Gains for Florida Peer Group of Schools

69

Figure 4-4. Detrended Normal Q-Q Plot of 17-18 Mathematics Gains for Florida Peer Group of Schools

Table 4-9. Shapiro-Wilk Normality Test of English Language Arts and Mathematics Learning Gains Florida selected peer sample for comparative analysis 2017-2018 School Year Degrees Statistic of Significance Peer Sample of Learning Gains for N Schools W Freedom p English Language Arts Learning Gains 0.893 8 0.251 Mathematics Learning Gains 0.912 8 0.367 a. N=8 Peer Group Schools (*p<.05)

With both English Language Arts and Mathematics Learning Gain dependent variables having demonstrated normality across the samples, this peer group was able to be utilized in a one-sample t-test for each type of learning gain for hypothesis testing of the Florida treatment school for 2017-2018.

70

New York Samples

In order to assign the correct statistical hypothesis test for each year, the first step was to assess the shape and normality of each dependent variable for the peer sample for each year of the study.

2015-2016 School Year

Table 4-10. Descriptive Statistics for Distribution of English Language Arts Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2015-2016 School Year English Language Arts Student Median Growth Standard Percentile Value for N Schools Statistic Error Mean 49.985 1.2796 95% Confidence Interval for Mean (Lower Bound) 47.382 95% Confidence Interval for Mean (Upper Bound) 52.589 5% Trimmed Mean 49.843 Median 50.500 Variance 55.6740 Standard Deviation 7.4615 Minimum 34.0 Maximum 71.0 Range 37.0 Interquartile Range 8.0 Skewness 0.263 0.403 Kurtosis 1.527 0.788 a. N=34 Peer Group Schools

71

Figure 4-5. Normal Q-Q Plot of 15-16 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools

Figure 4-6. Detrended Normal Q-Q Plot of 15-16 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools

72

Table 4-11. Descriptive Statistics for Distribution of Mathematics Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2015-2016 School Year Mathematics Student Median Growth Percentile Standard Value for N Schools Statistic Error Mean 49.926 1.6237 95% Confidence Interval for Mean (Lower Bound) 46.623 95% Confidence Interval for Mean (Upper Bound) 53.230 5% Trimmed Mean 49.644 Median 48.500 Variance 89.638 Standard Deviation 9.4678 Minimum 30.0 Maximum 77.0 Range 47.0 Interquartile Range 11.3 Skewness 0.607 0.403 Kurtosis 0.998 0.788 a. N=34 Peer Group Schools

73

Figure 4-7. Normal Q-Q Plot of 15-16 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools

Figure 4-8. Detrended Normal Q-Q Plot of 15-16 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools

74

Table 4-12. Shapiro-Wilk Normality Test of English Language Arts and Mathematics Median Student Growth Percentile Values for New York selected peer sample for comparative analysis 2015-2016 School Year Degrees Peer Sample of Median Student Growth of Percentile Values for N Schools Statistic Freedom Significance English Language Arts Student Growth 0.960 34 0.247 Mathematics Student Growth 0.974 34 0.569 a. N=34 Peer Group Schools (*p<.05)

With both English Language Arts and Mathematics Median Student Growth

Percentile dependent variables demonstrating normality across the samples, the selected peer group was used for hypothesis testing utilizing a one-sample t-test for each type of growth in student learning for the New York treatment school for 2015-

2016.

2016-2017 School Year

Table 4-13. Descriptive Statistics for Distribution of English Language Arts Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2016-2017 School Year English Language Arts Student Median Growth Standard Percentile Value for N Schools Statistic Error Mean 50.139 1.6972 95% Confidence Interval for Mean (Lower Bound) 46.558 95% Confidence Interval for Mean (Upper Bound) 53.720 5% Trimmed Mean 50.182 Median 50.500 Variance 51.8470 Standard Deviation 7.2005 Minimum 34.0 Maximum 65.5 Range 31.5 Interquartile Range 7.1 Skewness -0.337 0.536 Kurtosis 1.197 1.038 a. N=18 Peer Group Schools

75

Figure 4-9. Normal Q-Q Plot of 16-17 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools

76

Figure 4-10. Detrended Normal Q-Q Plot of 16-17 English Language Arts Student Growth Median Percentile Values for New York Peer Group of Schools

77

Table 4-14. Descriptive Statistics for Distribution of Mathematics Median Student Growth Percentile Values New York selected peer sample for comparative analysis 2016-2017 School Year Mathematics Student Median Growth Percentile Standard Value for N Schools Statistic Error Mean 49.667 2.7335 95% Confidence Interval for Mean (Lower Bound) 43.899 95% Confidence Interval for Mean (Upper Bound) 55.434 5% Trimmed Mean 50.241 Median 51.500 Variance 134.500 Standard Deviation 11.5974 Minimum 22.0 Maximum 67.0 Range 45.0 Interquartile Range 17.5 Skewness -0.862 0.536 Kurtosis 0.503 1.038 a. N=18 Peer Group Schools

78

Figure 4-11. Normal Q-Q Plot of 16-17 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools

79

Figure 4-12. Detrended Normal Q-Q Plot of 16-17 Mathematics Student Growth Median Percentile Values for New York Peer Group of Schools

Table 4-15. Shapiro-Wilk Normality Test of English Language Arts and Mathematics Median Student Growth Percentile Values for New York selected peer sample for comparative analysis 2016-2017 School Year Degrees Peer Sample of Median Student Growth Statistic of Significance Percentile Values for N Schools W Freedom p English Language Arts Student Growth 0.964 18 0.679 Mathematics Student Growth 0.941 18 0.299 a. N=18 Peer Group Schools (*p<.05)

With both English Language Arts and Mathematics Median Student Growth

Percentile dependent variables demonstrating normality across the samples, the peer group sample was utilized in a one-sample t-test for each type of learning growth for hypothesis testing of the New York treatment school for 2016-2017.

80

Statistical Hypothesis Test Results

In order to test for a statistically significant difference between the treatment schools and the respective peer samples of schools in terms of student growth scores, a One-sample T-Test was implemented for each dependent variable student growth value. The One-sample T-test was deployed as a parametric statistical hypothesis test for whether a specified or hypothesized value differed from the sample distribution’s mean at a statistically significant level. In this case, the student growth value was a known value that was compared to the peer sample’s actual population of scores with an assumption of the null hypothesis that no statistically significant effect exists at a .05 level of significance. Furthermore, the Cohen’s d was calculated as an effect size measure for an additional insight into the statistical power for the sample observed growth values utilized in each hypothesis test.

Florida Sample

Table 4-16. One-Sample T-Test results for Florida treatment school compared with peer sample 2017-2018 English Language Mathematics Arts Learning Learning Statistic Gains Values Gains Values N Schools 8 8 Sample Mean 38.38 40.75 Standard Deviation 3.249 9.765 Std. Error Mean 1.149 3.452 t value -0.544 -0.072 Degrees of Freedom 7 7 Significance p 0.603 0.944 Mean Difference -0.625 -0.250 95% Confidence Interval of the difference Lower -3.34 -8.41 Upper 2.09 7.91 Cohen's d -0.19237 -0.0256 a. N Schools in Peer Group Sample

81

In the Florida sample of the treatment school and the associated peer group, no statistically significant relationship was found to exist with either the English Language

Arts or Mathematics Learning Gain values.

New York Sample

Table 4-17. One-Sample T-Test results for New York treatment school compared with peer sample 2015-2016 English Language Arts Mathematics Median Student Median Student Statistic Growth Values Growth Values N Schools 34 34 Sample Mean 49.985 49.926 Standard Deviation 7.4615 9.4678 Std. Error Mean 1.2796 1.6237 t value -3.528 -16.058 Degrees of Freedom 33 33 Significance p 0.001 0.000 Mean Difference -4.5147 -26.0735 95% Confidence Interval of the difference Lower -7.118 -29.377 Upper -1.911 -22.77 Cohen's d -0.6051 -2.75391 a. N Schools in Peer Group Sample

In the New York 2015-2016 sample of the treatment school and the associated peer group, a statistically significant relationship at the .05 level was found to exist with both the English Language Arts (p=.001) and Mathematics (p=.000) Median Student

Growth Values. Therefore, we rejected the null hypothesis that there is no difference in student growth scores for the 2015-2016 New York treatment school when it was compared with its peer group of schools. Furthermore, Cohen’s effect size value (d=-

.6051) suggested a moderate to high practical significance in English Language Arts and a high practical significance in Mathematics (d=-2.7539).

82

Table 4-18. One-Sample T-Test results for New York treatment school compared with peer sample 2016-2017 English Language Arts Mathematics Median Student Median Student Statistic Growth Values Growth Values N Schools 18 18 Sample Mean 50.139 49.667 Standard Deviation 7.2005 11.5974 Std. Error Mean 1.6972 2.7335 t value -6.989 -7.438 Degrees of Freedom 17 17 Significance p 0.000 0.000 Mean Difference -11.8611 -20.3333 95% Confidence Interval of the difference Lower -15.442 -26.101 Upper -8.28 -14.566 Cohen's d -1.6473 -1.7533 a. N Schools in Peer Group Sample

In the New York 2016-2017 sample of the treatment school and the associated peer group, a statistically significant relationship at the .05 level was found to exist with both the English Language Arts (p=.000) and Mathematics (p=.000) Median Student

Growth Values. Therefore, we rejected the null hypothesis that there is no difference in student growth scores for the 2016-2017 New York treatment school when compared with its peer group of schools. Furthermore, Cohen’s effect size value suggested a high practical significance with both English Language Arts (d=-1.6473) and Mathematics

(d=-1.7533).

Summary

The descriptive analyses revealed comparability and statistical normality between all samples of Florida and New York schools selected within the study.

Hypothesis testing revealed that the Florida treatment school failed to reject the null

83

hypothesis in both English Language Arts and Mathematics Learning Gains values.

However, the alternate hypothesis was accepted for all cases of the New York treatment school over both the 2015-2016 and 2016-2017 school years. In these four instances of English Language Arts and Mathematics student growth values, the effect size ranged from moderate to high levels of practical significance.

84

CHAPTER 5 CONCLUSIONS

This study examined whether a substantial teacher compensation enhancement far beyond what is typically offered in most differentiated pay systems was able to produce a statistically significant impact on student learning. The measures utilized were each state’s own growth methodologies set forth by each state’s accountability structure for analyzing school and student performance. Data employed covered a range of three years from the 2015-2016 school year up through the 2017-2018 school year and included two different states, New York and Florida, but focused on treatment schools at the middle school level. The Florida treatment school studied was in its first year of implementation of the experimental pay model during the 2017-2018 school year, whereas the New York treatment school had been in operation with this distinctive compensation approach since its opening in 2009.1

Summary of Findings:

Results from this study were varied based on the two different treatment schools.

1. The two analyzed dependent variables of English Language Arts Learning Gains and Mathematics Learning Gains of the Florida treatment school failed to reject the null hypothesis and demonstrated no statistical significance when compared with the peer group of schools serving a similar population.

From the review of literature conducted in this study, this particular finding tended to align with Richard Ingersoll who concluded that while teacher pay is clearly an issue, there were many additional factors that also influence performance perhaps more

1 Furgeson, Joshua, Moira McCullough, Clare Wolfendale, and Brian Gill. 2014. The Equity Project Charter School: Impacts on Student Achievement. Final Report submitted to The Bill & Melinda Gates Foundation, Cambridge, MA: Mathematica Policy Research.

85

significantly than isolating level of compensation on its own.2 Additionally, the short term nature of implementation at this school may have been a critical consideration inasmuch that this strategy was deployed without the school leadership having an opportunity to observe the actual results of the most recently recruited staff. In that sense, the personnel flexibility afforded in conjunction with the substantial compensation bonus pay with the opportunity to remove current teachers and recruit additional teachers, ostensibly more effective teachers, may not have had sufficient time of implementation to have the true value of such a strategy be observed. Furthermore, the rapid turnaround of the Florida treatment school in the 2016-2017 school year may have created a statistical ceiling effect3 that further limited the ability for the 2017-2018 results to accurately reflect the full value provided from the substantial compensation approach.

2. In the New York treatment school, the dependent variables of English Language Arts and Mathematics Median Student Growth Values were found to be statistically significant across both years and both subject areas with a positive impact and moderate to large effect size observed when compared with the peer group of schools serving a similar population.

From the review of literature conducted in this study, this multi-year finding tended to support international judgements that higher levels of compensation do appear to align with higher performance outcomes for students,4 as well as Eric

2 Ingersoll, Richard. 2001. Teacher turnover, teacher shortages, and the organization of schools. Seattle: Center of the Study of Teaching and Policy, University of Washington.

3 Austin, Peter C, and Lawrence J Brunner. 2003. "Type I Error Inflation in the Presence of a Ceiling Effect." The American Statistician 57 (2): 97-104. Accessed December 2, 2018. https://www.tandfonline.com/doi/abs/10.1198/0003130031450.

4 Dolton, Peter, and Oscar Marcenaro-Gutierrez. 2011. "If You Pay Peanuts, Do You Get Monkeys? A Cross-country Analysis of Teacher Pay and Pupil Performance." Economic Policy 26 (65): 5-55. Accessed October 21, 2018. http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0327.2010.00257.x/full.

86

Hanushek’s economic premise that teacher level of pay would most certainly have an impact on the overall level of teacher quality supply to the extent supported by widely accepted economic theories.5 The deployment of this innovative model occurring for nearly a decade may have played a significant role in the maturity of this approach and the consequent statistically significant results observed in later years. However, the findings from this study were also consistent with the statistically significant positive results observed within this treatment school in earlier years by an evaluative report conducted by Furgeson, McCullough, Wolfendale, and Gill.6

Recommendations for Future Policy

The findings of this study have numerous potential policy implications. With minimal benefit for student learning currently evident from the single salary approach,7 there is little reason for this traditional approach to continue to serve as the predominant payment structure if improved student learning is held as the highest priority. However, while the single salary schedule serves as a restraining force in the implementation of meaningful differentiated pay structures,8 there is not a clearly evident optimal structure for compensation evident at this time. However, with an albeit small sample, a model such as employed in the New York treatment school suggests that larger class sizes

5 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

6 Furgeson, Joshua, Moira McCullough, Clare Wolfendale, and Brian Gill. 2014. The Equity Project Charter School: Impacts on Student Achievement. Final Report submitted to The Bill & Melinda Gates Foundation, Cambridge, MA: Mathematica Policy Research.

7 Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

8 Ibid.

87

and additional responsibilities for the teachers are valuable concessions when the additional financial benefit thusly afforded has such a profound, positive statistical impact as what was observed in this study. With evidence that a teacher salary roughly one-and-a-half times the observed median may have been a substantial factor in the observed statistically significant benefits in student learning, policies that introduce a similarly significant ratio of competitive salary level is clearly an opportunity for additional implementation and empirical examination. While the samples involved in this study were particularly limited, this only further underscores the opportunity for additional experimentation with this type of innovation. This is particularly advisable since the predominant status quo of the single salary schedule has absolutely no documented positive benefit for students.9 Even if this innovative compensation approach were applied for the most critical shortage positions or hard-to-staff schools, there is a clear opportunity to improve upon what is otherwise an insufficient compensation system as evidenced by the very fact that schools are termed as hard-to- staff or that certain teaching positions experience a shortage. Additionally, the findings of this study appear to align with the concept of teacher career ladder approaches with the caveat that significant compensation may be an underdeveloped key attribute.

Perhaps the most critical implication of this study is simply that additional context is necessary in order to properly frame any conversation on teacher pay to ensure that a level of sufficiency is included as a parameter to evaluate the effectiveness of the compensation. The utilization of a ratio for significance is not an approach that is evident in the research literature regarding teacher compensation and very little policy

9 Ibid.

88

articulates prescriptive language to compel a salary level with market-based guidelines.

This significantly impacts the ability to distinguish in policy-level conversation between bonuses that are very limited and potentially not impactful from substantial salary enhancements that may have a pronounced, dramatic effect on student learning. With the additional framing of the teacher compensation policy debate using terms that clearly articulate a relative market position, the opportunity to more effectively categorize the alternative compensation approaches may result in increased levels of this type of specific and potentially impactful implementation. With the proliferation of public charter schools which often have greater flexibility from collectively bargained labor agreements,10 there may be ample room for these types of experimental pay structures to capitalize on the most divergent salary distributions for contrasted study.

Recommendations for Future Research

The findings of this study have several implications for future research. One of the most critical research areas relating to teacher compensation reform should attempt to stratify the substantial nature of the bonuses or salary levels. With most research on bonus pay systems being consolidated as a single area of study, the level of the bonus itself is not sufficiently distinguished in the current professional literature on the

10 Abdulkadiroglu, Atila, Joshua D Angrist, Susan M Dynarski, Thomas J Kane, and Parag A Pathak. 2011. "Accountability and Flexibility in Public Schools: Evidence from Boston's Charters and Pilots." The Quarterly Journal of Economics 126 (2): 699-748. Accessed December 22, 2018. https://doi.org/10.1093/qje/qjr017.

89

subject,11 although there is increasing evidence that policy-makers are considering this critical distinction at present in some systems.12

In this study, the level of compensation approached a comparison of one-and-a- half times the typical salary in a geographical area and such ratio thresholds are not currently found as organizing features within educational compensation literature.

Additionally, since there is a limited sample of peer-reviewed, empirical study on the positive effects of the few isolated substantial compensation models, additional study is warranted to validate or challenge the limited observed statistical significance at this time. While this is an area of tremendous interest in the context of global competitiveness, there is very little cross-pollination of international compensation models firstly being organized within a replicable typology and secondly being thusly deployed in the United States. It is incumbent on researchers to better frame and organize international compensation approaches while attempting to contextualize this approach within those individual economies for an appropriate comparison to the

American economy. Lastly, for every innovative compensation approach deployed at a potentially replicable scale, research is recommended to be conducted, results analyzed, and prominently reported in a expansive manner to increase the level of

11 Odden, A,, and M. Wallace. 2004. "Experimenting with teacher compensation." The School Administrator (61): 24-28. Accessed October 29, 2018. http://www.wilsonweb.com.

12 Springer, Matthew G, Dale Ballou, Laura Hamilton, Vi-Nhuan Le, J. R. Lockwood, Daniel F. McCaffrey, Matthew Pepper, and Brian M. Stetcher. 2012. Final Report: Experimental Evidence from the Project on Incentives in Teaching. Nashville, TN: National Center on Performance Incentives at Vanderbilt University.

90

scholarly focus in what is becoming a major consideration in future economic growth in the global context.13

13 Dolton, Peter, and Oscar Marcenaro-Gutierrez. 2011. "If You Pay Peanuts, Do You Get Monkeys? A Cross-country Analysis of Teacher Pay and Pupil Performance." Economic Policy 26 (65): 5-55. Accessed October 21, 2018. http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0327.2010.00257.x/full.

91

LIST OF REFERENCES

Abdulkadiroglu, Atila, Joshua D Angrist, Susan M Dynarski, Thomas J Kane, and Parag A Pathak. 2011. "Accountability and Flexibility in Public Schools: Evidence from Boston's Charters and Pilots." The Quarterly Journal of Economics 126 (2): 699- 748. Accessed December 22, 2018. https://doi.org/10.1093/qje/qjr017.

Adkins, Gary A. 1983. Pros and Cons and Current Status of Merit Pay in the Public Schools. Eric Resources Information Center No. ED238-162, Radford, VA: Virginia Association of Teacher Educators.

Ansell, Susan E, and Melissa McCabe. 2003. Off Target. Accessed October 26, 2018. http://counts.cdweck.org/sreports/qc03/templates/article.cfm7slugM.

Austin, Peter C, and Lawrence J Brunner. 2003. "Type I Error Inflation in the Presence of a Ceiling Effect." The American Statistician 57 (2): 97-104. Accessed December 2, 2018. https://www.tandfonline.com/doi/abs/10.1198/0003130031450.

Bacolod, M. P. 2007. "Do Alternative Opportunities Matter? The Role of Female Labor Markets in the Decline of Teacher Quality." Review of Economics and Statistics 89 (4): 737-751.

Baptiste, I. 2001. "Educating Lone Wolves: Pedagogical Implications of Human Capital Theory." Adult Education Quarterly 51 (3): 184-202.

Berry, B, L Turchi, and D Johnson. 2003. The Impact of High-Stakes Accountability on Teachers' Professional Development: Evidence from the South. Chapel Hill, NC: Southeast Center for Teaching Quality.

Björk, L. G. 2004. Interim report: Differentiated Compensation Research Project. Frankfort, KY: Kentucky Department of Education.

Certo, J., and J. E. Fox. 2002. "Retaining quality teachers." High School Journal 86 (1): 57-76.

Clotfelter, Charles, Elizabeth Glennie, Helen Ladd, and Jacob Vigdor. 2008. "Would Higher Salaries Keep Teachers in High-Poverty Schools? Evidence from a Policy Intervention in North Carolina." Journal of Public Economics 92: 1352-1370.

Cohen, M., and A. Ginsburg. 2001. School Improvement Report: Executive Order on Actions for Turning Around Low-Performing Schools. Washington, DC: Department of Education.

Cresap, McCormick, and Paget, Inc. 1984. Teacher Incentives: A Tool for Effective Management. Washington: National Association of Secondary School Principals.

92

Davis, Hazel. 1943. "Teachers' Salaries." Review of Educational Research 13 (3): 276- 284.

Day, D V. 2000. "Leadership Development: A Review in Context." Leadership Quarterly 11 (4): 581-613. de Winter, J.C.F. 2013. "Using the Student's t-test with Extremely Small Sample Sizes." Practical Assessment, Research & Evaluation 18 (10): 1-12. Accessed November 19, 2018. https://pareonline.net/pdf/v18n10.pdf.

Dolton, Peter, and Oscar Marcenaro-Gutierrez. 2011. "If You Pay Peanuts, Do You Get Monkeys? A Cross-country Analysis of Teacher Pay and Pupil Performance." Economic Policy 26 (65): 5-55. Accessed October 21, 2018. http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0327.2010.00257.x/full.

Education Analytics, Inc. November 2017. Growth Model for Educator Evaluation 2016/17 Technical Report. Prepared for the New York State Education Department, NY: New York State Department of Education. Accessed December 18, 2018. http://www.nysed.gov/common/nysed/files/programs/state-growth- measures-toolkits/2016-17-technical-report-for-growth-measures.pdf.

English, Fenwick. 1992. "History and Critical Issues of Educational Compensation System." In Teacher Compensation and Motivation, edited by Larry Frase, 3-26. Lancaster, PA: Technomic Publishing.

Esch, C. E., C. M. Change-Ross, R. Guha, D. C. Humphrey, P. M. Shields, and J. D. Tiffany-Morales. 2005. The Status of the Teaching Profession 2005. Santa Cruz, CA: Center for the Future of Teaching and Learning.

Firestone, W. A. 1994. "Redesigning Teacher Salary Systems for Educational Reform." American Educational Research Journal 549-574. Accessed October 24, 2018. http://www.aera.net/Publications/Journals/JSTOR-Online-Archives.

Florida Department of Edcuation. n.d. Average Salaries for Teachers 2017-2018 Final Survey 3 Data. Tallahassee, FL. Accessed December 12, 2018. http://www.fldoe.org/accountability/data-sys/edu-info-accountability-services/pk- 12-public-school-data-pubs-reports/staff.stml.

Florida Department of Education. June 2018. "2017-2018 Guide to Calculating School and District Grades." Technical Assistance Report, Bureau of Accountability Reporting; Divsion of Accountability, Research, and Measurement. Accessed December 13, 2018. http://www.fldoe.org/core/fileparse.php/18534/urlt/SchoolGradesCalcGuide18.pdf .

93

Florida Department of Education. n.d. "FSA Technical Report: Volume 4." Evidence of Reliability and Validity, Tallahassee, FL, 1-92. Accessed December 21, 2018. https://fsassessments.org/assets/documents/V4_FSA_Technical_Report_Year_2 016-2017_FINAL_508.pdf.

Florida Department of Education. 2017. Project Application: Schools of Hope - Whole School Transformation Model. Tallahassee, FL, August 15. Accessed December 15, 2018. http://www.fldoe.org/core/fileparse.php/5673/urlt/orange-memorial.pdf.

Furgeson, Joshua, Moira McCullough, Clare Wolfendale, and Brian Gill. 2014. The Equity Project Charter School: Impacts on Student Achievement. Final Report submitted to The Bill & Melinda Gates Foundation, Cambridge, MA: Mathematica Policy Research.

Glennie, E., C. Coble, and M. Allen. 2004. Teacher Perceptions of the Work Environment in Hard-to-Staff Schools. Denver, CO: Education Commission of the States.

Goldhaber, D. 2006. Teacher Pay Reforms: The Political Implications of Recent Research. Washington, D.C.: The Center for American Progress.

Hanushek, E. A., and S. G. Rivkin. 2006. "Teacher Quality." In Handbook of the Economics of Education, edited by E. A. Hanushek and F. Welch, 1051-1078. Amsterdam: North Holland.

Hanushek, Eric A. 2011. "Paying Teachers Appropriately." In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 109-118. Cambridge, MA: Harvard Education Press.

Hanushek, Eric A. 2003. "The Failure of Input-Based Schooling Policies." Economic Journal 113 (485): F64-F98.

Hassel, B. C. 2002. Better Pay for Better Teaching: Making Teacher Compensation Pay Off in the Age of Accountability. Washington, DC: Progressive Policy Institute, 21st Century Schools Project.

Hirsch, E. 2001. Teacher Recruitment: Staffing Classrooms With Quality Teachers. ERIC Document Reproduction Service No. ED453199, Denver, CO: State Higher Education Executive Officers.

Ingersoll, Richard. 2003. Is There Really a Teacher Shortage? The Consortium for Policy Research in Education and The Center for the Study of Teaching and Policy. Accessed November 2, 2018. http://repository.upenn.edu/gse_pubs/133.

94

Ingersoll, Richard. 2001. Teacher Turnover, Teacher Shortages, and the Organization of Schools. Seattle: Center of the Study of Teaching and Policy, University of Washington.

Jupp, Brad. 2011. "What are Teachers in It For?" In The American Public School Teacher: Past, Present, and Future, by Darrel Drury and Justin Baer, 155-163. Cambridge, MA: Harvard Education Press.

Kaufman, Phillip, and Denise Bradbury. August 1992. Characteristics of At-Risk Students in NELS:88. Contractor Report, U.S. Department of Education, Office of Educational Research and Improvement, Berkeley, CA: National Center for Education Statistics. Accessed September 19, 2018. https://nces.ed.gov/pubs92/92042.pdf.

Kerchner, C T, J Koppich, and J Weeres. 1997. United Mind Workers: Unions and Teaching in the Knowledge Society. San Francisco: Jossey-Bass.

Koppich, J. 2008. Toward a More Comprehensive Model of Teacher Pay. Paper prepared for National Center on Performance Incentives’ Conference: Performance Incentives—Their Growing Impact on American K-12 Education, NCPI Working Paper 2008-06, Nashville: Vanderbilt University.

Koppich, Julia E., and Jessica Rigby. March 27, 2009. Alternative Teacher Compensation: A Primer. ERIC Document Reproduction Service No. ED510159, Berkeley, CA: Policy Analysis for California Education, PACE.

Kotterman, P. 2000. "The Teacher Quality Continuum: Recruitment, Preparation, and Retention in an Era of Teacher Shortages." Teachers: Supply and demand in an age of rising standards. Amherst, MA: National Evaluation Systems. 105-120. Accessed October 27, 2018. https://images.pearsonassessments.com/images/NES_Publications/2000_07Kott erman_430_1.pdf.

Lashway, Larry. 2003. Finding leaders for Hard-to-Staff Schools. ERIC Digest 173, December. Accessed October 11, 2018. https://scholarsbank.uoregon.edu/xmlui/bitstream/handle/1794/3234/digest173.p df;sequence=3.

Lavy, V. 2007. "Using Performance-Based Pay to Improve the Quality of Teachers." The Future of Children 17: 87-109.

Lipsky, D B, and S B Bacharach. 1983. "The Single Salary Schedule vs. Merit Pay: An Examination of the Debate." Collective Bargaining Quarterly 11 (4): 1-11.

95

Loeb, S, and M Reininger. 2004. Public Policy and Teacher Labor Markets: What We Know and Why it Matters. Lansing, MI: The Education Policy Center at Michigan State University.

Milanowski, A. 2003. "The Varieties of Knowledge and Skill-Based Pay Design: A Comparison of Seven New Pay Systems for K-12 Teachers." Education Policy Analysis Archives 11 (4). Accessed October 22, 2018. https://epaa.asu.edu/ojs/article/view/232.

Mitchell, D. J. B., D. Lewin, and E. E. Lawler. 1990. "Alternative Pay Systems, Firm Performance, and Productivity." In Paying for Productivity: A Look at the Evidence, edited by Alan S Blinder. Brookings.

Moehlman, Arthur B. 1927. Public School Finance. New York: Rand-McNally.

Murnane, Richard J, and David Cohen. 1986. "Merit Pay and the Evaluation Problem: Why Most Merit Pay Plans Fail and Few Survive." Harvard Education Review 56 (1): 1-17.

National Education Association. 2005. Rankings and Estimates: Rankings of the States 2004 and Estimates of School Statistics 2005. June. https://files.eric.ed.gov/fulltext/ED490871.pdf.

—. 2005. "Status of the American Public School Teacher, 2005-2006." National Occupational Employment and Wage Estimates. U.S. Department of Labor, Bureau of Labor Statistics, May.

New York State Education Department. n.d. Salary Percentiles for Classroom Teachers 2012-2013. New York City. Accessed December 12, 2018. http://www.p12.nysed.gov/irs/pmf/2013/2013_Stat-14.pdf.

Odden, A,, and M. Wallace. 2004. "Experimenting with Teacher Compensation." The School Administrator (61): 24-28. Accessed October 29, 2018. http://www.wilsonweb.com.

Odden, Allan, and Carolyn Kelly. 2002. Paying Teachers for What They Know and Do: New and smarter compensation strategies to improve schools. 2nd Edition. Thousand Oaks, CA: Corwin Press.

Painter, Suzanne, Thomas Haladyna, and Sally Hurwitz. 2007. "Attracting Beginning Teachers: The Incentives and Organizational Characteristics that Matter." Planning and Changing 38 (1-2): 108-127.

Podgursky, Michael, and Matthew Springer. 2011. "Teacher Compensation Systems in the United States K-12 Public School System." National Tax Journal 64 (1): 1-28. Accessed October 27, 2018.

96

https://faculty.missouri.edu/~podgurskym/doc/articles/18teachercompensationsys temsart2011-36.pdf.

Popham, James W. 2000. Modern Educational Measurement: Practical Guidelines for Educational Leaders. 3rd ed. Boston: Allyn and Bacon.

Prince, C. D. 2003. Higher Pay in Hard-To-Staff Schools. Lanham, MD: University Press of America, Inc.

Prince, Cynthia D. 2009. The Other 69 Percent: Fairly Rewarding the Performance of Teachers of Non-Test Subjects. Office of Elementary and Secondary Education, U.S. Department of Education, Washington: The Center for Educator Compensation Reform.

Protsik, Jean. 1995. History of Teacher Pay and Incentive Reform. Washington: Educational Resources Information Center.

Razali, Nornadiah Mohd, and Yap Bee Wah. 2011. "Power comparisons of Shapiro- Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests." Journal of Statistical Modeling and Analytics 2 (1): 21-33. Accessed December 12, 2018. https://www.researchgate.net/profile/Bee_Yap/publication/267205556_Power_Co mparisons_of_Shapiro-Wilk_Kolmogorov-Smirnov_Lilliefors_and_Anderson- Darling_Tests/links/5477245b0cf29afed61446e1/Power-Comparisons-of- Shapiro-Wilk-Kolmogorov-Smirnov-Lilliefors-an.

Ripley, Amanda. 2013. The Smartest Kids in the World. New York: Simon & Schuster, Inc.

Southeast Center for Teaching Quality. n.d. Recruiting Quality Teachers to Hard-to-Staff Schools. James B Hunt, Jr. Institute for Educational Leadership and Policy. Accessed September 27, 2018. http://www.nea.org/assets/docs/HE/mf_HuntHardtoStaff.pdf.

Springer, Matthew G. 2009. Performance Incentives: Their Growing Impact on American K-12 Education. Washington D.C.: Brookings Institution Press. Accessed November 2, 2018. https://muse.jhu.edu/.

Springer, Matthew G, Dale Ballou, Laura Hamilton, Vi-Nhuan Le, J. R. Lockwood, Daniel F. McCaffrey, Matthew Pepper, and Brian M. Stetcher. 2012. Final Report: Experimental Evidence from the Project on Incentives in Teaching. Nashville, TN: National Center on Performance Incentives at Vanderbilt University.

Stronge, J. H., C. R. Gareis, and C. A. Little. 2006. Teacher Pay and Teacher Quality: Attracting, Developing and Retaining the Best Teachers. Thousand Oaks, CA: Corwin Press.

97

Wise, A. 2004. "Teaching Teams." Education Weekly. Accessed October 25, 2018. https://www.edweek.org/ew/articles/2004/09/29/05wise.h24.html.

98

BIOGRAPHICAL SKETCH

Kenneth Savage was born in Willoughby, Ohio and moved to the State of Florida when he was an elementary school student. He grew up in Fort Myers, Florida and graduated from Cypress Lake High School and was accepted into film school at Florida

State University. While a student at FSU, Ken’s love for coaching sports motivated a change in majors to the field of education. After graduating from FSU with his Bachelor of Science, Ken immediately began his teaching career in a Gainesville, Florida middle school. After relocating to Jacksonville, Florida and working in an extremely at-risk middle school, Ken came face-to-face with a palpable level of poverty that he had never before experienced. Consequently, his passion for social justice was born and his desire to become an educational leader focused on equity became his driving professional thrust.

After working as an assistant principal and becoming a principal back in his

Florida home of Lee County, Ken’s leadership experiences in suburban schools, high performing Title I schools, and numerous turnaround schools became some of the defining experiences of his life. Having worked as a leader in all school levels, elementary, middle, and high school, Ken began a hybrid role as both a district and school leader. His role was to mentor other school principals in turnaround school leadership situations while still having the school-based role as well. Ken was honored as the State of Florida Principal of the Year for 2018. This tremendous honor afforded him an opportunity to discuss his passion for equity and his practical leadership in helping to successfully turn-around low performing schools. Ultimately, Ken was offered a position at the Florida Department of Education to serve on the executive cabinet for the K-12 Chancellor for Public Schools. This new vantage point offered Ken a chance to

99

learn about the complexities of policy-level leadership at the state level and an opportunity to serve the entire State of Florida.

At the time of graduation, Ken lived in Tallahassee, Florida with his wife,

Rebecca and their fraternal twins, Ruby Lorraine and Jack Dean. Not only did Rebecca also work in the field of education, but so too did Ken’s brothers, Frank and Phillip. As a family of educators, this journey in education was a labor of love and of public service.

The doctoral degree from the University of Florida represented a seminal milestone in

Ken’s professional career, but in his mind the real value would eventually be judged with how he was able to utilize this privilege he had been afforded to serve the greater good.

The pursuit of social justice was his life’s work and from that standpoint he viewed this doctoral degree as the beginning of that next phase of service that scholarly responsibility demanded.

100