2013 Transportation Efficiency Review Xxxxxschool District
2013 Transportation Efficiency Review [XxxxxSchool District]
Office of Superintendent of Public Instruction
Student Transportation
Transportation Efficiency Review
for: [District.]
6/19/2013
Table of Contents
Executive Summary3
System Overview4
Review Methodology5
SectionOne:Relative Efficiency Rating6
SectionTwo:Key Performance Indicators7
Section Three: Initial School District Interview XX
SectionFour: School District Comments XX
Appendix IEfficiency Detail Report XX
Appendix IIOnline Transportation Survey XX
Appendix IIIKey Performance Indications XX
Executive Summary
Revised Code of Washington (RCW) 28A.160.117 requires the Office of Superintendent of Public Instruction (OSPI) and the Regional Transportation Coordinators (RTCs) to provide an efficiency rating process to encourage school districts to operate their student transportation systems in an efficient manner. The ratings are generated by a widely used methodology known as Data Envelopment Analysis, a mathematical optimization process using linear programming. Except for geographic data, all the inputs used in the analysis are submitted by the district through the Student Transportation Allocation Reporting System (STARS), used to calculate the district’s annual transportation funding.
Xxxxx School District is a (size-descriptive) district in (region description) Washington with a full time equivalent enrollment for the 2011–12 school year of xxxx.x students. Of those enrolled, Xxxxxprovided home-to-school transportation service for an average of xxxx basic program riders and xxxx special program riders per day (combined AM and PM student counts divided by two). The district operated xxx school buses to provide this service, at a total cost of $xxx,xxx.xx.
The efficiency system target for XXXXXX School District would be to operate xxx school buses at a cost of $xxx,xxx.xx.
The Regional Transportation Coordinator review was conducted by Xxxxx Xxxxxx.
The regional coordinator found that the most significant factors regarding the district’s operating costs and efficiency were:
- Xxxx
- Xxxx
Suggested areas to evaluate for improvement in efficiency are:
- Xxxxx
- Xxxxx
System Overview
Revised Code of Washington (RCW) 28A.160.117 requires the Office of Superintendent of Public Instruction (OSPI) to provide an efficiency rating process to encourage school districts to operate their student transportation systems in an efficient manner. The majority of the data used by this efficiency rating process is identical to that used by the Student Transportation Allocation Reporting System (STARS), the new student transportation funding system implemented September 1, 2011.
It is important to note that any system providing an efficiency rating of a real world transportation operation does so by comparison to some arbitrarily defined standard. The possible efficiency of any school district’s transportation system is constrained by every school district having unique local characteristics, but also by individual student transportation requirements that are often beyond the district’s control. For instance, a school district may be required to provide relatively expensive transportation to enable access to appropriate services for a special needs student.The resulting impact on expenditures may be significant and have a corresponding negative impact on the relative efficiency rating1.
Rather than using an arbitrary standard such as comparing each district with a particular benchmark cost per student, OSPI determines the relative efficiency rating for each school district using a statistical method called theTarget Resource Model (TRM). For districts less than 100percent efficient, TRM creates a statistical “target district” from other school districts across the state that have environmental features, size characteristics and workload requirements that are the same or more challengingand compares the district’s total transportation costs and the number of buses used with this “target”. The target district establishes the resource requirements (expenditures and number of buses)that would be needed to achieve a 100percent relative efficiency score. For districts rated less than 100percent efficient, the “efficiency cohort districts” are those districts used in developing the target district characteristics. For additional details on the TRM, see “Development of Student Transportation Funding Methodology Options for Washington State” on OSPI’s Student Transportation website:
Under the requirements of RCW 28A.160.117, the Regional Transportation Coordinators (RTCs) are required to provide individual efficiency reviews of those school districts with a relative efficiency rating of less than 90percent, in order to assist those districts in determining what actions are available to improve operational efficiency. For the 2013 Efficiency Ratings, a statistical overview will be provided to all
- Ideally, the efficiency rating should be composed of two separate evaluations: basic program transportation efficiency and special education transportation efficiency. However, this is not possible since the school district accounting system does not separate these costs. (Requiring districts to separate these costs would be a significant multiplier of bookkeeping workload.)
districts with a rating less than 90percent and an additional in-depth review will be conductedas the RTC and district schedules allow. As required by statute, the results of reviews will be reported to the Washington State Legislature in December of each year. Districts will be provided with the opportunity to respond to the RTC review and those responses will be included in the legislative report along with details of any actions that school districts have taken to improve efficiency.
In many ways, the most important evaluation will come in the comparison of year-to-year changes in a district’s efficiency rating. If a district is taking actions to improve their operational efficiency, one would expect the year-to-year change in the published efficiency rating to be positive. However, due to the relative nature of the system,in order for a district to showan increase in efficiency rating, the changes they implement must providea greater relative improvement than the improvements to efficiency undertaken bythe efficiency cohort districts used to establish the target district. As district workload and expenditure characteristics change from year to year, the particular districts included in the efficiency cohort will change.
A good example to show the time required to improve efficiency ratings would be to examine bell time schedules. In many cases, the greatest potential change in operations to improve efficiency will be restructuring school bell times. But because changing bell times has a significant impact on parents, students and district staff, these changes must be undertaken in a manner that provides an opportunity for involvement of all parties. In some cases, changes are subject to constraints in current bargaining agreements. Thus, some districts will find that even once potential efficiency improvements have been identified; it may take significant time to implement those changes.
Changes in bell times typically are made on a school year basis. The disruption involved in changes in calendars and schedules is generally too extensive to permit such modifications in the middle of a school year. Since the initial rating release is in the second half of the 2012–13 school year, even in the best case, bell time changes will not be implemented until the beginning of the 2013–14 school year. As a result, the next release of efficiency ratings (March 2014) will not reflect improvements resulting from bell time restructuring. More typically, perhaps, would be for the evaluation and decision on bell times to be made during the 2013–14 school year and implemented with the beginning of the 2014–15 school year. This would result in the efficiency rating released in March of 2016 reflecting the impact of the bell time changes … a three year delay.
The use of other measures of efficiency, generally known as “Key Performance Indicators” (KPIs) may be important to demonstrate improvement in cases where the efficiency cohort has changed from year to year. KPI’s are also important as a method for districts that are rated at 100 percent efficiency to conduct a self-analysis review. The KPIs are also useful to review those districts that are rated at 100 percent efficiency due to their unique characteristics. If a district has one or more site characteristics whose values are calculated as significantly unique, there will be no comparable districts to provide appropriate efficiency cohorts. The result is that the district is given a default rating of 100 percent.
For all of these reasons, the efficiency rating system should be seen as a long-term (multiple-year) tool for school districts to use for improving their transportation operations.
Review Methodology
The approach used for the 2013 (initial) reviewof a school district’s transportation program involvesfour primaryparts:
- Determining and evaluating the relative efficiency results from the Target Resource Model;
- Determining and evaluating the key performance indicators (KPI) from the data already provided by a school district to attempt to identify where the primary efficiency concerns may reside and to establish a baseline for future evaluation;
- Conducting a meeting with school district personnel to review the answers to an online questionnaire (a copy of the district’s completed questionnaire is provided in Appendix II), to gain the perspective of those on the “inside” of the program, to review district specific data, and to review with school district staff the relative efficiency ratings and KPIs; and
- Providing a draft copy of the report to school district staff, providing an opportunity for school district staff to comment on the report, and providingany appropriate response to school district questions.
The online questionnaire wasdesigned to require a minimum amount of staff time, while providing the RTC with the information required in evaluating the impact of the district’s transportation policies, expenditures, logistics and operations on the efficiency of the district’s transportation system. These four terms (policies, expenditures, logistics and operations) are described in more detail in Section Three of thisreport. For the 2013 Efficiency Rating Reviews, all districts were requested to complete the online survey in order to establish the best practices of districts that are rated as efficient, besides those districts involved in the review process.
Using this process, each district’s transportation program is evaluated by quantifying the key indicators of performance as these relate to the efficient use of both state fiscal resources and the number of school buses used in transporting district students. As resources are available, the RTCs will provide a more subjective assessment of the district’s transportation operations through direct observation and interaction with the school district staffresponsible for the administration and management of the transportation program.
Section One: Relative Efficiency Rating
The STARS efficiency rating system uses a type of linear programming called Data Envelopment Analysis (DEA). The system was developed to be used as a tool for school districts to improve the efficiency of their transportation operations. The results returned includean efficiency rating based on a comparison of the district to a mathematically constructedtarget district’s expenditures and a rating based on the number of buses operated. Many times the two ratings are identical. If the ratings are different, the overall efficiency rating used by STARS is the higher of the two and this is the rating shown on the STARS Efficiency Detail Report.
During the analysis, districts are broken into quartiles based on total student ridership count (combined AM plus PM) to ensure that the efficiency cohorts only include those districts of a similar size. The quartile breakout for the 2011–12 school year is:
1st Quartile224 or fewer student riders
2nd Quartile225 to 799 student riders
3rd Quartile800 to 2993 student riders
4th Quartile2994 or more student riders
The efficiency process uses the same data set as the STARS funding system, with the exception that the efficiency system includes the number of route buses operated. However, the statistical process used in the funding system is a regression analysis not DEA. Since the two statistical processes are different, there are likely to be examples of districts being fully funded (where the calculated allocation is 100percent of the prior year expenditures) and the efficiency rating not being 100percent. Likewise, there will be examples where districts are rated at 100percent efficiency and the funding system does not provide 100percent of the district’s expenditures. This is a result of two different statistical systems being used for these calculations.
The 2013 relative efficiency score for the Xxxxxx School District was XX.X%. The Efficiency Detail Reportincluding the cohort districts and predicted resource requirements isprovided in Appendix I.
RTC Analysis and Comments
The regional transportation coordinator reviewed the data on the Efficiency Detail Report with district staff. XXXX School District is in the X Quartile and has X districts identified as efficiency cohorts. In the 2011–12 school year, the district reported operating XX buses to transport a total student ridership count (combined AM plus PM) of XXX with expenditures of $xxx,xxx.xx. The efficiency calculation identified target expenditures for XXXXX of $xxx,xxx.xx while operating xx buses.
SectionTwo: Key Performance Indicators
The development of Key Performance Indicators (KPIs) for student transportation is a current topic of national concern. Several national organizations are involved with attempting to determine a core set of KPIs that can be used to evaluate and compare any school district’s transportation operation. This work is progressing slowly due to the volunteer nature of the effort and the existence of extreme state-to-state variations in required service levels, school district size variation and expenditure reporting requirements.
Key Performance Indicators are sometimes referred to as “benchmarks”. The current view within the industry is that the term benchmark has connotations of being a one-size-fits-all standard that provides a hard quantitative line between efficient and inefficient transportation operations. This may be appropriate in some system analysis;however, actual student transportation operational efficiency is subject to many conditions totally outside the control of school districts. For instance, the population density of the district and thegeographic distribution of students may place severe constraints on the district’s operational efficiency. As an example, in a rural district there may be one student living a great distance up a canyon or isolated highway. The district has no choice but to send a bus to provide the required access to a public education, regardless of the impact on theirbudget or efficiency score. Similarly, a special education student may legally require expensive transportation service. Two districts with otherwise similar environmental conditions may vary in their relative efficiency rating or KPI numbers due to situations such as these. KPIs by themselves do not reveal the existence of these extenuating factors (nor does the STARS Efficiency Rating), when a KPI is used to compare one district to another in isolation.
The primary strength of KPIs is when they are used over time to provide an ongoing method of evaluating the district’s transportation operations. Year to year review of the basic statistics revealed by the KPIs are an essential part of evaluating the status of the transportation department’s performance.
A full list of all suggested KPIs would overwhelm the reader with statistics of questionable worth. For the purposes of this review, consideration will be limited to three KPIs that areamong the most widely recognized as critical indicators. These are:
- Basic program riders per basic program bus;
- Special education riders per special education bus; and
- Overall cost per rider.
Some additional description of these individual KPIs may help to clarify their relevance and impact on a district’s efficiency rating:
- Basic Program Riders per Basic Program Bus
The number of basic program riders per basic program bus is a key measure of efficiency. For calculating the riders per bus, the total student ridership count is divided by two in order to more closely reflect actual student loads per bus. The number of buses used is as reported in the STARS routing data.
This KPI is one that has a fairly widely recognized “benchmark” value of 100. This assumes a two-tier system with ~40 middle/high and ~60 elementary. Large school districts with high student density will be able to achieve higher riders per bus values much easier than small or rural districts where geographic constraints may make achieving a count this high impossible. In particular, in many small or rural districts the elementary school, the middle school and the high school all have the same bell time. This allows bus routes in remote areas to pick up all grades simultaneously and reduce costs. As a result, the total basic program student count per basic bus can never reach the “benchmark” of 100, since that number is only achievable with multiple routing.
2. Special Education Riders per Special Education Bus
The number of special education riders on a special education bus will vary significantly from district to district based on the individual student needs required to be served by the district. As in the basic riders KPI, the total special education student ridership count (combined AM plus PM) is divided by two and the bus count is as reported in the STARS route data. Also similar to the previous KPI is the fact that larger districts in urban settings will be more easily able to achieve higher rider per bus counts.
The typical requirement for special education routes is to provide door-to-door service which may severely constrain the total number of riders able to be served per bus. The national guideline on special needs riders per bus is ten … with the assumption that this reflects a two-tier system. This would place the average student load per route at five … reflecting the longer per student loading time and the wider geographical distribution of stops.
Additionally, there is a state guideline established by Washington Administrative Code(WAC) 392-172A-02095 that school districts should, when possible, limit the one-way transportation times for special education students to 60 minutes. A district may also have one or more students requiring individualtransportation (without other students) due to the student’s Individualized Education Program (IEP) requirement.