Comparative Performance Measurement Efforts

Comparative Performance Measurement Efforts Updated for the AASHTO Standing Committee on Performance Management Co-Chairs, Mara Campbell and Daniela Bremmer

October 2009

Three years ago, several members of the AASHTO Standing Committee on Quality recognized the need for states to band together and provide “apple-to-apple” comparisons on performance indicators relevant to the transportation industry as a whole. This team created a pilot of seven states to see if real comparative efforts could be successful. The pilot proved to be extremely insightful and subsequently led to the first NCHRP funded comparative performance measure effort. To date, these efforts have demonstrated the value of comparative performance measurement in three areas: Project Delivery, Smooth Pavements, and Safety. Each of these projects has involved the compilation of detailed performance data from multiple DOTs, calculation of performance measures for each agency, composition of peer groups for comparative analysis, identification of the top tier of agencies with respect to the selected measures, and interviews to determine practices that may be related to exemplary performance.

The first comparison of on-time, on-budget capital project delivery performance project successfully demonstrated how the comparative process developed by NCHRP 20-24(37) is gathering user support, as well as delivering timely feedback on best practices that have achieved high performance. In this particular study, 20 states participated and numerous best practices were identified. Perhaps most importantly, this initial project dispelled the myth that states couldn’t compare “apple-to-apple” data and proved that there was much to be learned by looking at other states’ performance.

The next comparative initiative, which was completed in April 2008, successfully demonstrated once again that states could benefit from comparative measures. This comparative study involved 32 states and highlighted five top performing states and the practical management methods and technical applications being used to deliver “smooth pavements.” The third comparative measures project on safety performance was just recently completed. This project used data from the Fatalities Accident Reporting System to compare safety performance among all 50 states. Safety trends for two three- year periods were used to identify high performing states. Interviews with 10 states identified best practices in governance, budgeting and technical methods, which resulted in reduced fatalities.

The first three comparative projects have been well received by the transportation community and have produced a wealth of data and information to be shared among agencies. These successful endeavors have created momentum for further interest in comparative measurement to which the SCOPM has responded with two new projects: bridge conditions and incident management. The bridge condition project already has 35 states participating. This study will identify casual links that may exist between network- level bridge condition and best practices, as well as produce recommendations regarding future use of comparative analysis of bridge performance. It will also identify common bridge performance indicators that can serve as a tool for identifying and sharing best practices and recommend steps that can, over time, achieve a greater degree of consistency and comparability in measures. The draft results of this study are expected to be available at the Annual TRB meeting in January. The incident management project has just begun the process of soliciting states to participate.