1 STATE OF CALIFORNIA – HEALTH AND HUMAN SERVICES AGENCY ARNOLD SCHWARZENEGGER, Governor EMERGENCY MEDICAL SERVICES AUTHORITY 1930 9th STREET SACRAMENTO, CA 95811-7043 (916) 322-4336 FAX (916) 324-2875 1

EMS System Guidelines EMSA #160

State EMS Data Collection, Evaluation, and Quality Improvement System Overview

EMSA #160 Commission on EMS December 5, 2007 2

2 1 1 Prepared by: 2 3 The California Emergency Medical Services Authority 4 5 AS DEVELOPED BY: 6 State EMS Data Committee 7 8 9 Arnold Schwarzenegger 10 Governor 11 12 ~ 13 14 Kim Belshé 15 Secretary 16 California Health and Human Services Agency 17 18 ~ 19 20 Cesar A. Aristeiguieta, M.D. 21 Director 22 California EMS Authority 23 24 ~ 25 26 Bonnie Sinz, RN 27 Chief, EMS Systems Division 28 California EMS Authority 29 30 31 32 33 34 35 36 37 38 39

2State EMS Data Collection, 2 3Evaluation, and Quality Improvement 4System Overview 1 1 ACKNOWLEDGEMENTS 2 3 EMS DATA COMMITTEE 4 5 6Steve Andriese, 46Kristy Harlan 7Administrator 47Data Coordinator 8Mountain Valley EMS Agency 48American Medical Response 9 49 10Barbara Stepanski, MPH 50Luanne Underwood, RN 11Epidemiologist 51QI Program Director 12San Diego County EMS Agency 52Los Angeles County Fire Department 13 53 14Glen Youngblood 54Ric Maloney, RN 15EMS Coordinator/Data Analyst 55CQI Manager/Paramedic Instructor 16San Mateo County EMS Agency 56Sacramento Metro Fire Department 17 57 18Linda Combs 58Maureen Hasbrouck, RN, MS 19Sierra-Sacramento Valley EMS Agency 59EMS Data System Coordinator 20 60Los Angeles County EMS Agency 21Craig Stroup 61 22Emergency Medical Sciences 62Adele Pagan 23Training Institute 63Information Systems Analyst 24 64Alameda County EMS Agency 25Allen N. Norman, MPA, EMT-P, Battalion 65 26Chief 66Bob O’Brien, Division Chief 27Commander, Quality Improvement Section 67Fremont Fire Department 28Los Angeles Fire Department 68 29 69Leif Juliussen 30Che-Chuen Ho 70CSFA EMS Committee 31Data Systems Manager/Trauma Data 71Milpitas Fire Department 32Analyst 72 33Santa Clara EMS Agency 73David Lindberg 34 74Todd Hatley 35Cindy Wilmshurst 75Project Consultants 36Special Projects Coordinator 76HealthAnalytics 37National EMSC Data Analysis Resource 77 38Center 78EMS Authority Representatives: 39 79Bonnie Sinz, RN 40Mike Denton, RN 80Ed Armitage 41Lawrence Livermore National Laboratory 81Charla Jensen 42 43Ginger Ochs, RN 44Quality Management Coordinator 45San Diego Fire Department

2State EMS Data Collection, 3 3Evaluation, and Quality Improvement 4System Overview 1 1 2 EMS System Guidelines 3 EMSA #160 4 5 State EMS Data Collection, Evaluation, and 6 Quality Improvement System Overview 7 8 “The ability of EMS to optimally meet communities’ and 9 individual patients’ needs in the future is dependent 10 on evaluation processes that assess and improve the 11 quality of EMS. Continuous evaluation is essential 12 and should pervade all aspects of every EMS system.” 13 14 Theodore R. Delbridge MD, MPH 15 16 17 Introduction 18 19 In the above quotation from the 1996 National Highway Traffic Safety 20 Administration’s (NHTSA) publication, Emergency Medical Services, Agenda for 21 the Future1, Dr. Delbridge stresses the importance of an effective evaluation and 22 improvement system if EMS systems in this country are to be successful in the 23 future. 24 25 This statement can be interpreted as containing both a “quality-of-care,” as well 26 as an “economic” warning. When economies tighten at the federal, state, and 27 local levels, it becomes extremely difficult to compete for limited healthcare 28 dollars if EMS systems cannot definitively show their worth to the community. 29 More importantly, the fact that EMS systems are administering medications and 30 practicing invasive medical procedures carries with it a fiduciary responsibility to 31 show that those systems are safe and effective for the patient being served, and 32 to demonstrate what affect the EMS system has on the final outcome of the 33 patient. 34 35 In an effort to encourage standardized data collection and system evaluation on a 36 national level, NHTSA developed their original standardized EMS data standards 37 in 1993, which most states, including California, adopted. However, these data 38 standards lacked many of the necessary elements required for effective system 39 evaluation, did not contain clear and concise definition for all data elements, and 40 did not include standardized quality indicators for which the data elements would 21 NHTSA, Agenda for the Future, 1996; p57 3 4State EMS Data Collection, 4 5Evaluation, and Quality Improvement 6System Overview 1 1 be used. Because the original NHTSA data standards had its limitations, and 2 because many states did not have the resources or capability for large scale 3 participation in EMS data collection programs, no real effective data were 4 produced on a national level. 5 6 In July 1997, NHTSA released “A Leadership Guide to Quality Improvement (QI) 7 for EMS Systems” which has become a standard for EMS QI nationwide. In 8 addition, in April 2004, following a two-year development process, NHTSA 9 announced the release of a complete rewrite of their EMS data standards 10 entitled, NHTSA Uniform Prehospital Dataset, Version 2.1. Even though this 11 document still does not contain standardized quality indicators for EMS, it does, 12 along with the 1997 QI Guide, provide a level of standardization in EMS 13 evaluation and improvement that has never existed before. 14 15 The leadership of NHTSA is important since their work finally provides a standard 16 for local EMS systems and EMS software vendors nationwide to utilize in their 17 data system designs. Without this leadership at the national level, software 18 vendors and state/local EMS offices would continue to develop new, non- 19 comparable and non-compatible data systems. With this new data set as a 20 foundation, those who choose to follow these standards will be able to produce 21 and compare data across local borders and state lines. Consistent with this 22 vision is the project by the State EMS Officials’ Association, in conjunction with 23 NHTSA, to implement the National EMS data repository and reporting system 24 (National EMS Information System [NEMSIS]). NHTSA is also supporting the 25 National EMS Performance Indicators project for development of a standardized 26 national set of EMS performance indicators, now under development. 27 28 With the foundation for a standardized approach to EMS system evaluation and 29 improvement in place at the national level, it has moved the challenge of 30 implementing these standards to the state level. 31 32 33 History of EMS System Evaluation and Improvement in California 34 35 By 1999, California had not been much more successful than the rest of the 36 nation in creating an effective EMS evaluation and improvement system. The 37 California EMS Vision Project, which was established by the EMS Authority and 38 EMS Commission in 1999, addressed several areas of perceived deficiency in the 39 EMS system in California, and identified the establishment of an effective, 40 standardized, data collection and quality improvement system among the top six 41 EMS needs within the state. The 1999 National Highway Traffic Safety 42 Administration’s (NHTSA) evaluation of the California EMS system reiterated this 43 finding when they reported that there is, “a lack of an integrated, statewide 2State EMS Data Collection, 5 3Evaluation, and Quality Improvement 4System Overview 1 1 information system that (has) the capability to monitor, evaluate and elucidate 2 emergency medical services and trauma care in California.”2 3 4 Much like the rest of the nation, California has had some success stories in local 5 system evaluation and improvement. In the 1980’s and early 1990’s, local EMS 6 agencies (LEMSAs) were encouraged, through the EMS System Guidelines and 7 special project funding, to develop data collection systems. However, without a 8 clear standard for how these systems should be designed and utilized, LEMSAs 9 developed their data collection systems independent of each other. While a state 10 data standards document (based on the original NHTSA data standards did exist, 11 it was void of any accompanying performance indicators, effective definitions, 12 benchmarks, or standardized reporting capabilities. The LEMSAs that did 13 develop data systems utilized their own definitions and data parameters. 14 Consequently, they were only able to measure their performance against 15 themselves. With the exception of some basic descriptive or structural data 16 reporting capabilities, no comparative analysis of performance from LEMSA to 17 LEMSA, or on a statewide basis was possible. In addition, even in areas where 18 data was collected, it was not always used effectively for quality improvement 19 purposes. Through both the Vision Process, and the NHTSA assessment, this 20 was identified as an unacceptable system deficit. 21 22 The Vision Process ran from 1999 to 2003. During that period, several multi- 23 agency groups, committees and task forces assisted in the development of the 24 EMS System Evaluation and Quality Improvement System outlined in EMSA 25 Series 160. Under the overall coordination of the “Vision Work Group D, System 26 Evaluation and Improvement,” the Paramedic Task Force, EMS Data Committee 27 and many EMS Constituency groups were instrumental in creating the referenced 28 documents below. The ultimate goal of the project was to create an outline for a 29 comprehensive EMS system evaluation and improvement system in California 30 that was compatible with all national standards in force at the time. The results of 31 those years of work make up EMSA Guidelines #160 -168. 32 33 Purpose and Format of the EMSA-160 Series Guidelines 34 35 In reviewing the 160 series guidelines, the reader will notice that the concepts of 36 EMS system evaluation and EMS quality improvement are treated as a 37 continuum. 38 39 System Evaluation, which consists of data collection, data analysis, and system 40 research, is futile from a quality of care perspective if that evaluation is not 41 reported and utilized for system improvement. Likewise, EMS Quality 42 Improvement cannot be effective if it is not based upon sound data and research.

22NHTSA, Assessment of Emergency Medical Services in California; August, 1999; p30 3State EMS Data Collection, 6 4Evaluation, and Quality Improvement 5System Overview 1 1 Therefore, these two concepts are presented as a single process, broken down 2 into the components of Guidelines #161-168 3 4 The guidelines contained in the 160 series should provide the step-by-step 5 guidance necessary for a local EMS service provider, a LEMSA, or the State 6 EMS Authority to identify their specific roles and responsibilities in a standardized, 7 statewide EMS evaluation and improvement system. While some of the 8 documents necessary for a comprehensive program have not yet been completed 9 (these are identified in the Table of Contents in gray italic print), the documents 10 that are currently included provide the key elements necessary to establish and 11 implement effective data collection, and quality improvement programs at the 12 provider, LEMSA, base hospital and state levels. 13 14 Listed below is a brief description of each of the Guideline documents included in 15 the Table of Contents. 16 17 I. Evaluations of EMS Regulatory Agencies 18 19 EMSA # 161 - State EMS Authority Assessment Guidelines (Not yet 20 Developed) 21 22 EMSA # 162 - Local EMS Agencies Assessment Guidelines (Developed 23 but not validated or tested) 24 25 Besides stressing the needed for a sound evaluation and improvement 26 system, the NHTSA EMS Agenda for the Future, referenced in the 27 introduction of this document, also conveys the importance of assessing 28 “all aspects” of the EMS system. A comprehensive EMS CQI program must 29 include evaluation and ongoing improvement within all components of the 30 system, not just the medical care provided in the field and designated 31 receiving facilities. To this end, guidelines are being developed that 32 include evaluation and improvement of the services provided by the state’s 33 EMS administrative agencies. The evaluation would include the State EMS 34 Authority and local EMS agencies, to ensure that planning, implementation, 35 and monitoring functions are being performed efficiently and effectively, 36 and that the day-to-day duties of these agencies are supportive of the care 37 being provided in the field. 38 39 40 41 42 II EMS System Evaluation 43

2State EMS Data Collection, 7 3Evaluation, and Quality Improvement 4System Overview 1 1 EMSA # 163 – State EMS Core Quality Indicators (Appendix M to the 2 Emergency Medical Services System Quality Improvement Program 3 Model Guidelines) 4 Historically, EMS data collection system design has begun with the 5 establishment of a data set which included those elements that the creators 6 felt would be worthwhile for future reporting. It hasn’t been until after the 7 systems were developed that the reporting questions were actually asked 8 of the system. Very often, because the software was not designed to 9 answer a particular question, system modification and additional 10 reprogramming would be necessary. As new questions were asked, new 11 modifications were required. 12 To ensure that the amount of system modifications was kept to a minimum, 13 prior to finalizing the California State EMS Data Standards, a different 14 approach was taken. Instead of building the data collection system, and 15 then asking the questions, the process was reversed. The Vision Work 16 Group D spent more than two years determining key questions that must 17 be answered to ensure that all key components of the EMS system could 18 be effectively evaluated. These questions were then translated into the 19 State EMS Core Quality Indicators (EMSA # 163). Only after these 20 questions were carefully established was the State EMS Data Standards 21 (EMSA # 164) finalized. This approach has helped ensure that the 22 necessary data elements to answer key system questions can be 23 answered once the system is fully functional. However, even with the 24 amount of time spent on the development of the Core Quality indicators, 25 the Work Group was not able to complete all the performance indicators 26 they set out to develop. Work on the development of new core quality 27 indicators will be on-going as EMS providers, LEMSAs and the State EMS 28 Authority continue to refine and evolve their approaches to EMS 29 performance improvement. When the initial version is fully completed, 30 EMSA #163 should include standardized indicator definitions and minimum 31 benchmark values to facilitate comparative analysis of local system 32 performance, quality of patient care, customer satisfaction, and system 33 cost on a local and state level. Once EMS Core Quality indicators have 34 been established at the national level, it is anticipated that the California 35 indicators will be revised to comply with those indicators. 36 37 38 39 40 EMSA # 164 - State EMS Data Standards 41

2State EMS Data Collection, 8 3Evaluation, and Quality Improvement 4System Overview 1 1 The State EMS Data Standards consists of a comprehensive list of 2 minimum data elements and definitions consistent with two main sources. 3 First it will always be in compliance with the NHTSA Uniform Prehospital 4 Dataset, National EMS Information System [NEMSIS]). Secondly, it also 5 contains the elements necessary for monitoring and evaluating the State 6 EMS Core Quality Indicators (EMSA # 163). 7 8 The State EMS Data Standards contains the names of each data element, 9 the source(s) from which the data can be obtained, and a detailed, practical 10 operational definition of the element. Additionally, validation criteria have 11 been developed to evaluate data quality and integrity at multiple points in 12 the data collection, transfer, storage, and analysis process. 13 14 The entire list of State EMS Data Standards is categorized as ‘Minimum,’ 15 which means they must be collected for a given incident in circumstances 16 where applicable. As an example of applicability; the date that the call was 17 received would be considered to be a data element that would be 18 applicable to all incidents. In contrast, the name of the patient may not be 19 applicable because in some circumstances, there may not be a ‘patient’ 20 identified for a particular EMS response. This may occur with an 21 automobile crash where the vehicles and their occupants cleared the scene 22 before any emergency responders arrived on-scene. 23 24 25 EMSA # 165 - State Data Collection and Reporting Guidelines (Under 26 Development) 27 28 The State EMS Authority has established a statewide EMS data collection 29 reporting system which has been named the California EMS Information 30 System or CEMSIS. Once fully online, this system will be able to collect 31 standardized data from all participating LEMSAs in the state and provide a 32 web-based reporting capability for the public, LEMSAs, and service 33 providers to utilize. This data collection system will develop over time and 34 is to be based on available resources at the LEMSA and provider agency 35 level. 36 37 At first, this system will be able to collect Patient Care Record (PCR) only 38 in accordance with the EMS Data Standards. Once the system has 39 matured, Computer-Aided Dispatch (CAD) data will be included with 40 linkage capabilities with the National Fire Incident Reporting System 41 (NFIRS). This will initially limit reporting capabilities to field operations and 42 field care. However, the system is designed to be expanded to link with the 43 State Trauma Registry, Office of Statewide Hospital Planning and

2State EMS Data Collection, 9 3Evaluation, and Quality Improvement 4System Overview 1 1 Development (OSHPD) which stores hospital and emergency department 2 outcome data. Further expansion to include the Department of Health 3 Services vital statistics data (e.g., death statistics), and the California 4 Highway Patrol CRASH data is also included in the long term expansion 5 plan. With these vital linkages, California will be well positioned to provide 6 many different types of EMS patient outcome data that have never before 7 been available. 8 9 EMSA # 165 is designed to provide the timeline, instructions and format 10 for LEMSAs and local providers to follow in order to successfully up-load 11 local data into CEMSIS as well as how to utilize the web-based reporting 12 features. The system features will ultimately include: 13 14  Process performance feedback via data reports to all participating 15 EMS agencies 16  Maintenance of data confidentiality and security 17  Mechanisms for feedback to prehospital personnel on the diagnosis 18 and disposition of their patients 19  Collection and sharing of data among EMS system participants, to 20 include integration and linkage of data with other private, state, and 21 federal agencies, and organizations as appropriate. 22 23 III. EMS System Quality Improvement 24 25 EMSA # 166 – State EMS System QI Program Model Guidelines 26 EMSA # 166 follows the guidelines published in NHTSA’s “Leadership 27 Guide to Quality Improvement (QI) for EMS System’s.” It includes 28 mechanisms to ensure that EMS performance data is utilized at the state, 29 local and provider level for continuous quality improvement aimed at 30 improving EMS services and quality of patient care, decreasing death and 31 disability, and reducing costs. It also establishes linkages with EMS 32 training and prevention programs to ensure that needs identified through 33 the evaluation process are integrated into EMS training curriculum and 34 prevention efforts. Finally, it provides an organizational structure and 35 standard operating procedures necessary to ensure maintenance of a 36 statewide EMS data collection, evaluation and quality improvement 37 process. 38 39 IV. System Evaluation and Improvement Training 40 2State EMS Data Collection, 10 3Evaluation, and Quality Improvement 4System Overview 1 1 EMSA #167 – State EMS System Evaluation Training Guidelines (To be 2 developed) 3 4 Once completed, the System Evaluation and Improvement Training 5 Guidelines will contain the recommended training modules for all levels of 6 EMS system personnel to ensure proper documentation, data entry, 7 analysis, utilization of data, and an understanding of the principles of 8 quality improvement and research. It is anticipated that there will be a 9 module developed for EMT-Is, EMT-Ps, and MICNs and a separate module 10 for LEMSA and State EMSA staff. Guidelines will also be developed to 11 assist provider agencies in developing agency specific training. 12 13 Ultimately, the curriculum developed for field personnel under this section, 14 should be added to their initial training requirements. 15 16 V. EMS System Research 17 18 EMSA # 168 - EMS Research Guidelines (To be developed) 19 20 Once completed, the EMS System Research component of the California 21 EMS QI Program will provide a response to the State EMS Commission’s 22 frequent expression of the need for guidelines for conducting and funding 23 of State supported, or State required EMS research. The National EMS 24 Research Agenda will be integrated into this process. 25 26 To that end, EMSA #168 should establish mechanisms to: 27  Identify the various types of state required or state funded research 28 that may be conducted to include: trial studies, treatment guideline 29 effectiveness research, EMS system cost evaluation studies, patient 30 outcome studies, etc. 31  Identify the various research design methodologies practical for 32 EMS such as quantitative, qualitative, survey, observation, historical, 33 experimental, evaluation, etc. 34  Identify current requirements and standards for conducting EMS 35 research for each research design methodology identified. 36  Establish a statewide IRB Process 37  Develop criteria for EMS research based upon above standards 38 which take into practical consideration research in rural areas with 39 low study populations. 40  Identify the types of research which should be given priority for state 41 funded research projects. 42  Develop minimum standards criteria for research projects to be 43 funded by the State EMS Authority. 2State EMS Data Collection, 11 3Evaluation, and Quality Improvement 4System Overview 1 1  Develop strategies which would encourage statewide EMS research 2 projects that meet the minimum standards developed above. 3  Identify all current barriers to conducting EMS research in California 4 such as restrictions for prehospital human subject review 5  Identify legislative/regulatory changes in bullet point format required 6 to reduce identified barriers 7  Identify areas of research outside of the realm of EMS that would 8 benefit EMS 9 10 11 Future Needs 12 13 As mentioned before, the Table of Contents includes documents that are not yet 14 completed. The unfinished documents were included in the Table of Contents to 15 ensure that the vision of the comprehensive structure was not lost. As time and 16 funding becomes available, those unfinished sections will be completed. Listed 17 below, is a suggested order of priority for those projects in need of completion. 18 19 1. Identify a standing State EMS System Evaluation and Improvement 20 Oversight Body to coordinate implementation of the data, and QI 21 efforts and coordinate future revisions to the statewide evaluation 22 and improvement process. 23 24 2. Completion of the State Data Collection and Reporting Guidelines 25 (EMSA #165) and submit to the State EMS Commission for adoption. 26 27 3. Implementation of the State EMS System Quality Improvement 28 Program Model Guidelines (EMSA #166) throughout the state. 29 30 4. Completion of the State EMS System Evaluation Training Guidelines 31 (EMSA # 167) to facilitate #1 above. 32 33 5. Conduct testing and validation of the LEMSA Assessment Tool 34 (EMSA #162) and submit final tool to the State EMS Commission for 35 adoption. 36 37 6. Completion of the uncompleted quality indicators (EMSA #163) to 38 ensure a complete and comprehensive list of EMS evaluation 39 indicators. 40 41 7. Completion and full utilization of the statewide CEMSIS data 42 collection and web-based reporting capabilities. 43 2State EMS Data Collection, 12 3Evaluation, and Quality Improvement 4System Overview 1 1 8. Establishment of data linkages between CEMSIS and a) OSHPD 2 Hospital Outcome and E.D. Data, b) DHS Vital Statistics Data, c) 3 CHP/FEDERAL Crash Data. 4 5 9. Establishment of data downloads to the National EMS Information 6 System. 7 8 10. Development of the State EMS Authority Assessment Guidelines 9 (EMSA 161). 10 11 11. Development of EMS Research Guidelines (EMSA 168). 12 13 Conclusion 14 15 By following the guidelines established in this Section, it is anticipated that EMS 16 service providers, LEMSAs, and the State EMS Authority will be able to, for the 17 first time, truly measure the effectiveness and worth of EMS systems in California, 18 and will be able to provide a comparative analysis between systems that was 19 never before possible. 20 21 The documents contained in this section are a best effort by all those involved. It 22 is a starting point. These documents will need to remain dynamic and be revised 23 as we collectively gain experience and knowledge on what constitutes effective 24 evaluation and quality improvement methodologies. 25 26

2State EMS Data Collection, 13 3Evaluation, and Quality Improvement 4System Overview