Methods and Measures Used in Primary Care Patient Safety Research

Beer knowledge for safer care

Methods and Measures used in Primary Care PaƟent Safety Research Results of a literature review

2008 Better Knowledge for Safer Care

Methods and Measures used in Primary Care Patient Safety Research

Results of a literature review

Authors Meredith Makeham, University of Sydney, Australia Susan Dovey, University of Otago, New Zealand William Runciman, Royal Adelaide Hospital, Australia I Larizgoitia, World Health Organization, Switzerland On behalf of the Methods & Measures Working Group of the WHO World Alliance for Patient Safety

Author for Correspondence Susan Dovey Department of General Practice Dunedin School of Medicine University of Otago PO Box 913 Dunedin New Zealand [email protected]

Page 1 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

ABSTRACT

Background and Aims: As most sites of national patient safety organizations and the English- patient safety research to date has language websites of 92 focused on hospital-related issues, we aimed to determine the methods international, national or provincial general practice/family medicine used in patient safety research conducted in primary care, their organizations, and one international and one national physician insurance strengths and weaknesses, the agency. measures they produced, and research gaps. Results: We identified nine

Methods: Review of MEDLINE, retrospective studies, 34 concurrent or prospective or single method in-process and PubMed-not- MEDLINE, OLDMEDLINE, CINAHL studies and six mixed methods studies. The most common method and EMBASE records from 1966 to was analysis of reports of patient 2007. Bibliographies of selected safety incidents made by primary articles were scanned for additional care clinicians, practice staffs, or publications. MeSH terms relating to patient safety, primary care and patients (22 papers). incident reporting were used. We We indentified no primary care excluded studies that examined only patient safety research from one type of patient safety incident or developing countries. No studies only one primary care process, and comparable to hospital-based studies based on hospital data only. retrospective record reviews or We included research using both autopsy research were found. primary care and hospital data and Patients’ perspectives were poorly research about community-based represented. complementary or alternative Estimates of patient safety incidents medicine. We searched the internet in primary care were 0.004-240.0 per Page 2 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

1000 primary care consultations and per 1000 treatments to 4% of 45%-76% of all “errors” were incidents resulting in death, 17%– preventable. Many studies included 39% resulting in harm, and 70%– measures of the relative frequency of 76% had potential for harm. different types of patient safety Conclusions: incident: 26%-57% of incidents Much useful work has been done but involved diagnostic “errors”; 7%-52% the study of patient safety in primary involved treatment; 13%-47% care is still in its infancy. More involved investigations; 9%-56% rigorous methods need to be used involved office administration; 5%- and clearer and more consistent 72% were communication errors. definitions of common terms would Harm from safety incidents ranged assist comparability of results. from 1.3 significant minor incidents

Page 3 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

INTRODUCTION

The internationally agreed definition research in acute hospital settings of primary health care is provided in (focusing on higher income point VI of the Declaration of Alma- countries), 4 and in transitional and Ata. 1 Although each country developing countries. 5 In addition interprets the concept slightly there are papers on the ontology, 6 differently, overall, primary health epistemology, 7 and scope of patient care describes the activity of health safety research 8 that set the context care providers who are the first point for these three reviews. of health system contact for patients To date concern about the safety of and who are based in a community, patients in hospital settings has rather than in a hospital.2 driven most research in the field. The The purpose of this paper is to seminal reports about patient safety critically appraise the methods used in the US 9 and the UK 10 excluded to research patient safety in primary primary care from their discussions. health care studies and the metrics The UK report was specific about (measures) this research uses and this exclusion and it was implicit in produces. This document has been the US report. Hospitals were the developed as part of a of a series of focus of attention and have remained reviews proposed by the World so to date.

Health Organisation (WHO) World A stronger emphasis on primary care Alliance for Patient Safety aiming at patient safety research is important understanding the tools available for because the overwhelming majority conducting research on patient of healthcare is delivered outside 3 safety in various settings. 11 hospitals, in primary care settings. Companion papers review methods Many safety incidents identified in and measures used for patient Page 4 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care hospitals originate elsewhere, often with immediate or close observable in primary care 12-14 and most burden consequences should be defined on health systems arises not from within the safety paradigm but rare mistakes with drastic incidents with delayed effects should consequences, but from the more be regarded as primarily a quality mundane incidents that have effects issue. 6 From a primary care that are magnified by frequent perspective this distinction based on repetitions and exposure of a large time is problematic, but there remain number of people. 15 challenges in identifying and measuring patient safety incidents Primary care may hold different threats to patient safety from hospital that are associated with lengthy latency, and where incomplete settings due to both the health care records may mask complete delivery environment and the type of health services provided. Primary understanding of contributing factors. care providers often have less Further complicating patient safety control over care management and research in primary care are the delivery than in the more characteristics of patients who continuously monitored hospital commonly present in primary care admissions, and more than one site with undifferentiated problems, is often required for an episode of uncertain diagnoses and multiple co- care (having implications for patient morbidities. 16 Appropriate and and information transfer). Primary inappropriate delays in protecting care sites are not necessarily their safety by making a correct designed for this purpose (for diagnosis must be subjectively example: patients’ homes, providers’ assessed. There are also substantial cars, or on roads). As well, episodes challenges in protecting patients’ of primary care may extend over very safety where the systems to support long time frames – sometimes years. safe care may be poorly defined and There is an argument that incidents idiosyncratic. Page 5 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

There is one notable exception to the Indexed Citations”, Ovid's collection overall dearth of patient safety of non-indexed National Library of research in primary care. Because of Medicine records, both the in- the need to test the safety of process and PubMed-not-MEDLINE pharmaceutical products before they records, and OLDMEDLINE (the are released onto the market, and to National Library of Medicine's online regulate their use after release, the database of approximately 1,700,000 pharmacovigilance literature has citations to articles from international generated a great deal of knowledge biomedical journals covering the about drug safety and much of this is fields of medicine, preclinical primary care-based (see, for sciences and allied health sciences). example 17-25 ). This paper reviews the The same Medical Subject Headings methods and measures used for (MeSH terms) relating to patient patient safety research conducted in, safety, primary care and incident about, and for primary care patients reporting (shown in Table 1) were and providers. We excluded papers also used to search the Cumulative concentrating on one particular Index to Nursing and Allied Health primary care process or function Literature (CINAHL) and Excerpta (such as prescribing) and we focus Medica (EMBASE). The reference instead on research aimed at lists of selected articles were investigating the full spectrum of scanned for any additional relevant patient safety issues in primary care. publications.

We excluded articles that did not METHODS report original research, studies that A review of the published scientific examined only one type of patient literature was undertaken using safety incident or only one primary OVID Medline from 1966 to care process, and studies based on December 2007. This database hospital data only. We included includes “In-Process and Other Non- research about patient safety Page 6 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care incidents where both primary care language websites of 92 and hospital data were used and we international, national or provincial included research about community- general practice or family medicine based complementary or alternative organizations, and one international medicine that met other review and one national physician insurance criteria. agency. The search strategy for Medline (repeated for searches of We also searched the internet sites the CINAHL and EMBASE of national patient safety organizations in Australia, Europe databases) is shown in Table 1. and North America and the English-

Table 1. Search strategy used for OVID Medline

#1 Family practice OR primary care OR primary health care OR general practice

#2 Medical error* OR medication error* OR diagnostic error* OR iatrogenic disease OR malpractice OR safety culture OR near failure OR near miss OR patient safety method* OR patient safety indicator* OR patient safety measure* OR patient safety report* OR safety event report* OR safety manage* OR risk manage* OR adverse drug reaction

#3 #1 AND #2

#4 LIMIT: Abstracts

#5 LIMIT: English language

#6 LIMIT: clinical trial OR meta-analysis OR randomized controlled trial OR review OR case report OR classical article OR comparative study OR evaluation studies OR controlled clinical trial

Page 7 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

RESULTS

Search results

The Ovid Medline search returned excluded the peripherally related 1057 titles and abstracts and the papers because their data were CINAHL and EMBASE searches derived hospitals only (8 papers), returned 773 potentially relevant their subject matter was not directly papers. related to patient safety (8 papers),

We reviewed the titles of all papers or they were focused on individual types of safety incident or care and if the title suggested eligibility we process (13 papers). also reviewed the abstract. Titles and abstracts suggested the eligibility of The number of papers reporting 126 papers and we read the full text patient safety research in primary of these. Eliminating duplicates, care increased rapidly over the discussion papers and letters, we period of the review. Figure 1 shows found 49 articles reporting original the publication year of the 49 research of direct relevance to the research reports included in this objectives of this paper, and 29 that review and Table 2 shows the were peripherally related. We research methods used in each of the three general types of research approach.

Page 8 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

Figure 1: Review-eligible papers by publication year

4 number of papers of number 2

0 1995 1997 1998 2002 2003 2004 1 2 32001 4 5 6 72005 8 2006 910 2007 Publication year reported in real-time, the Methods used in primary care research involving databases of patient safety research audits was completed The 49 articles directly related to retrospectively. patient safety research in primary 2. Concurrent or prospective or care were of three main types: single method studies using 1. Retrospective studies, data analyzed either including literature reviews qualitatively or and studies using medical quantitatively.35-68 We records and malpractice included reporting systems databases as their data studies in this category 26-34 sources. We included because, although reported studies of significant event incidents may have happened audits in the retrospective in the past (retrospectively), category because although reports used in these studies the audits themselves were

Page 9 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

were made especially for the patient safety incident happen in research, or in “real-time”. primary care?” 29, 30, 32, 33, 36-39, 41, 42, 46, 3. Mixed methods studies 48-54, 58-63, 65, 66, 68, 70, 71

reporting research that used Reported research was grouped two or more different according to the following generic 69-74 methods. reasons for the study: Within these three main research 1. To establish the types (and approaches, 10 different methods sometimes frequency) of patient were used. Forty-eight studies used safety incidents happening in only one main method but the six primary care. remaining studies used two 69-72, 74 2. To propose and/or test and four 73 different methods. The interventions to make primary most common method was analysis care safer for patients. of reports of patient safety incidents 3. To propose and/or test methods made by primary care clinicians, for patient safety research in practice staffs, or patients 36, 37, 39-42, primary care settings. 45, 48-50, 53, 54, 57-61, 63-66, 68 and the research question most often addressed was: “what types of

Page 10 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

Table 2: Generic research questions addressed in studies using different designs/methods

Research Design

Research Aim Retrospective Concurrent or Mixed methods Prospective To establish the Systematic literature Interview studies 38, 51 Survey + Interview 29, 33 71 types (and/or reviews Surveys 46, 52, 62 study frequency) of patient Studies of malpractice 36, 37, Survey + Systematic Reporting systems 70 safety events claims and risk management 39-42, 48-50, 53, 54, 57-61, 63-66, literature review happening in primary databases 30, 34 68 care To propose and/or Systematic literature Surveys 47, 56 Systematic literature test methods for reviews 26 45 review + Focus group Reporting systems 72 patient safety Studies of Significant Event study research in primary Audit databases 31 Study of Significant care settings Event Audit database + Studies of malpractice 74 claims and risk management Survey databases 32 To propose and/or Studies of significant event Interview studies 35, 44 Delphi study + Interview 27, 28 69 test interventions to audit databases Focus group 43, 55 study make primary care 40, 44, Systematic literature safer for patients Reporting systems 57, 58, 64, 67 review + Interview study + Focus group study 73

A. Strengths and weaknesses of retrospective research methods care patient incidents, followed by another in 2003 33 that aimed to both Retrospective research methods are describe incidents and estimate their generally used to find out what has 70, 72 26, 73 happened in the past in order to plan frequency. In 2006 and 2007 four more literature review papers improvements for the future. were published. Where a literature Systematic reviews of the review is reported in combination literature have been published both with other research methods, it was as a stand-alone research method 26, used in advance of other methods to 29, 33 and in combination with other develop a tentative definition, 70 methods.70, 72, 73 The first literature proposal, 72 or method 73 that was review appeared in 2002 29 aimed at then tested by the other methods. developing a way to describe primary

Page 11 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

The strengths of literature reviews lie Organizations, 70 the National Patient in their being able to summarize Safety Foundation, 29, 33 the Institute existing knowledge and identify for Healthcare Improvement, 29 the knowledge gaps. Results of a American Academy of Family literature review crucially depend on Physicians, 29 the American College the literature being reviewed, the of Physicians-American Society of means by which it is identified, and Internal Medicine, 29 the Institute of how it is interpreted. Medline was Medicine, 29 and the Medical accessed for all primary care patient Protection Society. 33 safety literature reviews although Measures of primary care patient one paper was silent on their search safety incidents from literature 70 strategy. Medline excludes many reviews: All literature reviews journals that publish primary care concentrated on qualitative analyses research so used alone it is unlikely of prior research, producing to provide a complete picture. Most 29, 70 definitions of “medical error” and searches were limited to English “preventable adverse events”, 29 language reports and this would also identification of factors impeding or limit their comprehensiveness. facilitating disclosure of “medical Most literature reviews used more errors”, 72 ways that mortality data are than one citation database, and used in general practice, 26 and a included searches of EMBASE, 26, 29, measure of patient safety culture in 73 CINAHL, 26, 73 the Cochrane primary care organizations. 73 Library, 29 E-PIC (Pharmacy One review found 25 different 73 information), the Health 70 definitions of “medical error”. Management Information Circular Another derived quantitative (HMIC) 73 and the websites or measures of “medical errors” in 70 bibliography collections of WHO, primary care (5-80 per 100,000 the Joint Commission for the consultations), “errors in diagnosis” Accreditation of Healthcare Page 12 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

(26-78% of all “errors”), and These studies were descriptive, “treatment errors” (11-42% of all small, and not designed for “errors”). 33 This study also estimated epidemiological generalizeability: the that 60-83% of all “errors” were 2003 study reviewed only 56 preventable. significant event reports 31 although 337 27 and 662 28 reports were Studies of significant event audits included in the later studies. are limited to the United Kingdom, Significant events described in these where since 2004 the Quality and studies tended to be serious, with Outcomes Framework has rewarded events that may threaten patient general practices for carrying out safety but not pose an immediate analyses of significant occurrences risk to life regarded as not significant (not necessarily involving negative enough to warrant inclusion. A patient outcomes) in an effort to limitation of significant event audit as improve care. One paper was a method for researching patient published shortly before conducting safety in primary care is that so far it such audits was associated with has been reported only in the UK. payment. 31 Three studies used We could find no evidence of its significant event audits alone 27, 28, 31 having been used for research in and in one study a significant event other countries. However, in the UK audit was used in combination with a it is now a compulsory activity for survey. 74 The former three studies general practices and in the future, aimed to describe the content of outcome measures for patient safety general practices’ significant event research may possibly be derived audits while the latter study used the from significant event reports. quality of significant event audits as Furthermore, if other countries adopt an outcome measure for an the same technique, it may provide a intervention aiming to improve risk means of making international management in general practice. comparisons. Page 13 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care

Measures of primary care patient health care providers, 34 and the third safety incidents from significant analyzed 49,345 US primary care event audits: The main measures malpractice claims. 32 This method produced by three studies were was not used in any mixed-method descriptions of the significant events studies. 27, 28, 31 reported in general practice. The study by Fischer et al 30 is the These descriptions grouped events earliest quantitative study of patient according to classifications derived safety incidents in primary care we 39, 53, from reporting system studies. found. Quantitative analyses make 60 Other measures used in significant an important contribution to the field event audit research were reasons of patient safety because they for significant event reports being highlight common problems that can “unsatisfactory” 27, 28 and severity of then be used to prioritize patient safety event outcomes. interventions. The main weakness of Serious or life-threatening events studies involving malpractice claims 28 were 6.5% of reports in one study or risk management databases is and 22% of reports in another. 31 that they provide a limited view of

Studies of malpractice claims and patients’ experiences with patient risk management databases are safety incidents. Most incidents do currently not a mainstream approach not prompt a malpractice claim and many claims do not arise from in primary care patient safety 75 research. We identified three preventable incidents. However, 30, 32, 34 they do give access to data about relevant studies. One was a incidents that patients have found study of incidents reported to a risk management database at one US unsatisfactory and, as in two of the 30 studies we identified, data can be academic medical centre, one was 32, 34 a case series (N = 94) of criminal found for entire countries. Another strength, demonstrated in cases in Hungary that involved the study by Phillips et al, 32 is that by Page 14 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care linking malpractice and other disability (34.5% 30 and 26% 32 of databases, additional information “errors”), and low severity or can be derived about the emotional outcome only (48.3% 30 characteristics of these incidents. and 18% 32 of “errors” ).

Measures of primary care patient B. Strengths and weaknesses safety incidents from malpractice of concurrent and databases: The main measures were prospective research the prevalence of incidents that Concurrent and prospective primary resulted in an injury, potential injury, care patient safety research methods or financial liability (5.4 per 100,000 are generally used to find out what is clinic visits 30 ) and the distributions of currently happening and the incidents of different types. Patient qualitative methods often used in safety incidents described in these concurrent and prospective primary malpractice and risk management care patient safety research are databases were due to treatment particularly important for providing in- (31% 30 ) or medication errors (8% 32 ), depth analyses of why patient safety 30 34 diagnostic mishaps (26%, 29% incidents happen. and 34% 32 ), failure to supervise or monitor (16% 32 ), improper Interview studies have been used performance (15% 32 ), failure or delay as a method for studying patient in referral (4% 32 ), ‘other’ errors safety in primary care for more than (26% 30 ) or no error, such as known a decade. They have been used as 35, 38, 44, 51 complications (17% 30 ). both a stand-alone method and in mixed-methods studies. 69, 71, 73 Severity of outcome was measured They have been used to describe as death (3.4% 30 and 37% 32 of patient safety incidents in primary “errors”), severe or permanent care 38, 51, 71 and to develop ways to disability (13.8% 30 and 19% 32 of make primary care safer. 35, 44, 69 “errors”), moderate or temporary They have involved primary care Page 15 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care clinicans 35, 44, 69, 71, 73 and trainees, 38 assessing severity of harm, and academics, 35 managers, 69, 73 support estimating incidence that are not personnel, 35, 69 policy-makers, 35, 69 assessable from other studies hospital staff 69 and patients. 35, 51, 69 producing these measures. The Methods used to enroll study privacy of the interview allows participants included “snowballing”, 35 exploration of topics such as anxiety purposive sampling of physicians, and guilt about incidents. Interview patients, practices, or studies can be economical because organizations, 38, 69, 71, 73 and random participants are ideally purposively sampling from physician 44 and sampled, ensuring that every general 51 populations. These studies interview makes a meaningful used interviews lasting between 25 contribution to the study’s goals. minutes 44 and 2 hours. 35 Most Random sampling, used in two of the interviews were conducted according identified studies, 44, 51 is often to an interview guide and considered wasteful in qualitative recorded, 35, 51, 69, 71, 73 and the research because it may cause verbatim transcripts were analyzed some unnecessary interviews to be together with field notes. 35, 73 conducted at the cost of potentially useful interviews not being carried Interview studies and other out. qualitative research techniques are Conversely, interview studies (and especially valuable for their ability to derive new information that have not other qualitative research 76 approaches) are sometimes been anticipated by researchers. considered uneconomical, because Data from individual interviews provides information on non-factual they are labour-intensive and time- consuming. They are difficult to data that is difficult to obtain by other means. One study highlighted integrate into regular routines for patient safety research, although difficulties with classifying errors, some reporting systems include Page 16 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care capacity for interviews in their physician-related errors (8%), patient confidential reporting processes. 77 communication errors (5%), and Some participants may also be preventable adverse events (4%)) 71 reluctant to fully disclose some or encountered by trainee doctors issues as there is no anonymity of (shortfalls in interpersonal skills, participants from investigators. diagnostic skills, and management Recall bias also plays an important skills). 38 Prevalence of observed part in colouring the discussions held “errors” was 24% of consultations in during interviews. Interview studies primary care office practice (3% to tend to be less familiar to healthcare 60% of encounters per physician). 71 planners and providers than Two studies identified the type of quantitative or epidemiological safety incident patients are research and may sometimes be concerned about. 51, 69 These were less valued because of the inevitably access restriction (29% of “problem small numbers of study participants incidents”), communication (relative to the large numbers of breakdown, relationship failure participants possible in some study (37%), technical error and designs – especially database inefficiency (24%) 51 and issues studies), the usual lack of random relating to the interface between sampling and the contextual primary and hospital care. 69 specificity of study groups. Consequences of observed “errors” Measures of primary care patient in office practice included “harm” safety incidents from interview (24% of consultations) and “potential studies: Measures produced by harm” (70% of consultations). 71 interview studies included the type of Patients spoke of anger, frustration, safety incident primary care belittlement, and loss of relationship physicians notice while seeing with and trust in their physician as patients in their offices (office consequences of safety events. 51 administration errors (17%), Consequences of patient safety Page 17 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care events that were memorable to The strengths and weaknesses of family physicians included patient focus group studies are similar to death (47% of memorable events), those of interview studies. Personally no adverse outcome (26%) and sensitive data are less likely to be malpractice suits (4 of 53 “errors”). 44 divulged in focus groups than in One study identified deficiencies in personal interviews, but focus groups computer systems, focusing on drug take advantage of group dynamics to alerts, and proposed ways to rectify spark new ideas that may be less these deficiencies 35 and another likely to arise from individual used interviews to test theories about interviews. They are therefore an patient safety culture in the process ideal method for exploring factors of developing an instrument to contributing to patient safety measure safety culture in general incidents because they promote practices. 73 discussion among group members,

Focus group studies were reported who are usually chosen because in two studies as a stand-alone they share common experiences. method 43, 55 and in another two Measures of primary care patient alongside other methods. 72, 73 The safety incidents from focus group studies reported on three 43 to studies: Patients identified issues in fourteen 73 focus groups involving primary care that were classified as 21 43 to 38 55 participants. In both relating to both quality (access to mixed methods studies the focus care, coordination of care, system group component was the final part, resources, and ability to pay) and carried out to determine whether the safety (“errors”). 43 “Errors” were tool developed by other methods classified as medication errors, would be useful. All studies analyzed errors of inattention, or technical recorded discussions and field notes. errors. One multi-method study produced a tool describing factors facilitating physician disclosure of Page 18 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care patient safety incidents patient safety risks to patients with (responsibilities to patients, the COPD. A Failure Modes and Effects profession, self, and to the Analysis (FMEA) 79 was tested. community) and barriers to Measures of primary care patient disclosure (attitudinal barriers, safety incidents from the Delphi helplessness, uncertainty, and fears study: Patient safety risks ranked and anxieties). 72 No new measures most important were “routine came from the other two focus group difficulties with access to patient investigations although the overall records post-discharge leads to product of one study was the decisions being made without Manchester Patient Safety adequate background information” Assessment Framework, 78 a and “information about discharged framework for exploring ways of patients sometimes does not reach improving patient safety culture in relevant primary care staff”. 69 primary care teams. Surveys were used as the only A Delphi study was reported in one 46, 47, research method in six papers paper as part of a mixed-method 52, 56, 62, 67 and as a complementary study aimed at testing a method to method in a further three papers. 70, research patient safety events 71, 74 Participants in these studies occurring at the hospital-primary care 47 included random and non- 69 interface. The Delphi component 56, 62, 70, 71 random samples of primary followed an interview study phase 46, 47, 52, 56, 62 care clinicians and that identified quality of care and staff, 46, 52, 62 and complementary care patient safety issues associated with 53 providers. Response rates were the total healthcare of patients with reported in five papers and ranged Chronic Obstructive Pulmonary 70 56 56 from 29% to 76%. Only one had Disease (COPD). A two-stage a response rate greater than 50%. process was used to identify specific Surveys were used to describe

Page 19 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care patient safety events in primary it has rarely been used in primary care, 46, 52, 62, 70, 71 and they care settings because it is a labour- contributed to the development of intensive process that provides both interventions to improve information specific to the institution safety, 67 and research tools. 47, 56, 74 in which it is conducted. Primary care practices are in general too small to Surveys are a strong research support the infrastructure needed to design for estimating prevalence (for conduct FMEAs. The technique example, prevalence of safety events involves studying one task in detail, in primary care) but to be effective in identifying steps where failure might this function survey participants must occur and designing interventions to be a randomly selected sample of avoid failure at these points. sufficient size to produce results that are generalizeable to the population Measures of primary care patient from which the sample was drawn. safety incidents from survey studies: Only one of the studies in this group Despite the above design concerns, used a random sample, 47 although the surveys reviewed produced this design strategy appears to have measures of factors contributing to been possible for at least two deaths among primary care patients others. 52, 56 Response rates were (patient behaviors (40% of deaths), also very low (compromising general practice teams (5%), generalizeability of results) and only hospitals (6%), and the environment one study reported efforts to improve (3%)), 46, 52, 62 the type and frequency response rates by follow-up of initial of adverse events encountered by non-responders. 56 patients of acupuncturists, 53 factors influencing clinical educators’ One of these surveys was used to responses to “medical errors” develop an FMEA. 62 Although this is (trainees’ prior history, clinical a relatively common approach to use knowledge levels, receptivity to on safety data collected in hospitals, Page 20 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care feedback, training level, emotional primary care, but we excluded these reaction, and whether they studies from the current review. apologized or offered an excuse), 55, Participants in reporting system 67 and attitudes to reporting studies included primary care significant patient safety events (18% 36, 37, 39, 41, 42, 45, 48-50, 53, 54, 57-61, doctors, 47, 56 favored mandatory reporting, 63-66, 68 practice staff 57, 58, 60, 61, 63, 65, 66 70 47, 56 6% -41% had difficulty defining 58 and patients. Reporting systems a significant patient safety event). have been designed for anonymity Other surveys did not aim to produce (where reporters can never be any outcome measures other than a 39, 53, 58 identified), and confidentiality contribution to a larger research (where reporters can be identified for study. 47, 56 46, 52, 62 as long as it takes to correctly record 45, 50, 65 Patient Safety Incident Reporting the event) and have used 36, 37, System studies dominate the (alone or in combination) paper, 39, 58, 60 39, 58, 59, 65, 68 research on patient safety in primary electronic, and telephone reporting.66 They have care settings. We defined surveys as 54, 57, 60, 65, 66 restricted pieces of research, in involved regional, national, 39, 58, 61 and international 53, 59, contrast to reporting systems 64, 68 participants. studies, where the method of data collection was intended to generate The earliest patient safety event research data to address a number reporting study was the Australian of different questions. Sometimes study of Incident Monitoring in more than one included study was General Practice, involving data produced by a single patient safety collected between 1993 and 1995. 36, event reporting system. Reporting 37 In 2004 the UK government- systems have also been used to sponsored National Patient Safety study individual processes used in Agency opened an anonymous online web-based reporting route for

Page 21 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care any healthcare staff. This system can paper has been published in be viewed at: German. 80 http://www.npsa.nhs.uk/health/ The main strengths of reporting reporting/reportanincident . Shaw et systems studies are that they give a al 62 reported an analysis of 28,998 robust indication of the types of safety incidents reported to this patient safety incidents observed by system from 18 NHS Trusts healthcare providers and they are a (including one primary care Trust). well-established method in the One study was designed as a patient safety literature, as reporting randomized controlled trial that systems are embedded in the aimed to compare paper and institutional processes of many computer reporting of patient safety hospital systems. The problem with events but has only ever been patient safety incident reporting published as a reporting system systems for primary care research is study. 40 Along with many other that many primary care providers 36, 37, 54, 58, 60, reporting system studies work outside the organized systems 61, 63 it aimed to develop a way to with established incident reporting. In describe the patient safety events many hospitals reporting systems encountered in primary care. There are part of continuous quality was one international study that improvement processes and data are involved participation from primary routinely collected that can later be care doctors in Australia, Canada, used to address specific research Germany, the Netherlands, New questions. Primary care practices are Zealand, the United Kingdom and usually much smaller institutions and the United States. Only the English- unable to support either the routine language papers from this reporting collection of patient safety data or system study are included in this the infrastructure to use these data review 48, 53, 59, 64, 68 but an additional for research. Therefore, most of the studies we found related to reporting Page 22 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care systems set up in universities, suited for calculating epidemiological specifically for research. Report statistics (such as incident providers contributed their data on prevalence). As well, data from the understanding that they were reporting systems are difficult to participating in a time-limited generalize because contributors are research project, rather than seldom statistically representative. engaging in an ongoing quality We identified only one study where improvement programme. This representativeness was a concern means that most studies were and a random selection of doctors relatively small and the reporting contributed to the study. 55 A further systems had no long-term life limitation of reporting systems is that expectancy. The exception is the over time they accumulate massive UK’s national reporting system, amounts of complex data that can be which is available to all healthcare very difficult to extract meaningful providers, including those working in information from. So far this has not primary care. To date, contributions been a problem for primary care to the system from primary care have reporting systems because they been very small relative to the have been specifically designed for contribution from hospitals but this research. As incident reporting may change now that the patient becomes a routine activity for safety agenda is moving to primary care providers this issue is incorporate primary care. likely to become increasingly relevant. A well-recognised, important, and inevitable limitation of reporting Measures of primary care patient systems is under-reporting. 81, 82 safety incidents from reporting Runciman et al have estimated that system studies: Most reporting as few as 5% of incidents are notified system studies developed a way to to reporting systems. 83 They cannot, describe the patient safety incidents in general, be treated as databases reported, often in an hierarchical Page 23 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care taxonomy and according to www.cudfm.org/carenet/asips/taxono categories such as: office my . Other ways of classifying administration (between 15% 63 and reported events were: adverse 31% 39 of reports) including events (21% 51 of reports) and near appointments (2% 63 to 14% 58 of misses (64% 51 ). One paper reported reports), investigations (6% 58 to remedial strategies.64 33% 67 of reports), treatments The “error” report rate was calculated 61 36, (including medication (8% to 52% 60 at 75.6 per 1000 appointments and 37 58 of reports), communication (4% 2 per 1000 patients seen per year. 55 to 80% 68 of reports), payment mistakes, clinical mistakes (3% 60 to Strengths and weaknesses of 10% 63 of reports), wrong diagnosis mixed-methods research 39 36 (4% to 34% of reports), wrong Mixed-methods studies are where a treatment decisions, and equipment single main research aim is 36, 37 61 (5% to 16% of reports). Causes addressed progressively using or contributing factors (work different research methods to either organization, excessive task develop the tools to answer the demands, and fragmentation), research question definitively, or to prevention strategies and build different perspectives to a 50 consequences (harm (17% to research issue by approaching it in 68 43% of reports) and potential for different ways. This latter approach 36 50 serious harm (27% to 76% of is sometimes called “triangulation”. reports)) and other consequences We identified six mixed-methods 37 and contributing factors are studies in this review. They methods sometimes also classified. Some of they used included combinations of these descriptions have been surveys, 70, 71, 74 interview published electronically: studies, 69,71,73 systematic literature www.errorsinmedicine.net/ reviews, 70,72,73 focus group taxonomy/aafp and studies, 72,73 significant event audits, 74 Page 24 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

Better Knowledge for Safer Care and a Delphi study. 69 Four studies data collected from interviews and a took the approach of successive tool Delphi process 69 and concentrated development 69,70,72,73 and the other on in-depth analyses that produced two adopted a triangulation outcomes with practical application. approach. 71,74 Measures of primary care patient safety incidents derived from mixed- Each mixed-method study incorporates the strengths and method studies are reported above, as part of the review of their weaknesses of their individual individual methods. methods (as above). Additionally, however, they develop the science of Overview patient safety research by creating Table 3 in the Appendix summarizes new multi-faceted processes, such the retrospective, concurrent or as “care process mapping”. 69 Care prospective, and mixed-method process mapping identified key care primary care patient safety studies decisions on the care pathway (from included in this review. Their aims, primary care to hospital and back to methods, measures, and high-level primary care), aiming to identify and conclusions are shown with a note of remedy processes and problems that the design limitations of the study. adversely affected patient safety. This analytic method was applied to of patient safety events in primary DISCUSSION care have been developed.

Compared to hospital-based Methods used in primary care research, qualitative methods for patient safety research: researching safety incidents in We identified research that used one primary care are relatively common, of three general methodological the body of quantitative research is immature, and few robust measures approaches: retrospective,

Page 25 of 49

Methods and Measures used in Primary Care Patient Safety Research Makeham, Dovey et al

concurrent or prospective, and safety incident might be spread over mixed-methods. Retrospective several different health care methods included systematic providers in different locations - literature reviews 26, 29, 33 and unlike a hospital record, where retrospective analyses of significant multiple providers contribute to a incident, 27, 28, 31 risk management, single set of patient notes. However, and malpractice claims 30, 32, 34 a study of this type has the potential databases. Concurrent and to draw some comparisons with prospective methods included the hospital-based studies using this qualitative approaches of method and may warrant further interviews,35, 38, 44, 51 focus groups,43, exploration. We also found no 55 and safety incident reporting 36, 37, primary care research comparable to 39-42, 45, 48-50, 53, 54, 57-61, 63-68 and the the hospital-based research that quantitative approach of surveys. 46, uses autopsy reports, 86-88 although 47, 52, 56, 62 Mixed-method studies mortality databases have been used combinations of two, 69-72, 74 or identified as a potentially important three 73 of these methods. source of safety information and one that primary care researchers are Although almost all of the methods 26 employed in hospital-based research prepared to use. have also been used in primary care, There is a dearth of research on the some are missing from the primary types of methods that may better care literature. No studies were engage patients in safety research in found that were directly comparable primary care, and the value of their to the retrospective record reviews input in addressing different aspects used in hospital-based patient safety of patient safety is poorly studies 84, 85 and the measures that understood. Patients have engaged might arise from such reviews. successfully in qualitative research in Numerous barriers to using this community settings about perceived method could apply in primary care harm, 51 but not in other types of settings, where a record of the patient safety research.58 This should breakdown in care that leads to a

be considered in future research included measures of the relative planning. frequency of different types of patient safety incident. Most commonly Measures used in primary care these were “errors” in: diagnosis patient safety research: (26%30 -57%44 of all incidents), The terms most often used to treatment (7%43 -52%37 of all describe patient safety incidents in incidents), investigating (13% 53 - primary care settings were “error”, 47%45 of incidents), office “medical error” and “preventable administration (9% 64 -56% 58 ), and adverse events”. There was no communication (5%71 -72%65 of standard definition of these terms. incidents). Two studies 43, 51 One literature review found 25 investigated the type of safety different definitions of “medical incident patients are concerned error”. 70 about. These were mainly Estimates of the rate of patient safety relationship (37% 43 -77% 51 of incidents occurring in primary care concerns) and access problems varied enormously, ranging from (29% 51 of concerns). 0.004 30 to 240 71 per 1000 primary Causes or contributing factors to care consultations. Estimates of patient safety incidents included: preventability ranged from 45% 71 - environmental hazards (3% 46 -14% 30 ) 76%36, 37 of all “errors”. including work organization, The types of participants involved physician factors (5% 46 -91% 44 ) primary care patient safety incidents including excessive task demands were primary care physicians and and fragmentation, patient factors 29, 30, 32, 34-42, 44-46, 48, 50, 53-62, 64- trainees (40% 46 -72% 44 ), and hospital factors 66, 68-73, 89, 90 43, 51 patients, nurses and (6% 46 ). other practice staff, 45, 47, 57, 58, 65, 66, 69, Reported harm arising from patient 73 paramedics, 34 pharmacists, 34 safety incidents ranged from 1.3 computer suppliers, 35 academics, 35 significant minor incidents per 1000 acupuncturists, 52 optometrists, 63 and treatments 52 to 4% of incidents managers. 35, 69 Many studies resulting in death, 36, 37 17% 50 –39% 42

of incidents resulting in harm, and Patient Safety Assessment 70% 71 –76% 50 of incidents having Framework, 78 a tool for developing potential for harm. Patients spoke of improvements in patient safety anger, frustration, belittlement, and culture in primary care teams. loss of relationship with and trust in Limitations of this review: their physician as consequences of safety incidents. Consequences of The primary care patient safety patient safety incidents that were research in this review generally memorable to family physicians 44 spans little more than a decade, included patient death (47% of although there is an older and more memorable incidents), no adverse extensive literature interpretable as outcome (26%) and malpractice suits relevant to patient safety in bounded (8%). areas such as medications use and diagnosis. The scope of the current Factors influencing clinical research did not include an analysis educators’ responses to “medical of these specific safety topics, which errors” were trainees’ prior history, may provide further insight into clinical knowledge levels, receptivity methods and measures of relevance to feedback, training level, emotional to patient safety incidents in reaction, apologizing, and offering an community settings. We included excuse. 55 A taxonomy of factors only research publications enabling and inhibiting voluntary investigating a wide view of primary disclosure of “errors” was also healthcare activity and its risks for developed. 72 Attitudes to reporting patient safety. We excluded studies significant patient safety incidents focused on a single bounded activity were measured in one study: 56 18% or cluster of activities, such as favored mandatory reporting. medications use or diagnosis. There Other studies did not aim to produce are many studies in both of these any outcome measures other than a general areas, usually focusing on contribution to a larger research particular drugs or drug classes and study. 47, 73 They contributed to particular diagnoses. Because of the products such as the Manchester

broad scope of primary care, it was to develop new methods and this not possible to review this literature. process is already starting. We We included patient safety research included in this review some papers at the interface between hospital and that used the approaches of hazards primary care only if it involved data or Failure Modes and Effects derived from both settings. We Analysis in primary care. 62, 66, 69 included only English-language These new methods have recently papers. For these reasons it is started to be reported in the unlikely that we have identified all literature. However, there is also a relevant literature but our search need to use older methods (such as methods have likely captured the survey research) with greater essence of the literature as a whole. attention to designing in research elements to enhance scientific This review appraised research conducted mainly in the USA, the robustness. Greater use of random samples and more attention to UK, and Australia. We caution that because the overwhelming body of increasing response rates are obvious early targets for improving published research about patient this type of research. All measures of safety in primary care comes from this limited set of countries, it is primary care safety incidents identified in this review require unlikely to address issues of importance to many other countries – further refinement in other primary care settings to test their reliability especially in the developing world. and validity. More appropriate methods and measures need to be found for In the absence of a definitive and patient safety research in a wider internationally understood set of range of countries. terms and definitions the need to create classification systems, Recommendations for further descriptions, and definitions has research: been compelling for researchers of It is becoming clear that patient patient safety in primary care safety research in all settings needs settings. Point estimates were

calculated for many types of safety patient safety research. We incident but the difficulty with recommend that such research measurement is reflected in the fact should be on the agendas of all that these estimates varied by more countries because of the likely than 1000-fold between studies. This opportunities for improvement of may be due to different research patient safety in the most widely data and methods, or different used sector of any health system. interpretations of the same terms (in Conclusions: two studies 6% 70 and 41% 56 of participants had difficulty defining a Primary care patient safety research “significant patient safety event”). is at an early stage of development, Several studies grouped safety with research efforts concentrating incidents according to classifications on describing the safety environment derived from incident reporting rather than intervening to improve it. systems.39, 53, 60 The WHO’s As recently as five years ago, International Classification for Patient primary health care providers were Safety (beta version released July more or less exempt from 2007) 91 is an international tool considerations about patient safety designed to facilitate understanding and they were excluded from the about patient safety across health seminal patient safety reports from 9 10 sectors and between countries. the US and the UK. Since that Refinement and use of this time there has been a growing classification system may obviate the recognition of the increasingly urgent need for further development of need to reduce patient safety threats taxonomies in primary care patient in primary care settings. safety research. The methods of primary care patient While Australia, the US, and the UK safety research are well recognized have started to build research and replicable so it is likely that they programs about patient safety in will become more widely used, primary care settings, most countries refined, and ultimately deliver more have not yet engaged in primary care useful knowledge than is currently

available. The methods tend to be Only a small amount of research has mainly pragmatic, dominated by investigated patient safety in primary studies using reporting systems that care from the perspective of patients. have been set up specifically for To date patients’ views have been research purposes. These studies heard only through small-scale have not yet resulted in national qualitative studies or in the analysis patient safety strategies appropriate of complaints and risk management for primary care. However, they systems. An early challenge to provide a form of anticipatory testing address is how to incorporate and show that primary health care patients’ perspectives on patient providers are generally receptive to safety using valid methods that are the idea of identifying and rectifying devoid of medico-legal threats to risks to patient safety. An integrated clinicians. The sustainable methods information and incident of reporting that have identified management system is probably threats to patient safety from ideal for managing threats to patient providers’ perspectives have been safety in both primary and other far less successful in eliciting health care settings. 92 To develop patients’ experiences of patient the study, measurement, and safety threats. There is a need to improvement of patient safety in develop methods that allow patients primary care settings, there is a more voice in researching the patient pressing need to address the rigor safety agenda in primary care (and with which research is designed in other) settings. Involving patients in order to make their results this type of research is likely to result generalizeable. Researchers need to in measures of patient safety that are consider methods that will address different from the current metrics, all the internal validity of the measures of which are focused on the provider produced by their research, as well perspective. as maintaining the current concern Measures of primary care patient for external validity. safety are still under development and there are no agreed outcome

measures of “safer” care. Identifying consider ways to engage a broader and measuring harms related to range of communities and health primary care patient safety incidents care settings, including developing is a research gap. Some harms such countries and different cultural as death may be applicable across groups. health care settings but others, such Much useful work has been done but as wrong side surgery, are not the study of patient safety in primary relevant to primary care research. care is still in its infancy. Barriers to healthcare access, extended waiting times and emotional disaffection, generally not considered serious harms in hospital-based research, may turn out to be important outcomes of patient safety incidents in primary care because of their long-term consequences in terms of reducing trust in the health system, consequent low use of preventive care and resultant higher need for emergency and acute care. The debate currently is whether these outcomes relate to quality or safety. More research is needed.

Relatively few countries appear to be engaged in primary care patient safety research. This review shows the dominantly western nature of the published scientific literature. Attempts to increase the efforts at an international level should ideally

Acknowledgements

The authors wish to thank Dr David Bates of the Division of General Internal Medicine, Brigham and Women’s Hospital and External Research Lead of the WHO World Alliance for Patient Safety for his contribution and leadership. The members of the Methods & Measurement working group of the WHO World Alliance for Patient Safety are: Ross Baker, William B Runciman, Carlos Aibar, Susan Dovey, Rhona Flin, Richard Lilford, Philippe Michel, Santawat Asavaroengchai, Claudia Travassos, and William Weeks.

Health System . 1999, National REFERENCES Academy Press: Washington 1. Declaration of Alma-Ata . in DC. International Conference on Primary Health Care . 1978. 10. An organisation with a Alma-Ata, USSR: World memory: Report of an expert Health Organization. group on learning from adverse events in the NHS 2. Starfield B, Primary Care: chaired by the Chief Medical Balancing Health Needs, Officer . 2000, London: The Services and Technology . Stationery Office. 1998, New York: Oxford University Press. 11. Green L, Fryer G, Yawn B, Lanier D, Dovey S. The 3. Michel P, Strengths and ecology of medical care weaknesses of available revisited . N Engl J Med 2001; methods for assessing the 344(26): 2021-5. nature and scale of harm caused by the health system: 12. Woods D, Thomas E, Holl J, literature review . 2006, WHO: Weiss K, Brennan T. Geneva. Ambulatory care adverse events and preventable 4. Jeffs L, Law M, Baker G. adverse events leading to a Patient safety measures in hospital admission . Qual Saf acute care settings . In Health Care 2007; 16(2): 127- preparation. 31.

5. Michel P,Wilson R. Adverse 13. Davis P, Lay-Yee R, Briant R, events in developing Ali W, et al. Adverse events in countries . In preparation. New Zealand public hospitals II: preventability and clinical 6. Runciman W, Baker G, Michel context . NZ Med J 2003 P, Dovey S, et al. Patient 116(1183): U624. safety: tracing the ontology . In preparation 2008. 14. Davis P, Lay-Yee R, Briant R, Scott A. Preventable in- 7. Runciman W, Thomson R, hospital medical injury under Hibbert P, Baker G, et al. The the "no fault" system in New epistemology of patient safety Zealand . NZ Med J 2003; research . In Preparation 2008. 12(4): 251-256.

8. Runciman W, Baker R, Michel 15. Runciman W, Edmonds M, P. Patient Safety Research - Pradhan M. Setting priorities Scope . In Preparation 2000. for patient safety . Qual Saf Health Care 2002; 11(3): 224- 9. Kohn L, Corrigan J, 9. Donaldson M, eds. To Err Is Human: Building a Safer

16. The future of general practice: 23. Schadlich P. [Quality of event a statement by the Royal data in detection of unwanted College of General drug side-effects in general Practitioners . 2004, Royal practice of established College of General physicians] . Practitioners: London. Gesundheitswesen 1993; 55(Suppl 1): 8-12. 17. Haaijer-Ruskamp F,deJong- vandenBerg L. Drug utilization 24. TorelloLserte J, Ferrando JC, studies and drug monitoring in Lainez M, Morillas MG, The Netherlands . Annali Gonzalez AA. [Adverse Dell'Istituto Superiore di reactions to drugs reported by Sanita 1991; 27(2): 217-23. the primary care physicians of Andalusia. Analysis of 18. Hall G, Luscombe D, Walker underreporting] . Atencion S. Post marketing surveillance Primaria 1994; 13(6): 307-11. using a computerised general practice data base . 25. Wood L,Martinez C. The Pharmaceut Med 1988; 2: general practice research 345-51. database: role in pharmacovigilance . Drug Saf 19. Inman W. Postmarketing 2004; 27(12): 871-81. surveillance of adverse drug reactions in general practice. 26. Baker R, Sullivan E, I: search for new methods . Br Camosso-Stefinovic R, Med J (Clin Res Ed) 1981; Rashid A, et al. Making use of 282(6270): 1131-2. mortality data to improve quality and safety in general 20. Kelleher H,Carmichael A. practice: a review of current Yellow card reporting . Br J approaches . Qual Saf Health Gen Pract 1993; 43(366): 35. Care 2007; 16(2): 84-9.

21. Lawson D. Postmarketing 27. Bowie P, McCoy S, McKay J, surveillance versus clinical Lough M. Learning issues trials: which benefits the raised by the educational peer patient? Cardiol 1994; 85 review of significant event (Suppl 1): 18-23. analyses in general practice. Qual Prim Care ; 13(2): 75-83 . 22. Perrio M, Wilton L, Shakir S. A Qual Prim Care 2005; 13(2): modified prescription-event 75-83. monitoring study to assess the introduction of Seretide 28. Cox S,Holden J. A Evohaler in England: an retrospective review of example of studying risk significant events reported in monitoring in one district in 2004-2005 . Br J pharmacovigilance . Drug Saf Gen Pract 2007; 57(542): 2007; 30(8): 681-95. 732-6.

29. Elder N,Dovey S. general practice computer Classification of medical systems for patient safety: errors and preventable Qualitative study of key adverse events in primary stakeholders . Qual Saf Health care: A synthesis of the Care 2007; 16: 28-33. literature . J Fam Pract 2002; 51(11): 927-32. 36. Bhasale A, Miller G, Britt H. Analysing potential harm in 30. Fischer G, Fetters M, Munro Australian general practice . A, Goldman E. Adverse Med J Aust 1998; 169(2): 73- events in primary care 6. identified from a risk- management database . J 37. Britt H, Miller G, Steven I, Fam Pract 1997; 45(1): 40-6. Howarth G, et al. Collecting data on potentially harmful 31. Murie J,McGhee C. Assessing events: a method for risk by analysing significant monitoring incidents in events in primary care . Qual general practice . Fam Pract Prim Care 2003; 11(3): 205- 1997; 14(2): 101-6. 10. 38. Diamond M, Kamien M, Sim 32. Phillips R, Bartholomew L, M, Davis J. A critical incident Dovey S, Fryer G, et al. study of general practice Learning from malpractice trainees in their basic general claims about negligent, practice term . Med J Aust adverse events in primary 1995; 162(6): 321-4. care in the United States . Qual Saf Health Care 2004; 39. Dovey S, Meyers D, Phillips 13(2): 121-6. R, Green L, et al. A preliminary taxonomy of 33. Sandars J,Esmail A. The medical errors in family frequency and nature of practice . Qual Saf Health medical error in primary care: Care 2002; 11(3): 233-8. understanding the diversity across studies . Fam Pract 40. Dovey S, Phillips R, Green L, 2003; 20(3): 231-6. Fryer G. Family physicians' solutions to common medical 34. Varga T, Szabo A, Dosa A, errors . Am Fam Physician Bartha F. Criminal liability of 2003; 67(6): 1168. physicians and other health care professionals in Hungary 41. Dovey S, Phillips R, Green L, (review of case law between Fryer G. Types of medical 1996-2000) . Med Law 2006; errors commonly reported by 25(4): 593-9. family physicians . Am Fam Physician 2003; 67(4): 697. 35. Avery A, Savelyich B, Sheikh A, Morris C, et al. Improving

42. Dovey S, Phillips R, Green L, and validating a Canadian Fryer G. Consequences of taxonomy of errors . Can Fam medical errors observed by Physician 2007; 53: 270-6. family physicians . Am Fam Physician 2003; 67(5): 915. 49. Kostopoulo O. From cognition to the system: Developing a 43. Dowell D, Baier-Manwell L, multilevel taxonomy of patient Maguire A, An P, et al. Urban safety in general practice . outpatient views on quality Ergonomics 2006; 49(5-6): and safety in primary care . 486-502. Healthcare Quart 2005; 8(2): 2-8. 50. Kostopoulo O,Delaney B. Confidential reporting of 44. Ely J, Levinson W, Elder N, patient safety events in Mainous A, Vinson D. primary care: Results from a Perceived causes of family multilevel classification of physicians’ errors . J Fam cognitive and system factors . Pract 1995; 40(4): 337-44. Qual Saf Health Care 2007; 16(2): 95-100. 45. Fernald D, Pace W, Harris D, West D, et al. Event reporting 51. Kuzel A, Woolf S, Gilchrist V, to a primary care patient Engel J, et al. Patient reports safety reporting system: A of preventable problems and report from the ASIPS harms in primary care . Ann collaborative . Ann Fam Med Fam Med 2004; 2(4): 333-40. 2004; 2(4): 327-32. 52. MacPherson H, Thomas K, 46. Holden J, O'Donnell S, Walters S, Fitter M. A Brindley J, Miles L. Analysis of prospective survey of adverse 1263 deaths in four general events and treatment practices. Br J Gen Pract . Br J reactions following 34,000 Gen Pract 1998; 48(432): consultations with 1409-12. professional acupuncturists. Acupuncture Med 2001; 19(2): 47. Hutchinson A, Cooper K, 93-102 . Acupuncture Med Dean J, McIntosh A, et al. Use 2001; 19(2): 93-102. of a safety climate questionnaire in UK health 53. Makeham M, Dovey S, care: factor structure, County M, Kidd M. An reliability and usability . Qual international taxonomy for Saf Health Care 2006; 15(5): reporting general practice 347-53. error in Australia and five other countries . Med J Aust 48. Jacobs S, O'Beirne M, 2002; 177: 68-72. Derflingher L-P, Vlach L, et al. Errors and adverse events in 54. Makeham M, Kidd M, Saltman family medicine: Developing D, Mira M, et al. The Threats

to Australian Patient Safety a method for detecting errors . (TAPS) study: Incidence of Qual Saf Health Care 2003; reported errors in general 12(6): 443-7. practice . Med J Aust 2006; 185(2): 95-8. 61. Shaw R, Drever F, Hughes H, Osborn S, Williams S. 55. Mazor K, Fischer M, Haley H, Adverse events and near miss Hatem D, Quirk M. Teaching reporting in the NHS . Qual Saf and medical errors: Primary Health Care 2005; 14(4): 279- care preceptors’ views . Med 83. Ed 2005; 39(10): 982-90. 62. Singh R, Singh A, Taylor J, 56. McKay J, Bowie P, Murray L, Resenthal T. Building learning Lough M. Attitudes to the practices with self-empowered identification and reporting of teams for improving patient significant events in general safety. J Health Manage ; practice . Clin Govern 2004; 8(1): 91-118 . J Health Manag 9(2): 96-100. 2006; 8(1): 91-118.

57. Parnes B, Fernald D, Quintela 63. Steele C, Rubin G, Fraser S. J, Araya-Guerra R, et al. Error classification in Stopping the error cascade: A community optometric report on ameliorators from practice - a pilot project . the ASIPS collaborative . Qual Ophthal Physiol Opt 2006; 26: Saf Health Care 2007; 16: 12- 106-10. 16. 64. Tilyard M, Dovey S, Hall K. 58. Phillips R, Dovey S, Graham Avoiding and fixing medical D, Elder N, Hickner J. errors in general practice: Learning from different lenses: Prevention strategies reported Reports of medical errors in in the Linnaeus primary care by clinicians, Collaboration’s Primary Care staff, and patients . J Patient International Study of Medical Saf 2006; 2(3): 140-6. Errors . NZ Med J 2005; 118(1208). 59. Rosser W, Dovey S, Bordman R, White D, et al. Medical 65. Westfall J, Fernald D, Stanton errors in primary care: Results E, VanVorst R, et al. Applied of an international study of strategies for improving family practice . Can Fam patient safety: A Physician 2005; 51: 386-12. comprehensive process to improve care in rural and 60. Rubin G, George A, Chin D, frontier communities . J Rur Richardson C. Errors in Health 2004; 20(4): 355-62. general practice: Development of an error 66. Wilf-Miron R, Lewenhoff I, classification and pilot study of Benyamini Z, Aviram A. From

aviation to medicine: Applying physicians during outpatient concepts of aviation safety to visits . Ann Fam Med 2004; risk management in 2(2): 125-9. ambulatory care . Qual Saf Health Care 2003; 12(1): 35- 72. Kaldjian L, Jones E, 9. Rosenthal G, Tripp-Reimer T, Hillis S. An empirically derived 67. Williams E, Maxwell L, Konrad taxonomy of factors affecting T, Linzer M. The relationship physicians' willingness to of organizational culture, disclose medical errors . J Gen stress, satisfaction, and Int Med 2006; 21(9): 942-8. burnout with physician- reported error and suboptimal 73. Kirk S, Parker D, Claridge T, patient care: Results from the Esmail A, Marshall M. Patient MEMO study. Health Care safety culture in primary care: Manag Rev ; 32(3): 203-12. developing a theoretical Health Care Manag Rev 2007; framework for practical use . 32(3): 203-12. Qual Saf Health Care 2007; 16: 313-320. 68. Woolf S, Kuzel A, Dovey S, Phillips R. A string of 74. Wallace L, Boxall M, mistakes: the importance of Spurgeon P, Barwell F. cascade analysis in Organizational interventions to describing, counting, and promote risk management in preventing medical errors . primary care: the experience Ann Fam Med 2004; 2(4): in Warwickshire, England . 317-26. Health Serv Manag Res 2007; 20(2): 84-93. 69. Dean D, Hutchison A, Escoto K, Lawson R. Using a multi- 75. Report of the Harvard Medical method, user-centred, Practice Study to the State of prospective hazard analysis to New York. Patients, doctors, assess care quality and and lawyers: medical injury, patient safety in a care malpractice litigation, and pathway . BMC Health Serv patient compensation in New Res 2007; 7: 89. York . 1990, Harvard University Press: Cambridge, 70. Elder N, Palleria H, Regan S. Mass. What do family physicians consider an error? A 76. DiCicco-Bloom B,Crabtree B. comparison of definitions and The qualitative research physician perception . BMC interview . Med Ed 2006; Fam Pract 2006; 7: 73. 40(4): 314-321.

71. Elder N, VonderMeulen M, 77. Pace W, Staton E, Higgins G, Cassedy A. The identification Main D, et al. Database of medical errors by family design to ensure anonymous

study of medical errors: A Adverse drug events and Report from the ASIPS medication errors in Australia . collaborative . Am Med Inform Int J Qual Health Care 2003; Assoc 2003; 10(6): 531-40. 15(suppl_1): i49-59.

78. Ashcroft D, Morecroft C, 84. Brennan T, Leape L, Laird N, Parker D, Noyce P. Safety Hebert L, et al. Incidence of culture assessment in adverse events and community pharmacy: negligence in hospitalized development, face validity, patients. Results of the and feasibility of the Harvard Medical Practice Manchester Patient Safety Study I. N Engl J Med 1991; Assessment Framework . Qual 324(6): 370-6. Saf Health Care 2005; 14(6): 417-21. 85. Wilson R, Runciman W, Gibberd R, Harrison B, et al. 79. DeRosier J, Stalhandske E, The Quality in Australian Bagian J, Nudell T. Using Health Care Study . Med J health care Failure Mode and Aust 1995; 163(9): 458-71. Effect Analysis: the VA National Center for Patient 86. Burton E, Troxclair DA, Safety's prospective risk Newman III W. Autopsy analysis system . Jt Comm J diagnoses of malignant Qual Improv 2002 28(5): 248- neoplasms: How often are 67. clinical diagnoses incorrect? JAMA 1998; 280(14): 1245- 80. Beyer M, Dovey S, Gerlach F. 1248. Fehler in der Allgemeinpraxis - Ergebnisse der 87. Burton J,Underwood J. internationalen PCISME- Necropsy practice after the Studie in Deutschland [Errors “organ retention scandal”: in general practice - results of requests, performance, and the international PCISME tissue retention . J Clin Pathol study in Germany] . Z Allg Med 2003 56(7): 537-41. 2003; 79: 1-5. 88. Working PTRCoPoAA. The 81. Barnes J. Pharmacovigilance decline of the hospital of herbal medicines . Drug Saf autopsy: a safety and quality 2003; 26(12): 829-51. issue for healthcare in Australia . Med J Aust 2004; 82. Gough S. Post-marketing 180(6): 281-5. surveillance: a UK/European perspective . Curr Med Res 89. Dovey S, Hickner J, Phillips R, Opin 2005; 21(4): 565-70. Developing and using taxonomies of errors , in 83. Runciman WB, Roughead EE, Patient Safety: Research into Semple SJ, Adams RJ. Practice , Walshe K,Boaden R,

Editors. 2006, Open University Press, McGraw-Hill Education: Manchester. p. 93- 107.

90. Mazor K, Fischer M, HL H, D H. Factors influencing preceptors’ responses to medical errors: A factorial survey . Acad Med 2005; 80(Suppl 10): S88-S92.

91. (2007) International Classification for Patient Safety. Version 1.0 for Use in Field Testing 2007-2008 .

92. Runciman W, Williamson J, Deakin A, Benveniste K, et al. An integrated framework for safety, quality and risk management: an information and incident management system based on a universal patient safety classification . Qual Saf Health Care 2006; 15(Suppl 1): i82-i90.

Appendix: Table 3: Studies measuring patient safety events in primary care settings identified in the literature

Systematic Literature Reviews Study Research aim Research methods Research measures Design Overall conclusions limitations Elder and Describe and Searches of Medline Preventable adverse events (diagnostic, English language Little is known about Dovey classify process and the Cochrane treatment, preventive care incidents); Process only primary patient safety in (2002) 29 errors and Library errors (clinician, communication, administration, care. Most studies focus preventable “blunt end” factors) on physician perspectives. adverse events Sandars and Identify frequency Searches of Medline, 5-80 “medical errors” per 100,000 consultations: English language Nature and frequency of Esmail and nature of error Embase and NPSF 26-78% of “errors” relate to diagnosis only “error” in primary care is (2003) 33 in primary care; database 11-42% of “errors” relate to treatment poorly understood consider causes for <1-11% of prescriptions have “errors” because of diversity in diversity in reported 60-83% of “errors” are preventable definitions and error rates approaches. Baker et al Review how Searches of Medline, Impact of primary care on mortality, methods of English language Mortality data are not (2007) 26 mortality data are Embase and CINAHL monitoring mortality, role of audit and death only used systematically in used in primary registers in quality and safety improvement. primary care to improve care quality and safety. Studies of Significant Event Audits Murie and Describe significant Content analysis of Ratio of near-miss to adverse events = 1:6 Regionally General practitioners can McGhee events 56 significant event 44% of reports relate to operational risk limited work in a national (2003) 31 reports from one 36% of reports relate to clinical risk framework for risk Scottish Primary Care 16% of reports relate to strategic risk assessment. There is a Trust 4% of reports relate to human resources need for consistency in 2% of reports relate to political incidents definitions and coding. 2% of reports relate to legislative incidents 56% had no or minimal harm 40% resulted in a disabling injury or death Bowie et al Identify Content analysis of 25% involved a learning issue Regionally If significant event (2005) 27 satisfactory and 662 significant event 11% of reports were judged unsatisfactory limited reporting is to be effective unsatisfactory reports from the west 17% of reports relate to prescribing in improving quality and significant event of Scotland 16% of reports relate to diagnosis safety, there must be a reports 13% of reports relate to communication valid way to check reports. Cox and Describe significant Content analysis of 19% of reports involved a learning issue Regionally Significant event audits Holden events 337 significant event 29% of reports relate to prescribing limited valuable for education and (2007) 28 reports from one 27% of reports were patient safety incidents clinical governance that English Primary Care 7% were serious or life-threatening highlight patient safety Trust 20% were potentially serious issues.

Studies of malpractice claims and risk management databases Fischer et al Describe the Review of 51 incident 3.7 “adverse events” per 100,000 clinic visits Regionally Serious adverse events (1997) 30 prevalence of reports to a risk- 14% of injuries due to environmental hazard limited are probably infrequent in adverse events management 69% of injuries due to medical mismanagement: Limited to an primary care. Better database over .55 31% relate to treatment academic health systems are needed to year period 26% relate to diagnosis center track events. Phillips et al Describe negligent Review of 49,345 68% of claims were for negligent events in Data definitions The total burden of high (2004) 32 adverse events claims in a national primary care may not be severity outcomes and from primary care; US malpractice 34% of negligent claims relate to diagnosis robust death is higher when assess condition- database over a 15 16% of negligent claims relate to failure to Limited to the US negligent events occur in specific rates of year period monitor primary care than when claims 15% of negligent claims relate to improper they occur in hospitals. performance 8% of negligent claims relate to prescribing 4% of negligent claims relate to delay in referral Varga et al Describe criminal Review of 94 criminal 82% of criminal cases involved physicians Limited to Primary care physicians (2006) 34 liability of cases involving 29% of physicians were primary care Hungary are at the centre of more healthcare physicians and other physicians and the main reason for their case criminal cases than any providers in healthcare providers, was failing to examine the patient or other healthcare Hungary over a 4 year period diagnostic error professional group. 10% of criminal cases involved nurses 5% of criminal cases involved paramedics 3% of criminal cases involved pharmacists Interview studies Diamond et al Describe GP Qualitative analysis of 180 critical incidents (4.6 per doctor) Purposive Analysis of critical (1995) 38 trainee experiences open-ended 50% of critical incidents involved difficult sampling from incidents can accelerate with positive and interviews with 39 GP patients, children, counseling skills, the doctor- one training learning and help plan negative incidents trainees patient relationship, obstetrics and gynaecology, program curricula. relationships with other health professionals and practice staff, and cardiovascular disorders. Ely et al Determine In-depth interviews 76% response rate (53 physicians/errors) No sample size Physicians remembered (1995) 44 perceived causes with a random sample 57% of errors related to missed diagnosis justification errors with often serious of family physician of 70 family 21% of errors related to surgical mishaps consequences that they error physicians 25% of errors related to treatment attributed to a variety of Mean of 8 causes for each error case causes. 91% of errors caused by physician stressors 91% of errors caused by care processes 72% of errors caused by patient factors 62% of errors caused by physician factors Kuzel et al Develop patient- Structured interviews 38 interviews analysed No sample size Medical errors related by

(2004) 51 focused typologies with 40 people Mean of 5.8 problematic incidents per interview justification patients contrast sharply of medical errors recruited by random 37% of incidents involved breakdown in doctor- with errors reported by and harms in digit dialling patient relationship family physicians. primary care 77% involved disrespect or insensitivity 29% of incidents involved breakdown in access 27% involved excessive office waiting times 16% involved difficulty in contacting the office 16% involved delay in obtaining appointments Mean of 4.5 harms per interview (170 harms) 70% of harms were psychological Avery et al Define ways Semi-structured 4 main themes Limited to Primary care computer (2006) 35 computer systems interviews with 31 Designing systems for safety stakeholders with systems could be can be improved to clinicians, computer Accurate and relevant information computer improved and this would enhance safety in system suppliers, Taking human ergonomics into account expertise enhance patient safety. primary care academics, and Audit trails Excluded non- policymakers Electronic information transfer medical team Optimizing computer safety features members and Recording data accurately patients Call and recall reminders Training for safe and effective computer use Regulations and guidelines Safety culture Focus group studies Dowell et al Describe patients’ 3 focus groups 187 comments Limited to Patients provide important (2005) 43 experiences of including 21 patients 44% of comments were about system issues English insights into complex quality and safety from 3 urban clinics 37% of comments were about interpersonal skills language- systems issues. in primary care 9% of comments were about knowledge proficient 7% of comments were about medication and patients technical errors and errors of inattention Mazor et al Describe how and 7 focus groups Preceptors provided corrective and supportive Analysis may be Future research should (2005) 55 why preceptors involving 38 primary responses to trainees’ errors influenced by focus on faculty respond to trainees care preceptors from Factors influencing preceptors’ responses were researcher bias development to optimize when medical north-west USA error type, clinical outcome, and their connection, learning from errors and errors occur and the learner’s response reduce future recurrence Surveys Holden et al Determine pattern Audit of all 1263 679 avoidable factors contributed to deaths Limited practice An audit of deaths has (1998) 46 of deaths and deaths occurring over 40% were patient factors (eg smoking, suicide) participation educational value for GPs preventable factors 40 months, using a 6% were hospital factors (eg delayed diagnosis) and is a source of ideas in 4 general standard data 5% were GP factors (eg delayed referral) for service improvement practices collection form 3% were environmental factors (eg falls) and further study.

MacPherson Describe the type Postal survey of 1848 31% response rate (574) Low response Acupuncture is a relatively et al (2001) 52 and frequency of professional 34,407 treatment reports rate safe treatment. adverse events acupuncturists No serious adverse events Non-random and transient 1.3 significant minor adverse events per 1000 sampling reactions after treatments (eg nausea, fainting) consultations with 15% of treatments had mild transient acupuncturists reactions 3 avoidable errors (2 forgotten needles, 1 moxibustion burn) McKay et al Describe GPs’ Census of 617 GP 76% response rate Non-random The success of the NPSA (2004) 56 attitudes to principals in south- 73% would not report all significant events sampling system will be hindered by mandatory west Scotland 75% favoured anonymous reporting mandatory reporting. significant event 41% had difficulty defining a significant event reporting Singh et al Develop and test a 2 practices with 45 Each site identified its own hazards, with little No denominator The method empowered (2005) 62 method to form respondents overlap reported practice teams to develop learning teams that Non-random a common vision of their can prioritize sampling practice microcosm. patient safety problems Hutchison et Develop a patient Census of 3650 staff Response rate 33% for primary care Trusts Low response A 22-item version of the al (2006) 47 safety climate of 4 acute hospital Removing 5 items from the questionnaire rate questionnaire is usable as questionnaire (the trusts and 9 primary improved the internal reliability of the Non-random a research instrument in MaPSaF) care trusts in questionnaire’s two domains of teamwork and sampling primary care and acute England. Factor and safety climate hospital settings. reliability analyses Patient Safety Event Reporting Systems Britt et al 37 Describe safety Modified critical 51-52/100 mishaps involved drug treatments Participation The incident monitoring Bhasale et incidents in incident technique, 37-42/100 mishaps involved non-drug treatments limited to one technique can be used in al 36 (1997-98) General Practice with 297-324 GPs 28-34/100 mishaps involved diagnosis sentinel practice general practice. anonymously 5/100 mishaps involved equipment research network submitting 500-805 76 % of incidents were preventable and additional paper incident reports 17% of incidents resulted in major harm volunteers 4% of incidents resulted in death Poor communication was the most common contributing factor Early intervention was the most common mitigating factor Dovey et al 39- Describe the types 344-416 errors 31% were reports of administrative errors Participation Family physicians will 42 (2002-03) of errors reported reported by 42 family 25% were reports of investigation failures limited to one report errors and their

by family physicians using both 23% were reports of treatment errors sentinel practice consequences and physicians and paper and computer 6% were reports of communication errors research network propose solutions. develop an error based questionnaires 337 reports of consequences taxonomy 38% were health consequences 35% were care consequences 22% were financial and time consequences 288 ideas about solutions to medical errors 34% of solutions were “don’t make mistakes” 30% of solutions were “better communication” 26% of solutions were “provide care differently” Makeham et To describe the Paper and computer 437 error reports: 132 from Australia, 81 from Small number of Errors are common and al,53 Woolf et types of errors based questionnaires Canada, 63 from England, 14 from Holland, 66 reports per similar in nature in general al, 68 Rosser et reported by GPs submitted by 100 from New Zealand, 75 from the US. country practice and family al,59 Jacobs et and family GPs and family 9-25% of reports related to office processes Limited time medicine settings around al, 48 Tilyard et physicians in 7 physicians over a four 14-30% of reports related to treatment frame for data the world. al 64 (2002-05) countries and month period 14-22% of reports related to investigations collection The PCISME develop an 7-19% of reports related to communication project international error 2-3% of reports related to workforce taxonomy 1-2% of reports related to finances 77% of reports documented a chain of errors 29-39% of reported incidents resulted in harm 66 different prevention strategies 70% of reports related to more diligence 23% of reports related to providing care differently 20% of reports related to communication Wilf-Miron et Apply aviation Root cause analysis 1300 events were accidents and near misses Regionally Aviation safety concepts al (2003) 66 safety principles to of 2000 adverse 21% involved family medicine limited and tools were reporting errors in a event reports over 5 33% of errors related to processes of care Multidisciplinary successfully adapted to large ambulatory years reported by 21% of errors related to treatment study did not ambulatory care. healthcare setting telephone hotline to a 18% of errors related to judgment report primary specialized risk 15% of errors related to auxiliary tests care results management unit 13% of errors related to poor communication separately 470 recommendations made to improve care Fernald et Develop a system 128-754 confidential 97- reports of patient safety events Participation Confidential reports afford al, 45 Parnes et for confidential or anonymous reports 71-72% of reports involved communication limited to two greater analysis of cause al, 57 Westfall error reporting, submitted by phone, 33% of reports involved delay in care sentinel practice than anonymous reports. et al 65 (2003- describe types of electronically, or on 20-35% of reports involved medication events research 04) error and paper from 14-33 17-47% of reports involved diagnostic testing networks The ASIPS differences practices with 150- 209 reports of harm from patient safety events Regionally

project between 475 clinicians and 68% were no known harm limited confidential and staff 6% were unstable anonymous reports 7% were non-clinical harm 9% were increased risk of clinical harm 10% were clinical harm now Rubin et al, 60 Classify errors in 540 events reported 136 reports of 940 errors from general practice Regionally Medical error descriptions Steele et al63 general practice using an anonymous and 439 from optometric practice limited generated in primary care (2003-06) and community paper form, reporters 18-42% of errors related to prescriptions Limited time are applicable to different optometric clinics in coded events using a 30-36% of errors related to communication frame for data types of provider. north-east England descriptive 12-16% of errors related to equipment collection classification 2-7% of errors related to appointments 10-35% of errors related to clinical care 75.6 “errors” per 1000 GP consultations Shaw et al Describe the Electronic patient 28,988 safety incident reports were made No data reported Majority of reported (2005) 61 implementation of a safety incident reports 32 (0.1%) came from the primary care Trust for primary care incidents from all sources national incident from 1 Primary Care specifically were slips, trips and falls. reporting system and 17 other Trusts in Primary care engages England and Wales poorly in the system. Makeham et Determine the 418 anonymous web- 26% (84) of the random sample of 320 agreed to Regionally Incidence of GP-reported al (2006) 54 incidence of based error reports participate in the study limited errors can be calculated reported errors in from a random 1-25 reports per participating GP Low response when a secure general practice sample of 320 New 5.3 reports on average, per participating GP rate anonymous reporting South Wales 1 error per 1000 GP encounters per year system is provided. (Australia) GPs over 2 errors per 1000 patients seen per year 12 months Incidence of reported errors per patient = 0.24% Phillips et al Compare the types 126 reports from 108/126 patient reports expressed satisfaction Study design Clinicians and (2006) 58 of medical error patients and 717 6/18 patient error reports were about waiting unsuitable for administrative staff reports made by reports from 401 2/18 patient error reports were about learning of perceived errors primary care clinicians and staff mistaken identity patients’ views of differently (through clinicians, reporting 717/726 provider reports included 935 errors patient safety different “lenses”). administrative staff, electronically, in 56% of errors related to office administration Patients did not engage and patients. paper reports, or by 15% of errors related to treatments well with the study’s telephone. 14% of errors related to investigations processes. 9% of errors related to communication 4% of errors related to knowledge and skills 3% of errors related to payment Kostopoulou Describe patient 78 reports from 5 21 adverse events and 50 near misses Small number of Cognitive and system (2007) 50 safety incidents in general practices with 47 reports had information about the active reports limits factors both contribute to general practice follow up interviews failure leading to the patient safety incident description patient safety incidents in based on cognitive from investigators 45% involved situation assessment errors breadth primary care. psychological 23% involved response execution errors

theory 17% involved memory errors 15% involved perception errors 17% had fairly or very serious patient harm 76% had potential for patient harm Mixed Method studies Elder et al Describe errors 15 family physicians 117 errors reported in 83 (24%) consultations Regionally There is variation between (2004) 71 identified by family in 17 practices 17% of error visits were administration error limited doctors in how harm is physicians and identified errors 8% of error visits were physician related error defined and how different determine during 351 patient 5% of error visits were communication error error categories are physician’s visits. Interviews at 45% of error visits were preventable adverse interpreted. perception of the end of consulting events resulting harm sessions collected 24% of error visits harmed patients data about harm. 70% of error consultations involved potential harm Dean et al Evaluate feasibility Observation of care Themes from interviews were mainly about Limited to one Mixed methods study (2007) 69 of prospective processes for patients communication problems chronic disease delivered a clear picture of hazard analysis of with COPD, The Delphi study ranked difficulties in accessing the quality and safety of care pathways interviews with 16 hospital records, information transfer to primary care in a care pathway. crossing primary patients and 7 staff, care, and failure to communicate medication and secondary two-round modified changes as the most important events care interface Delphi study Elder et al Define what Systematic review 25 different definitions of “error” found in the English language Definition of “error” is an (2006) 70 physicians consider followed by census of literature literature only area of confusion an “error” active members of The most common definition is Reason’s 9 Low response the American 29% response rate to the survey rate to survey Academy of Family Family physicians’ definition of error depends Survey sample Physicians on event outcomes, whether the event was size not justified rare or common, and whether it was related to the system of care or an individual mistake. Kaldjian et al Describe factors Literature review; 5 Literature review revealed 53 factors impeding English language A conceptual framework (2006) 72 affecting voluntary focus groups disclosure and 38 factors facilitating disclosure literature only describing factors disclosure of Focus groups added 27 factors MEDLINE facilitating and impeding medical errors by The final taxonomy had 4 facilitating factors: database search disclosure of medical physicians Responsibility to patient; Responsibility to only errors was developed. profession; Responsibility to self; Responsibility to community and 4 factors impeding error disclosure: Attitudinal barriers; Helplessness; Uncertainty; Fears and anxieties Kirk et al Test a framework Literature review, 9 dimensions of organization safety culture were: Regionally Provides a way of

(2007) 73 for making the survey,33 semi- Overall commitment to quality; priority given limited understanding why and concept of a safety structured interviews, to patient safety; perceiving the causes of how the shared values of culture meaningful 14 focus groups patient safety incidents; investigating patient staff can together create a to primary care safety incidents; organizational learning safety culture in practice providers following a patient safety incident; communication about patient safety; personnel management; staff education; team-working 5 levels of safety culture maturity were: Pathological; reactive; calculative; proactive; generative Wallace et al To examine the Survey of 75 practice Risk management competence scores of Non-random There was evidence that (2007) 74 effect on practice managers to derive a practices after the intervention did not change sample risk management safety culture of a risk management significantly from before. Before-and-after improved over the period health authority’s competence score 6 general measures of competence: study without of the study but little promotion of risk before and after an Scope: before = 11, after = 13 controls evidence that this was management intervention of a Staff involved in risk management: before = caused by organizational training package to 12, after = 15 culture improve risk Documenting risk management activity: management skills before = 6, after = 13 Access to records: before = 15, after = 15 Written practice policies: before=9, after=11 Written procedures: before = 8, after = 8