Patient Safety Climate: a Study of Southern California Healthcare

175

, professor, Department of professor, and director, Management, of PhD , McGuire

, assistant professor and coordinator, Graduate Healthcare Management Healthcare Graduate assistant professor, and coordinator,

PhD MMARY , SU Stephen J.J. TIVE Avramchuk

U For a subsample of hospitals in our study, preliminary results indicate a predictive qual- a predictive indicate results preliminary study, in our hospitals of a subsample For leaders healthcare urge we health outcomes, between PSC and the association Given We developed and tested a short survey instrument intended as an organization-level organization-level an as survey intended a short tested instrument and developed We to use various means, such as our survey, to monitor the degree to which their organiza- to the degree monitor to survey, our as such means, use various to for areas pinpoint to data use such and safety patient fosters that a climate maintain tions improvement. ity of the model. The higher the PSC score, the lower the number of violations detected by detected of violations number the lower the higher The PSC the model. the score, of ity the the fewer and inspections, Services & Medicaid in complaint Medicare for the Centers Group. The Leapfrog by reported problems safety measure of PSC with good psychometric properties that can be used in hospitals, clinics, or or clinics, be can used in hospitals, that properties good PSC with psychometric of measure California 61 Southern healthcare from data of Analysis settings. provider healthcare other From (1) Assistance factors: reliable distinct, four in a PSC model with resulted organizations (3) Behavior, and in Policy Support of Messages (2) Leadership the Organization, Others and ranging A PSC score, Reporting Behavior. (4) Error and Environment, Work and Resources each organization. for generated was 100, 0 to from hospitals. Human error results from healthcare providers’ attitudes and behaviors toward toward behaviors and attitudes providers’ healthcare from results error Human hospitals. and attitudes ofthe taking periodicsnapshots Therefore, settings. in different patients (PSC) is climate safety patient in its manifested and organization in an prevalent behaviors essential. Human error remains the most important factor in unnecessary deaths and suffering in U.S. in suffering in unnecessary and factor deaths important the most remains error Human EXEC © 2018 Foundation of the American College of Healthcare Executives Healthcare College the of American of © 2018 Foundation DOI: 10.1097/JHM-D-16-00004 www.ache.org/journals For more information regarding the concepts in this article, contact Dr. Avramchuk at at Avramchuk Dr. contact in this article, the concepts regarding information more For [email protected]. toequally this article. contributed authors The interest. of no conflicts declare authors The graduate College programs, of Business LosAngeles & Economics, California State University, Patient Safety Climate: A Study Climate: A Safety Patient of Southern Organizations California Healthcare S. Andre University, State California & Economics, of College Business of Department Management, Program, and Angeles; Los © 2018 Foundation of the American College of Healthcare Executives. All rights reserved. © 2018 Foundation of the American College of Healthcare

Downloaded from https://journals.lww.com/jhmonline by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3oaxD/vH2r75uyl+s+ajggZNtmts+1VfgtM9h1Tc/L7I= on 03/19/2020 Downloaded from https://journals.lww.com/jhmonline by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3oaxD/vH2r75uyl+s+ajggZNtmts+1VfgtM9h1Tc/L7I= on 03/19/2020 JHM-D-16-00004 Journal of Healthcare Management

INTRODUCTION Reports and U.S. News and World Report. One study estimates that more than Thus, healthcare managers are increas- 400,000 people die unnecessarily in U.S. ingly interested in evidence-based tools hospitals every year because of health- and approaches that help them achieve care provider errors (James, 2013), and improved patient outcomes and safety 11 million may die unnecessarily in the records and, consequently, the organiza- decade to come (Levitt, 2014). The average tion’s reputation (Birk, 2015; Kurtzman, annual cost of preventable medical errors 2015; Weaver et al., 2013). in hospitals nationwide has been about $29 Patient safety can be jeopardized by billion (Institute of Medicine, 2000). The many factors, including infrastructure and U.S. healthcare industry has invested in technology (Holden, 2011; Wachter, 2012). technology, training, and systems to reduce This research, however, centers on the human error, yet problems persist. Look- variables that foster patient safety and on ing at patient safety from different angles creating a survey instrument to measure is important (Farup, 2015), and no single an organization’s climate, which indicates training tool or assessment instrument will its underlying culture based on behavioral be sufficient to safeguard patient well-being and attitudinal perspectives. We define in today’s complex and changing health- patient safety climate (PSC) as a set of care settings. shared values and practices that prioritize While concern for patient safety is patient safety, whereby safety-enhancing inherent in the healthcare professions, behaviors are not only expected, but acknowledging, measuring, and managing expected to be exhibited to a high degree. a climate of patient safety are relatively recent This definition is in line with previous developments. Healthcare’s increased research drawing on shared perceptions attention to the phenomenon will likely and behaviors to assess the PSC (Hughes, continue to be driven by regulations for Chang, & Mark, 2009). Medicare reimbursements under the Given the continued suffering and Affordable Care Act (ACA) (Hacker & costs associated with unsafe healthcare Walker, 2013). Under the ACA, hospitals environments (Levitt, 2014), creating may no longer be reimbursed for prevent- and maintaining a climate that fosters able readmissions of discharged patients patient safety must remain a top prior- within 30 days, so healthcare leaders ity for healthcare leaders. This priority is have a financial incentive to identify the especially acute in the context of manag- causes of care-delivery errors and address ing organizational change, when cultural variables that lead to preventable readmis- climate shifts may be advantageous or sions. The federal government and private detrimental to the perception and reality of sources are making information publicly patient safety. On the other hand, an orga- available regarding healthcare providers’ nization’s capacity to absorb and sustain quality and safety, further driving consumer changes related to improving patient safety awareness of healthcare environments. may depend on the established culture at Ratings and rankings of hospitals are now work (Kash, Spaulding, Gamm, & Johnson, published in magazines such as Consumer 2013). The assessment of the PSC before,

during, and after change implementa- the shared perceptions of members about tion may provide healthcare leaders with the organization’s policies, procedures, and important information regarding desired practices, as well as what gets rewarded cultural transitions. With increased advo- in the organization. In addition, climate cacy for comprehensive, whole-system may include behaviors typically exhibited changes in their organizations (Toussaint, by employees, and the climate can worsen 2015), healthcare managers need to assess or improve over time. Therefore, changes the PSC regularly alongside other orga- in organizational climate may be suscep- nizational dimensions to achieve a broad tible to changes in policies, practices, and view of the associated issues and opportu- rewards, as well as the behavior demon- nities for improvement. Our study offers strated by organizational leaders. one empirically generated tool for doing so. Keyton (2014) remarked that “climate is the text of an organization, whereas culture BACKGROUND is the subtext,” and studies of organiza- Although the vast literature on patient tional climate “tend to focus on individual safety and relevant cultures (Parker, Wensing, employees’ attitudes and feelings (that are Esmail, & Valderas, 2015; Singer, Falwell, aggregated to some level) of a variety of et al., 2009; Wachter, 2012; Waterson, 2014) organizational behaviors and character- undoubtedly informed our research, we istics” (p. 132). Research has converged focus here on key aspects specific to the on two primary ways in which individual development of our PSC assessment. To members of an organization come to complement the PSC definition stated share meaning about their organization: earlier, we clarify the difference between social interaction and leadership behaviors often-conflated notions of organizational (Ostroff, Knicki, & Tamkins, 2003; Schneider & culture and climate, note the history of Reichers, 1983; Zohar & Hofmann, 2012). safety climate research and its PSC variant, If meaning is socially constructed, then it and provide a review of selected assess- is through social interaction that members ment tools and a conceptual framework for of a group make sense of their observations our study. and behaviors. Meaning is determined by “the interplay between one’s own percep- Organizational Culture and Climate tions and those of others in the same situa- Organizational culture shapes, and is tion” (Zohar, 2010, p. 1519). Over time, the shaped by, the way in which work groups different perceptions of group members attempt to solve problems. Behaviors found tend to fuse as individuals influence each to be successful are usually repeated and other and the organizational culture. Deter- taught to new organizational members mining meaning in the organization is an (Schein, 2010). While culture is seen as ongoing, interpretive process (Weick, 2012), an enduring construct, organizational and so should be the assessment of organiza- climate is considered to be a temporary tional climate. manifestation—a snapshot—of an under- Organizational members must make lying culture. Schneider, White, and Paul sense of the words and behaviors of (1998) described organizational climate as their supervisors and the organization’s

leaders (Zohar & Hofmann, 2012). When outcomes across industries and coun- supervisors’ and leaders’ behaviors are tries” (p. 1517). ambiguous (or contradict policy or oral One type of safety climate is unique statements), organizational members to healthcare organizations: PSC. Accord- may seek out others to make sense of ing to the Institute of Medicine’s (2000) these discrepancies. Such discrepancies foundational report, a safety climate is cre- may be mitigated in part by an ongo- ated through these cultural and structural ing assessment of organizational policies, attributes of organizations: procedures, or practices. In an analysis of 21 studies, Bronkhorst, Tummers, Steijn, 1. The actions management takes to and Vijverberg (2015) suggested that improve both patient and worker healthcare organizations would “benefit safety from incorporating organizational climate 2. Worker participation in safety factors in their health and safety policies” planning (p. 254). Whether the shared perceptions 3. The availability of appropriate pro- of organizational climate are rooted in tective equipment social interaction or leadership behaviors 4. The influence of group norms and practices, the need for continuous regarding acceptable safety practices assessment of this climate is evident. 5. The organization’s socialization process for new personnel Patient Safety Climate Safety climate research began approxi- PSC research has drawn on these com- mately 35 years ago in the manufacturing ponents and descriptors of organizational industry as a result of concerns about culture that contribute to patient safety, workplace accidents. Zohar (1980) and and some assessments have also incorpo- others began identifying employees’ rated national cultural differences (Singer shared perceptions about certain char- & Vogus, 2013). Workplace cultures that acteristics of the environment related to exhibit the above practices are presumed safety. Zohar (2003) noted that a safety to have a better safety climate. Differences climate constitutes the shared percep- in leadership behaviors or availability of tions of organizational members about the resources and training affect the PSC, and importance of safety compared with other cultures known for higher levels of engage- organizational priorities. Several studies ment and encouragement also have higher have shown that safety climate scores pre- indicators of PSC (Armstrong, Laschinger, & dict both objective and subjective safety Wong, 2009; McFadden, Stock, & Gowen, outcomes in different industries (Chris- 2015). Armstrong et al. (2009) surveyed tian, Bradley, Wallace, & Burke, 2009). 300 nurses in acute hospital settings and Zohar (2010) reviewed three decades of found that they felt empowered by support safety climate research, and highlighted for their professional practice, recogni- the achievement of “an enormous task of tion for doing a good job, and involvement validating safety climate as a robust lead- in decision-making. Strong perceptions ing indicator or predictor of safety of empowerment directly related to the

climate of patient safety. Hospital cultures and “safety deficiencies in the organiza- in which an environment of trust and tional infrastructure” (Singer, Gaba, et al., open communication between leaders and 2009, p. 30). employees has been developed and culti- The Safety Attitudes Questionnaire vated also have been associated with an was originally designed to measure com- increased awareness around patient safety mercial aviation crew attitudes and was (Auer, Schwendimann, Koch, De Geest, & adapted for inpatient settings and later Ausserhofer, 2014). Specifically, the “ease transformed into a 62-item ambulatory- of reporting, unit norms of openness, setting version (Modak, Sexton, Lux, and participative leadership are positively Helmreich, & Thomas, 2007). Because of related to staff perceptions of patient safety some challenges with longer PSC surveys climate” (Zaheer, Ginsburg, Chuang, & (e.g., usability of resulting data), the Cana- Grace, 2015, p. 13). dian PSC survey (Can-PSCS), based in Some of the interest in PSC research part on Zohar’s work in safety climate, was is based on the nationally driven patient- a brief, 19-item instrument with robust safety agenda from organizations such as psychometric properties (Ginsburg, Tre- the National Patient Safety Foundation gunno, Norton, Mitchell, & Howley, 2014). (www.npsf.org) that focus on common Its development and testing occurred in threats (e.g., medication and diagnostic a publicly funded, universal healthcare errors, hospital-based falls, and infections). system in Canada; and the success of These outcomes are often attributed to the Can-PSCS has not been consistently rep- behaviors of organizational members pro- licated in other systems, “perhaps owing viding care, a lack of training and support, to unique country characteristics, types of and breaches in communication. One state health systems, samples, [and] cultural dif- health authority in Australia summarized ferences” (Ginsburg et al., 2014, p. 2). these difficulties: “Measuring and improv- Recognizing the need for numerous ing the safety climate can be challenging changes to its widely used 2004 Hos- as it involves such factors as values, beliefs, pital Survey on Patient Safety Culture people, and practices” (VMIA, 2015). (HSOPSC), the Agency for Healthcare Research and Quality (AHRQ, 2016) PSC Assessment has been working on a new survey to be Several tools have been created over the released in 2018. Based on a decade of years to address these challenges. Singer feedback from surveyed hospitals, some et al. (2007), for instance, described the key changes in this comprehensive assess- development and validation of a 38-item ment of safety climate pertain to bridging PSC survey (with an additional six demo- the divide between clinical and nonclinical graphic questions). The survey was geared personnel’s understanding of survey ques- toward hospital settings and examined dif- tions, broadening the survey’s appeal and ferences in employee perceptions of safety translatability of questions for international culture. The survey results indicated that audiences, and reducing the instrument’s direct-care nurses were more attuned than complexity and length. Although HSOPSC physicians to patient safety considerations and some other popular instruments have

“safety culture” or “safety attitudes” in their behaviors lies employee sense-making titles, they are usually considered to be about measuring safety climate (Agnew, Flin, & Mearns, 2013; Colla, Bracken, Kinney, & • employees’ and managers’ reliance Weeks, 2005; Jackson, Sarac, & Flin, 2010; on mutual commitment to resolu- Soh, Barker, Moreno, Dalton, & Brand, tion of patient safety concerns, 2016). A constant evolution and diversity • positive messaging and action mod- of approaches and applications characterize eling for patient safety, the development of patient safety culture • a developmental environment that and climate assessment tools, which is enables adequate training, and projected to continue for years to come • transparency and knowledge (Farup, 2015). exchange regarding errors.

Conceptual Framework Because healthcare leaders must ensure The leadership, culture, and organizational that any innovation or transformation is technologies (LCOT) framework (Kash not detrimental to patient safety (Holden, et al., 2013) fits with our research review 2011; Naylor, Aiken, Kurtzman, Olds, because of its practical relevance for man- & Hirshman, 2011), PSC assessments— aging change. Measuring cultural readi- within the LCOT framework—might ness for change is at the heart of LCOT, provide useful indicators when determin- and we view any reliable PSC assessment ing absorptive capacity for change toward a as a gauge of readiness for patient safety safer healthcare setting. improvements. Specifically, the LCOT Our review of the literature and exist- model relies on measuring the absorptive ing PSC assessment instruments led us capacity for change—the ability to dis- to focus on the following aspects: (1) the cover and exploit innovation, across and willingness of organizational members to within healthcare organizations (Kash et ask for (and receive) reliable assistance al., 2013). Zahra and George (2002, p. 186) when facing a difficult situation with defined this capacity as a “set of organi- implications for a patient’s safety; (2) the zational routines and processes by which prominence of patient safety in organi- firms acquire, assimilate, internalize, and zational leaders’ policies and messages exploit knowledge to produce a dynamic (e.g., when safety is competing with other organizational capability.” The model’s organizational priorities); (3) the extent to other two components—leadership and orga- which the training and work environment nizational technologies—are also culturally ensures that employees can provide quality and structurally embedded in leadership care and are knowledgeable about safety behaviors (e.g., promoting cooperation practices and policies; and (4) the extent to and generating alternatives [Hazy, 2008]) which employees are encouraged to discuss and organizational tools and processes and report errors and other detriments to (e.g., communication systems and train- patient safety. Our incorporation of these ing [Gamm, Kash, & Bolin, 2007]). At the four factors into our framework is summa- intersection of related work routines and rized next.

Factor 1: Assistance From Others and Factor 3: Resources and Work the Organization (ASST) Environment (RWE) This variable captures the extent to which This variable captures employees’ perceptions employees perceive that their efforts to that the work environment is conducive to ensure patient safety are supported by learning about, making decisions toward, the organization, and that they can rely and behaving appropriately with regard to on and receive assistance from managers patient safety. In addition, the organiza- and coworkers when faced with difficult tion provides employees with sufficient situations. As Singer, Falwell, et al. (2009) resources—primarily training—to per- showed, effective teamwork and help- form their jobs effectively. Most validated oriented group dynamics may characterize PSC assessments (e.g., Singer et al., 2007) a culture in which patient safety is in focus. include a section measuring the availability Factor 1 also correlates with the LCOT of appropriate resources. Aimed at both model’s cultural attribute of teamwork cultural and technological resource avail- commitment. ability, RWE focuses on training and ease of acquiring skills to promote patient safety. Factor 2: Leadership Messages of Support in Policy Factor 4: Error Reporting and Behavior (LMS) Behavior (ERB) All employees receive mixed messages Systematic reporting and recording of because organizations typically have errors is essential to understand and address competing values (Stock, McFadden, & threats to patient safety. The ability to report Gowen, 2007). A mixed message is sent, these potential threats and the cultural for example, when organizational lead- norms pertaining to open communication ers prioritize one goal over another in have been associated with perceptions of a situations where the achievement of both better PSC (e.g., Zaheer et al., 2015). When appears to be in conflict (e.g., safety versus employees feel safe reporting their errors productivity). According to Zohar (2010), and bad decisions, work teams and manag- “From an employee standpoint, it is the ers obtain an accurate account of events overall pattern and signals sent by the and are better able to address and minimize complex web of rules and policies across these behaviors in the future. On the other competing domains that ultimately must hand, in organizations in which errors are be sorted out in order to discern what not always reported (or perhaps are covered role behavior is expected, rewarded, and up), taking appropriate corrective action is supported” (p. 1518). To the extent that difficult. Therefore, this variable focuses on managers consistently model and reward the organization’s behavioral and structural pro-safety behavior, the message they are transparency regarding communication sending is quite strong. The LMS vari- of errors. able encompasses both the organizational These four factors reflect the need for policy (Bronkhorst et al., 2015) and lead- employees to feel free to request support, ers’ communication and behaviors regard- make decisions, innovate, and gener- ing patient safety. ally do the right things to perform their

jobs effectively. Similar elements have then rated these items for applicability to been theorized to contribute to the cul- patient safety. We eliminated items that tural notion of workplace empowerment, were not readily understood by the SMEs which has been validated as a predictor and those that received a score of 3.0 or of PSC (Armstrong et al., 2009). Specific below. A pilot study was then carried out to leadership styles in healthcare settings, with a group of 32 students in a doctoral McFadden et al. (2015) stated that “trans- program for nurse practitioners. An initial formational leaders are said to transform factor analysis and reliability analyses led the organization by empowering fol- us to retain 38 items. lowers through inspirational motivation From March to August 2014, we efforts and encourage innovation, which administered the 38-item survey to a is required for creating a safety climate” convenience sample of multiple key (p. 25). Organizations must demonstrate informants at 67 healthcare organizations flexibility in navigating competing values in Southern California. A total of 274 to improve patient safety. Similarly, Stock, individuals completed surveys, well above McFadden, and Gowen (2010) expected our initial goal of 150. The survey asked that a cultural “flexibility orientation, respondents to specify their present job in which includes group and developmental the organization and check a box if they culture types, would be related to better had regular, direct contact with patients. patient safety outcomes” (p. 12). We excluded surveys that did not specify Consequently, we set out to develop “registered nurse” or “nurse practitioner” and validate a PSC assessment to capture as well as “direct contact” with patients. In these aspects, and our resulting survey is addition, we established a minimum tenure a simple, 22-item psychometric scanning of 3 years at the organization to ensure that tool from a U.S.-based, organizational-level respondents were well acquainted with perspective. its culture and could assess the current climate. Thus, we excluded surveys from METHODS respondents who indicated they had less Our research was iterative in nature and than 3 years’ tenure. intertwined data collection, analysis, and To assess organizational climate, data sample refinement by design. In seeking were first analyzed for within-group con- to assess an organizational-level PSC, we sensus (e.g., sufficient agreement among wanted to sample nurses with direct-care respondents from the same organization responsibilities because they likely would to justify aggregation). Insufficient agree- be well attuned to patient safety concerns ment among key informants suggests very in their organizational environments. different perceptions of the organization Our research plan was to obtain com- by employees. We expected all organiza- pleted surveys from three key informants tions to satisfy three criteria for inclusion: from 50 healthcare organizations. Follow- (1) an interclass correlation coefficient ing our literature review, we generated (average measures) of 0.7 or above (Land- 70 potential items. Using a 5-point Likert ers, 2011); (2) an interrater within-group scale, two subject matter experts (SMEs) agreement measure—r∗WG(J)—of 0.7 or

above (Kozlowski & Hattrup, 1992); and reports on several hospitals in our sample.

(3) an average deviation index (ADMJ) These reports are publicly available (Asso- with a statistical significance ofp < .05 ciation of Health Care Journalists, 2015). (Smith-Crow & Burke, 2003). Rather than “averaging out” differences of opinion RESULTS when aggregating the data, we eliminated Our final sample included 61 healthcare from the study any organization whose key organizations and 268 key informants. informants had insufficient agreement. Six For these 61 organizations, the interclass organizations were thus eliminated, result- coefficient (average measures) ranged ing in an effective sample of 61 healthcare from .716 to .945 (p < .05), r∗WG(J) ≥

organizations. 0.70, and ADMJ < 1.167. Thirty organiza- We used an exploratory factor analysis tions were hospitals, and 31 were clinics to determine which preliminary survey or healthcare centers without a hospi- items would load into the conceptual tal facility. Respondents’ tenure ranged model from the literature review. This between 3 and 34 years (mean, 9.79 years, model contained the following four factors standard deviation [SD], 8.16 years). All (i.e., variables), as described earlier: respondents were registered nurses or nurse practitioners with direct patient • Factor 1: ASST contact. Respondents’ sex was not identi- • Factor 2: LMS fied. To ensure that each organization • Factor 3: RWE contributed equally, we conducted subse- • Factor 4: ERB quent analyses using the mean score for each organization. Table 1 presents the To determine how well the proposed number of items, scale reliabilities, means, model fit the data, we then subjected the standard deviations, and correlations for model to a confirmatory factor analysis all variables in the study. using structural equation modeling in AMOS (IBM SPSS, Armonk, NY). We Exploratory Factor Analysis wanted to determine if there was justifica- The mean scores for the 61 organizations tion for aggregating the model’s factors into on the 38-item survey were subjected to one PSC score, and a possibility to com- an exploratory factor analysis (principle pare aggregate, organizational-level scores components with equamax rotation), with any established, independently mea- which suggested that 22 items loaded on sured indicators of healthcare quality and four factors with Eigenvalues greater than safety. For example, The Leapfrog Group 1, explaining 78.6% of the variance in (2015) conducts a survey of several hospi- the data (Table 2). The internal consis- tal safety practices and produces a biannual tency reliability of the four scales exceeded score, which is publicly available. Several Nunnally’s (1978) 0.70 minimum. hospitals in our sample are included in Figure 1 shows the 22 items in our The Leapfrog Group’s survey. Similarly, the revised instrument we called the Cali- Centers for Medicare & Medicaid Services fornia PSC Survey (Cal-PSCS). The items (CMS) recently filed safety inspection corresponded to the four distinct concepts

c 1 7 .876 −.314 1 6 .248 −.087 −.030

c d 1 5 .288 .245 −.713 −.60

Pearson correlations

c d b d 1 4 .215 .580 .489 −.752 −.616

b b d 1 3 .311 .589 .486 .258 −.237 −.143

At At the .05 level. d c b b 1 2 .084 .126 .656 .604 .557 −.281 −.090

d d b b b b 1 1 At At the .01 level. c .246 .360 .840 .867 .796 .804 −.477 −.678 SD 0.70 0.88 8.16 0.79 0.66 5.47 0.87 0.75 16.17 M 4.03 3.66 9.79 3.45 6.22 4.16 70.59 −.023 −.043 α — — Scale 0.843 0.903 0.839 0.788 0.729 0.619 0.851 4 8 4 5 5 3 5 — — items No.of Correlation is signifi cant at the .001 level (2-tailed). 9 n b 61 61 61 61 61 20 20 61

a scores. z ASST = Assistance From Others and the Organization; ERB = Error Reporting Behavior; LF = Leapfrog; LMS = Leadership Messages of Support in Policy and Behavior; PSC and in Policy Support of Messages Leadership = LMS Leapfrog; = Reporting Behavior; LF Error = ERB the Organization; Others and From Assistance = ASST Variable TABLE TABLE 1 for All Study Variables Deviations, Scale Reliabilities, and Intercorrelations Means, Standard 9. LF_ Positives 8. LF_ Problems 1. PSC Index 2. ASST 4. RWE 6. Tenure 5. ERB 7. Violations 3. LMS Items are Index = Patient Safety Climate Index; M = Mean; n = number; RWE = Resources and Work Environment; SD = standard deviation. deviation. standard = Note. SD Environment; Work and Resources = n = number; RWE M = Mean; Index; Climate Safety Patient = Index a

2 TABLE 2 χ = .250 ( p < .01), GFI = .998, normed fi t Exploratory Factor Analysis of Patient index = .997, root mean square error of Safety Climate Items approximation = .000, root mean square Factor 1 2 3 4 residual = .05. Th erefore, we aggregated the ASST_08 .768 four factors into an overall score by calcu- ASST_10 .750 lating a PSC Index, ranging from 0 to 100. ASST_09 .721 From our data, PSC indexes ranged from ASST_18 .700 35.6 to 98.75 (mean [ SD ] = 70.59 [16.17]). ASST_11 .685 Th e internal consistency reliability of the ASST_05 .662 PSC Index was α = .843. ASST_06 .566 ASST_02 .556 Comparing Our Results With LMS_19 .830 Violations Reported by CMS LMS_15 .786 We identifi ed the number of violations LMS_22 .643 reported by CMS in complaint-based LMS_01 .615 inspections of hospitals in 2014. Reports LMS_07 .587 were available for nine hospitals in our RWE_12 .830 RWE_17 .707 data set. When more than one report was RWE_14 .703 fi led during this period, we consulted the RWE_13 .635 most recent report. We opted to use the ERB_21 .767 total number of violations as a proxy for ERB_03 .738 a patient safety outcome measure. For the ERB_04 .595 nine hospitals, the number of violations ERB_20 .560 ranged from 1 to 15 (mean [ SD ] = 6.22 ERB_16 .507 [5.47]).

Note. Rotated component matrix is shown. Extraction Th ese results provide initial evidence method = principal component analysis; rotation method of the predictive validity of the model, as = equamax with Kaiser normalization. the PSC Index negatively predicted the number of hospital violations: the higher in our model: (1) ASST, (2) LMS, (3) RWE, the PSC Index score, the lower the number and (4) ERB. of violations ( R2 = .459, p = .045, standardized beta = −.678). Moreover, two of Confi rmatory Factor Analysis the PSC variables (ERB and RWE) were and the PSC Index negatively correlated with the number of We found that the model fi t the data well. violations reported, as shown in Table 1 . Figure 2 presents the path diagram of a confi rmatory factor analysis with four fac- Comparing Our Results With the tors and the latent variable “patient safety Leapfrog Group Hospital Survey Data climate.” Goodness of fi t indexes (GFIs) Th e Leapfrog Group (2015) measures 28 suggest a robust model with four variables safety aspects, 10 positive and 18 nega- and a latent variable corresponding to the tive. Data were available for 20 hospitals in overall construct “patient safety climate”: our sample. We tested all data in the study,

FIGURE 1 Items From the California Patient Safety Climate Survey

LMS_01. Management consistently encourages everyone to report any errors or unsafe work behavior. ASST_02. In this organization, we do not hesitate to notify management or coworkers that we need help. ERB_03. Staff usually report their bad decisions to their managers and teams. ERB_04. This organization provides sufficient resources for staff to report problems with the safety of patients. ASST_05. The training we receive appropriately prepares us for our duties. ASST_06. This organization is dedicated to assisting staff in improving the quality of service delivered. LMS_07. The CEO and managers of this organization emphasize the importance of practicing safety measures. ASST_08. Most employees are fully aware of all the appropriate channels of communication between coworkers and management. ASST_09. In this organization, staff and managers are readily available for assistance. ASST_10. When staff members are unsure of policies or assignments, they usually consult their managers or coworkers. ASST_11. When I am not sure what to do, I can easily approach management and coworkers for assistance. RWE_12. This organization frequently offers training to help improve employees’ skills. RWE_13. This organization instills confidence in its staff. RWE_14. This organization encourages all employees to attend nonmandatory training. LMS_15. There is a policy for dealing with medical errors, patient complaints, staff concerns, and patient safety. ERB_16. Coworkers usually report their errors and mistakes to their managers and teams. RWE_17. The training we receive in this organization meets my expectations for patient safety. ASST_18. People who work here get enough assistance to ensure patient safety. LMS_19. This organization has a clear policy on reporting medical errors. ERB_20. In this organization, all employees are informed about any errors or mistakes that occur at the workplace. ERB_21. People here are not afraid of reporting errors or patient complaints. LMS_22. Management has made it clear how to report medical errors, patient complaints, and patient safety.

including the Leapfrog data, for skewness collapsed lungs, death from serious treat- and kurtosis. Eight of the Leapfrog Group able complications, urinary tract infection measures indicated skewness and kurtosis during a stay in the intensive care unit within the acceptable range of −2.0 to +2.0, (ICU), surgical site infection aft er colon and were retained; all other measures were surgery, and dangerous bed sores. We also excluded from the study. We standard- included three positive items in a scale we ized the Leapfrog scores to overcome the called “LF Positives” ( α = .62); these items problem of diff erent measurement scales, were specially trained physicians for patients typically representing the number of times in the ICU, eff ective leadership to prevent a problem occurred per a specifi ed number errors, and physicians order medication via of patients. We used mean z scores to cre- a computer. ate two variables. Five negative items were Th e PSC Index negatively predicted LF included in a scale we called LF Problems Problems (R 2 = .228, p = .033; standardized ( α = .73); these included problems with beta = −.477). While the direction of the

FIGURE 2 Confi rmatory Factor Analysis, Patient Safety Climate

Note. Goodness of fi t index = .998; normed fi t index = .997; root mean square error of approximation = .000; root mean square residual = .005.

relationship between the PSC Index and continually assess their organizational LF Positives was positive, the equation climates and track changes over time, as was not signifi cant. Two PSC variables well as in comparison with those of other were negatively correlated with LF Prob- organizations. lems: RWE ( r = −.616, p = .004) and ERB Healthcare managers can use the four ( r = −.606, p = .005). Th e PSC variable PSC dimensions to identify strengths and RWE was positively correlated with LF weaknesses in their organization’s PSC. Positives ( r = −.489, p = .029). Consistent with the LCOT framework of Kash et al. (2013) , they also can assess the DISCUSSION organization’s cultural readiness for change We developed a short, general, organiza- toward a safer environment. tional-level measure that may be used to We share the vision of researchers and assess PSC in healthcare organizations. practitioners that healthcare organiza- Results indicate that the four-factor PSC tions will continue to use multiple means model is reliable and a good fi t with the of evaluating PSC. For instance, AHRQ data from the 22-item Cal-PSCS. Th ese off ers several patient safety culture survey fi ndings also off er initial evidence of the tools for diff erent healthcare settings (e.g., predictive validity of the model. Given the hospitals, medical offi ces, pharmacies) importance of patient safety and the cost and a number of non-English versions for to healthcare organizations of poor patient international use. AHRQ’s surveys are used safety outcomes, healthcare leaders must around the world, alongside other tools to

aid in planning or evaluating patient safety of climate as a fitting descriptor). Within programs (Parker et al., 2015). In revising these limitations, we are encouraged the core HSOPSC survey, AHRQ acknowl- by the positive signs of support for our edges challenges in administering fairly approach, as reflected in organization- complex or lengthy assessments, translat- level reports from CMS and The Leapfrog ing survey items for international use, and Group. Although we were unable to share achieving uniformity of interpretation our findings with all organizations in the across clinical and nonclinical personnel study, most study informants previewed (AHRQ, 2016). the organization-specific results, and their Farup (2015) compared the admin- feedback has been positive. istration of a Norwegian version of the In their systematic literature review, HSOPSC in two similar medical depart- Hessels and Larson (2016) identified mul- ments, and found that a higher number tiple studies from 2000 to 2014 suggesting of patients experienced adverse events in that PSC survey factors relate positively to the department with significantly better healthcare worker behaviors, and that PSC patient safety culture scores (independently improvements may enhance adherence to measured by a review of patient records). standard precaution practices in acute care This unexpected finding prompted Farup hospitals. Hessels and Larson reviewed high- to question “the reliability and validity of quality research with “data from self-report the tools used for measuring the patient surveys including validated PSC measures safety culture and the adverse events” or measures of management support and (p. 3). At the same time, this finding con- leadership” (p. 349). As a self-report sur- firms the importance of work context and vey with validated psychometrics, our PSC robust psychometric properties of the tools instrument—when used in a continuum of used for interpretation of results. patient safety improvement efforts—might, In developing the Cal-PSCS, we therefore, also be helpful in improving the took into consideration the challenges of safety of healthcare work environments. complexity and uniformity of appeal as We look forward to future research well as the challenges of establishing clear that may lead to additional evidence of the boundaries for the organizational settings instrument’s validity. We also look forward and levels of analysis. While the AHRQ to generating enough data to provide use- surveys may be more useful in planning ful benchmarks for comparison over time comprehensive patient safety programs within the same organization, as well as (Jones, Skinner, Xu, Sun, & Mueller, 2008), with a set of external organizations. Con- our Cal-PSCS can be used by a healthcare trary to expectations, we found only one organization for a quick diagnosis of its significant difference between hospitals current, overall PSC. We envision that and clinics: the LMS variable (mean differ- our instrument will be used primarily in ence = .537, p = .004; Table 3). Although U.S.-based healthcare settings, and recom- this finding suggests that our instrument mend it be administered for the purpose may be applicable to both types of orga- of attaining an organization-level snapshot nizations, significant differences between of the patient safety culture (hence, use hospitals and clinics may be revealed in

TABLE 3 Mean Differences Between Hospitals and Clinics Mean Variable N M SD diff erence p Value PSC Index Hospitals 30 73.1 13.9 4.933 .235 Clinics 31 68.2 18.0 ASST Hospitals 30 4.03 .645 .0127 .944 Clinics 31 4.02 .76 LMS Hospitals 30 4.43 .46 .537 .004 Clinics 31 3.9 .89 RWE Hospitals 30 3.86 .694 .395 .080 Clinics 31 3.46 1.01 ERB Hospitals 30 3.37 .83 −.156 .445 Clinics 31 3.53 .76 Tenure Hospitals 30 11.61 10.32 3.568 .094 Clinics 31 8.04 4.86

Note. N = total number; M = mean; SD = standard deviation.

future studies with a larger sample. Specifi c satisfaction, trust, teamwork, and organiza- to the LCOT model’s key goal of measur- tional commitment ( Valentine, Nembhard, ing absorptive capacity for change, we are & Edmondson, 2015 ; Zohar, 2010 ). intrigued with the continued discussion about the value of diff erentiating climate CONCLUSION based on PSC level (i.e., averaged score) Care quality and legal and fi nancial and strength (i.e., degree of agreement imperatives place increasing pressure on between responders) ( Ginsburg & Gilin healthcare leaders to assess periodically Oore, 2016 ). Evaluating the strength of a the organization’s PSC. We developed a climate may be particularly important for simple U.S.-based diagnostic instrument understanding the depth and breadth of to accomplish that in various healthcare change interventions needed to transform settings. Th e instrument can be used as a an underlying culture. gauge of an organization’s overall PSC in a Future studies with larger data sets and stable practice environment or as an assess- additional measures of safety outcomes are ment tool during organizational change. needed. Researchers also might include Although we noted the instrument’s limita- other variables that have been shown tions, the strength of our PSC survey’s to relate to a safety climate, such as job psychometric properties is reinforced by

the preliminary signs of validation from Auer, C., Schwendimann, R., Koch, R., De Geest, CMS and The Leapfrog Group reports. We S., & Ausserhofer, D. (2014). How hospital encourage healthcare leaders to use a vari- leaders contribute to patient safety through the ety of PSC assessment measures in light development of trust. Journal of Nursing Admin- istration, 44(1), 23–29. of their association with, and the critical Birk, S. (2015). Accelerating the adoption of a safety importance of, patient safety outcomes. culture. Healthcare Executive, 30(2), 18–26. Bronkhorst, B., Tummers, L., Steijn, B., & Vijverberg, Acknowledgments D. (2015). Organizational climate and employee The authors are grateful for administrative mental health outcomes: A systematic review of and other assistance from Diana Alkatib, studies in health care organizations. Health Care Management Review, 40(3), 254–271. Serj Eyvazian, Liza Hamid, Chia-Chen Lan, Christian, M. S., Bradley, J. C., Wallace, J. C., & Nirali Desai, Erika Rosby, Armen Saradjian, Burke, M. J. (2009). Workplace safety: A meta- and other former students in the Master of analysis of the roles of person and situation Science in Healthcare Management program factors. Journal of Applied Psychology, 94(5), at California State University, Los Angeles, 1103–1127. and the Doctor of Nursing Practice program Colla, J., Bracken, A., Kinney, L., & Weeks, W. (2005). Measuring patient safety climate: A offered by the California State University review of surveys. Quality and Safety in Health campuses at Fullerton, Long Beach, Los Care, 14(5), 364–366. Angeles, and Northridge. These students Farup, P. G. (2015). Are measurements of patient assisted with the study design and data col- safety culture and adverse events valid and reli- lection performed by Dr. McGuire. Students able? Results from a cross sectional study. BMC from these programs continue to provide Health Services Research, 15, 186. Gamm, L., Kash, B., & Bolin, J. (2007). Organiza- invaluable contributions to the authors’ tional technologies for transforming care: Mea- understanding of patient safety in practice. sures and strategies for pursuit of IOM quality aims. Journal of Ambulatory Care Management, 30(4), 291–301. REFERENCES Ginsburg, L., & Gilin Oore, D. (2016). Patient Agency for Healthcare Research and Quality safety climate strength: A concept that requires (AHRQ). (2016). Update on the Hospital Survey more attention. BMJ Quality & Safety, 25(9), on Patient Safety Culture and Health IT Patient 680–687. Safety supplemental items. Retrieved from Ginsburg, L. R., Tregunno, D., Norton, P. G., Mitch- http://www.ahrq.gov/professionals/quality- ell, J. I., & Howley, H. (2014). Not another safety patient-safety/patientsafetyculture/hospital/ culture survey: Using the Canadian patient update/index.html safety climate survey (Can-PSCS) to measure Agnew, C., Flin, R., & Mearns, K. (2013). Patient provider perceptions of PSC across health set- safety climate and worker safety behaviours in tings. BMJ Quality & Safety, 23(2), 162–170. acute hospitals in Scotland. Journal of Safety Hacker, K., & Walker, D. K. (2013). Achieving Research, 45, 95–101. population health in accountable care organiza- Armstrong, K., Laschinger, H., & Wong, C. (2009). tions. American Journal of Public Health, 103(7), Workplace empowerment and Magnet hospital 1163–1167. characteristics as predictors of patient safety climate. Hazy, J. (2008). Patterns of leadership: A case study Journal of Nursing Care Quality, 24(1), 55–62. of influence signaling in an entrepreneurial firm. Association of Health Care Journalists. (2015). In M. Uhl-Bien & R. Marion (Eds.), Complexity Search hospital inspections. Retrieved from leadership: Conceptual foundations (pp. 379–415). http://www.hospitalinspections.org Charlotte, NC: Information Age.

Hessels, A. J., & Larson, E. L. (2016). Relationship Kurtzman, J. H. (2015). A community hospital– between patient safety climate and standard county health department partnership to reduce precaution adherence: A systematic review of preventable readmissions: Lessons learned the literature. Journal of Hospital Infection, 92(4), for population health management. Journal of 349–362. Healthcare Management, 60(4), 258–267. Holden, R. J. (2011). What stands in the way of Landers, R. N. (2011, Nov. 16). Computing intra- technology-mediated patient safety improve- class correlations (ICC) as estimates of interra- ments? A study of facilitators and barriers to ter reliability in SPSS. [Web log post]. Retrieved physician’s use of electronic health records. from http://neoacademic.com/2011/11/16/com- Journal of Patient Safety, 7(4), 193–203. puting-intraclass-correlations-icc-as-estimates- Hughes, L. C., Chang, Y., & Mark, B. A. (2009). of-interrater-reliability-in-spss/#.VYo31UbGq-4 Quality and strength of patient safety climate in Leapfrog Group. (2015). Leapfrog hospital safety medical-surgical units. Health Care Management grade. Retrieved from http://www.hospitalsafe- Review, 34(1), 19–28. tyscore.org Institute of Medicine. (2000). To err is human: Levitt, P. (2014, March 16). When medical errors kill. Building a safer health system. Washington, DC: Los Angeles Times, p. A24. National Academies Press. McFadden, K. L., Stock, G. N., & Gowen, C. R., III. Jackson, J., Sarac, C., & Flin, R. (2010). Hospital (2015). Leadership, safety climate, and continu- safety climate surveys: Measurement issues. Cur- ous quality improvement: Impact on process rent Opinion in Critical Care, 16(6), 632–638. quality and patient safety. Health Care Manage- James, J. (2013). A new, evidence-based estimate ment Review, 40(1), 24–34. of patient harms associated with hospital care. Modak, I., Sexton, J. B., Lux, T. R., Helmreich, R. L., Journal of Patient Safety, 9(3), 122–128. & Thomas, E. J. (2007). Measuring safety culture Jones, K. J., Skinner, A., Xu, L., Sun, J., & Mueller, K. in the ambulatory setting: The safety attitudes (2008). The AHRQ Hospital Survey on Patient questionnaire—ambulatory version. Journal of Safety Culture: A tool to plan and evaluate General Internal Medicine, 22(1), 1–5. patient safety programs. In K. Henriksen, J. Naylor, M. D., Aiken, L. H., Kurtzman, E. T., Olds, D. B. Battles, M. A. Keyes, & M. L. Grady (Eds.), M., & Hirshman, K. B. (2011). The importance Advances in patient safety: New directions and of transitional care in achieving health reform. alternative approaches (Vol. 2). Culture and Health Affairs, 30(4), 746–754. redesign. Rockville, MD: Agency for Healthcare Nunnally, J. C. (1978). Psychometric theory (2nd ed.). Research and Quality. New York, NY: McGraw-Hill. Kash, B. A., Spaulding, A., Gamm, L. D., & John- Ostroff, C., Knicki, A. J., & Tamkins, M. M. (2003). son, C. (2013). Leadership, culture, and orga- Organizational culture and climate. In W. C. nizational technologies as absorptive capacity Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), for innovation and transformation in the Handbook of psychology (Vol. 12, pp. 565–593). healthcare sector: A framework for research. New York, NY: Wiley. Change Management: An International Journal, Parker, D., Wensing, M., Esmail, A., & Valderas, J. M. 13(1), 1–13. (2015). Measurement tools and process indicators Keyton, J. (2014). Communication, organizational of patient safety culture in primary care. A mixed culture, and organizational climate. In B. methods study by the LINNEAUS collaboration Schneider & K. M. Barbera (Eds.), The Oxford on patient safety in primary care. European Jour- handbook of organizational climate and culture nal of General Practice, 21(1), 26–30. (pp. 118–135). Oxford, UK: Oxford University Schein, E. H. (2010). Organizational culture and leader- Press. ship (4th ed.). San Francisco, CA: Jossey-Bass. Kozlowski, S. W., & Hattrup, K. (1992). A disagree- Schneider, B., & Reichers, A. E. (1983). On the etiol- ment about within-group agreement: Disen- ogy of climates. Personnel Psychology, 36, 19–39. tangling issues of consistency versus consensus. Schneider, B., White, S. S., & Paul, M. C. (1998). Journal of Applied Psychology, 77(2), 161–167. Linking service climate and customer

perceptions of service quality: Tests of a causal settings: A review of survey instruments. Medi- model. Journal of Applied Psychology, 83(2), cal Care, 53(4), 16–30. 150–163. VMIA. (2015). Patient safety climate. Retrieved from Singer, S. J., Falwell, A., Gaba, D. M., Meterko, M., https://www.vmia.vic.gov.au/risk/risk-tools/ Rosen, A., Hartman, C. W., & Baker, L. (2009). patient-safety-climate Identifying organizational cultures that promote Wachter, R. M. (2012). Understanding patient safety patient safety. Health Care Management Review, (2nd ed.). San Francisco, CA: McGraw-Hill. 34(4), 300–311. Waterson, P. (2014). Patient safety culture: Theory, Singer, S. J., Gaba, D. M., Falwell, A., Lin, S., Hayes, methods and application. Burlington, VT: J., & Baker, L. (2009). Patient safety climate in Ashgate. 92 US hospitals: Differences by work area and Weaver, S. J., Lubomksi, L. H., Wilson, R. F., Pfoh, E. discipline. Medical Care, 47(1), 23–31. R., Martinez, K. A., & Dy, S. M. (2013). Promot- Singer, S., Meterko, M., Baker, L., Gaba, D., Falwell, ing a culture of safety as a patient safety strategy: A., & Rosen, A. (2007). Workforce percep- A systematic review. Annals of Internal Medicine, tions of hospital safety culture: Development 158(502), 369–374. and validation of the Patient Safety Climate in Weick, K. E. (2012). Making sense of the organiza- Healthcare Organizations survey. Health Services tion: The impermanent organization (Kindle ed.). Research, 42(5), 1999–2021. Thousand Oaks, CA: Sage. Singer, S. J., & Vogus, T. J. (2013). Safety climate Zaheer, S., Ginsburg, L., Chuang, Y.-T., & Grace, S. L. research: Taking stock and looking forward. BMJ (2015). Patient safety climate (PSC) perceptions Quality & Safety, 22, 1–4. of frontline staff in acute care hospitals: Examin- Smith-Crowe, K., & Burke, M. J. (2003). Interpret- ing the role of ease of reporting, unit norms of ing the statistical significance of observed AD openness, and participative leadership. Health interrater agreement values: Correction to Burke Care Management Review, 40(1), 13–23. and Dunlap (2002). Organizational Research Zahra, S., & George, G. (2002). Absorptive capacity: A Methods, 6(1), 129–131. review, reconceptualization, and extension. Acad- Soh, S.-E., Barker, A., Moreno, R., Dalton, M., & emy of Management Review, 27(2), 185–203. Brand, C. (2016). Measuring safety climate in Zohar, D. (1980). Safety climate in industrial orga- acute hospitals: Rasch analysis of the safety nizations: Theoretical and applied implications. attitudes questionnaire. BMC Health Services Journal of Applied Psychology, 65(1), 96–102. Research, 16, 497–506. Zohar, D. (2003). The influence of leadership and Stock, G. N., McFadden, K. L., & Gowen, C. R., III. climate on occupational health and safety. In (2007). Organizational culture, critical success D. A. Hofmann & L. E. Tetrick (Eds.), Health factors and the reduction of hospital errors. and safety in organizations: A multi-level International Journal of Production Economics, perspective (pp. 201–230). San Francisco, CA: 106(2), 368–392. Jossey-Bass. Stock, G. N., McFadden, K. L., & Gowen, C. R., Zohar, D. (2010). Thirty years of safety climate III. (2010). Organizational culture, knowledge research: Reflections and future directions. management, and patient safety in U.S. hospitals. Accident Analysis and Prevention, 42(5), Quality Management Journal, 17(2), 7–26. 1517–1522. Toussaint, J. S. (2015). The framework for a whole- Zohar, D., & Hofmann, D. A. (2012). Organizational system transformation. Journal of Healthcare culture and climate. In S. W. Kozlowski (Ed.), Management, 60(6), 386–389. Oxford handbook of industrial and organizational Valentine, M. A., Nembhard, I. M., & Edmondson, psychology (Vol. 1, pp. 643–666). New York, NY: A. C. (2015). Measuring teamwork in health care Oxford University Press.