April 2017

Guide to Monitoring and Improving Safety

Generated by the Integrated Safety Management Working Group

Energy Facility Contractors Group

Guide to Monitoring & Improving Safety Culture Page i

Guide to Monitoring and Improving Safety Culture

Approval: Guide to Monitoring and Improving Safety Culture is approved by Energy Facility Contractors Group (EFCOG) and recommended for use by all member contractors.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page ii

ACRONYMS AND ABBREVIATIONS

BP British Petroleum

CSB U.S. Chemical Safety Board

CAIB Columbia Accident Investigation Board

DOE U.S. Department of Energy

EFCOG Energy Facility Contractors Group

ISM Integrated Safety Management

ISMS Integrated Safety Management System

KPI key performance indicators

NTSB National Transportation Safety Board

NEI Nuclear Energy Institute

SCIT safety culture improvement team

SLT senior leadership team

SME subject matter expert

WMATA Washington Metropolitan Area Transit Authority

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page iii

CONTENTS

LIST OF FIGURES ...... v

1.0 Introduction ...... 1

2.0 Background ...... 1

2.1 Integrated Safety Management System and Safety Culture ...... 3

2.2 Purpose of Document ...... 3

3.0 Overview ...... 4

4.0 Leadership Team ...... 4

5.0 Safety Culture Improvement Team ...... 6

5.1 Team Purpose...... 6

5.2 Team Composition ...... 6

5.3 Emergent Issues ...... 6

5.4 Team Processes ...... 7

6.0 Building Performance Indicators ...... 8

6.1 Introduction ...... 8

6.2 Organizational Objectives and Safety Culture Attributes ...... 9

6.3 Sources of Data ...... 9

6.3 Performance Indicators ...... 10

6.4 Quantitative and Qualitative Methods ...... 10

7.0 Assessing Performance Indicators and Recommending Improvements ...... 11

8.0 Safety Culture Improvement Initiatives ...... 14

8.1 Build on Strengths...... 15

8.2 Work on Cultural Areas that Inhibit Success ...... 16

8.3 Improve Work Environment ...... 17

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page iv

9.0 Evaluating Effectiveness of Change Initiatives ...... 18

10.0 Safety Culture Communication...... 19

10.1 SCIT Communication ...... 20

10.2 Organizational Communication ...... 22

10.3 External Communication ...... 22

11.0 Effectiveness of the SCIT ...... 23

12.0 Records ...... 24

13.0 References ...... 24

Appendix A ...... 26

Potential Sources of Data for Safety Culture Monitoring ...... 26

Appendix B ...... 27

Potential Performance Indicators and Metrics ...... 27

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page v

LIST OF FIGURES

Figure 1. Example Process Flow for Safety Culture Improvement 8

Figure 2. Eight Steps to Transform Your 15

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page i

In September 2015, the Energy Facility Contractors Group (EFCOG) Integrated This guide is intended to offer Safety Management (ISM) Working Group suggestions and examples of how an issued A Guide to Safety Culture Evaluation. organization might approach safety This guide is a practical source of culture monitoring and improvement. information on conducting a safety culture Not all recommendations or examples in evaluation including planning, data this document will be appropriate for all collection, data analysis, reporting, and ; organizations should use continuous improvement. The companion their best judgment on their own document, Safety Culture Monitoring & implementation. Likewise, this Improvement, provides DOE contractors document should not be used as a with guidance on how to monitor culture guideline for judging an organization’s changes between assessments and how to implementation of safety culture continually improve their organization’s monitoring. As such, prescriptive safety culture. language has been avoided to promote flexibility across a variety of settings. To be most effective and efficient, the 2.0 Background culture monitoring process should leverage existing structures and Failure to monitor an organization’s safety processes as much as possible. As the culture can have catastrophic consequences. culture monitoring process matures, Three high-profile accidents illustrate the organizations may find that key features importance of establishing an effective of the monitoring process can be safety culture. As these tragedies attest, accomplished more efficiently within management involvement is critical for other existing processes. developing an effective culture. When management fails to monitor safety culture,

significant property loss and the irreplaceable loss of life can occur. 1.0 Introduction 2009 Washington, D.C. Metro Instituting processes to monitor and sustain collision (NTSB. 2010) – On June 22, an organization’s focus on safety culture 2009, Washington (D.C.) Metropolitan makes good business sense, particularly Area Transit Authority (WMATA) given the hazardous and complex missions Metrorail Train 112 struck the rear of a present across the U.S. Department of Metrorail train that had stopped on the Energy (DOE). These processes help ensure track. The accident cost the lives of the that the investment in safety culture train’s operator and eight passengers, activities is having the desired effect. and a total of fifty-two people were Improving safety culture enhances transported to hospitals due to injuries organizational performance, which translates suffered in the crash. The National to meeting organizational goals and Transportation Safety Board (NTSB) accomplishing the mission. investigation revealed that among the

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 2

contributing causes of the accident were 2003, the Space Shuttle Columbia broke WMATA’s poor safety culture and the up upon re-entry into the Earth’s failure of an automatic train control atmosphere killing Columbia’s seven- system. The NTSB chairman member crew. In response to the characterized the WMATA safety accident, more than 25,000 individuals culture as “anemic” at a public hearing worked to recover debris from the regarding the crash. The poor safety accident, which was strewn across culture was exemplified in part by several Western states. In response to the WMATA Metrorail managers’ failure to , the Columbia Accident address problems with the train control Investigation Board (CAIB) was system. Consequently, the NTSB report convened to ascertain direct and indirect concluded that management’s failure to causes. The CAIB reported that while appropriately prioritize safe operations the physical cause of the disaster was a “likely influenced the inadequate breach in Columbia’s Thermal response to such malfunctions by Protection System, there were important automatic train control technicians, organizational causes as well. The CAIB operations control center controllers, and reported that “cultural traits and train operators.” organizational practices detrimental to safety were allowed to develop. These 2005 BP refinery explosion (U.S. included: organizational barriers that Chemical Safety Board, 2007) - On prevented effective communication of March 23, 2005, explosions and fires at critical safety information and stifled the British Petroleum (BP) refinery in professional differences of opinion; lack Texas City, Texas, killed fifteen people of integrated management across and injured another 180. The U.S. program elements; and the evolution of Chemical Safety Board (CSB) an informal chain of command and investigated the causes of the accident decision-making processes that operated and identified systemic organizational outside the organization’s rules.” causes in addition to the specific technical causes. Among these Each of these events is directly linked to organizational causes were significant leadership failure to establish a strong safety management failures, including the culture. Management must enhance failure of BP executive management to communication regarding safety, ensure that “implement adequate safety oversight, project management is appropriately provide needed human and economic integrated, and insist that safety rules are resources, or consistently model followed. Management must demonstrate adherence to safety rules and that it values safety and will not tolerate procedures.” Further, BP executive informal processes that circumvent safe management and refinery management operations. Ultimately, management must did not create a positive learning and create and maintain a robust safety culture reporting culture, which emphasizes the that is resilient to organizational drift. importance of reporting safety threats and effectively investigating accidents. 2003 Columbia breaks up (National Aeronautical and Space Administration, 2003) - On February 1,

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 3

2.1 Integrated Safety decades. The DOE ISMS safety culture Management System and Safety focus areas are: Culture • Leadership • Employee Engagement The purpose of every DOE organization is • . to successfully accomplish its assigned Organizational Learning mission while working within an approved The attributes for each safety culture focus Integrated Safety Management System area promote a shift from mere compliance (ISMS). This system includes the to continuous improvement in safety and implementing mechanisms, processes, and production performance, with continual methods to be used to systematically adjustments to stay within the approved integrate safety into management and work ISMS. The three focus areas and their practices at all levels in the planning and attributes are the foundation upon which this execution of work. It is at this juncture Guide was developed. between how work is planned and how work is performed where the importance of safety 2.2 Purpose of Document culture lies. This document provides a guide for Safety culture represents the collective continuously monitoring safety culture so response of an organization to its work organizations can take appropriate actions to environment. An organization with a improve their safety performance and healthy safety culture proactively seeks effectively accomplish their mission. feedback, responds to issues openly, and This guide is based, in part, on guidance engages stakeholders to stay within its described in documents generated by approved ISMS. nuclear organizations including Fostering a Because of the relationship between safety Strong Nuclear Safety Culture (Nuclear culture and mission success, DOE has Energy Institute, NEI 09-07, Revision 1), committed to the safety culture focus areas Safety Culture (International Atomic Energy and attributes (also called “traits” in other Agency Safety Series No. 75-INSAG-4), industries) described in Attachment 10 of its Guide to Safety Culture Evaluation (Energy Integrated Safety Management System Facilities Contractors Group, 2015), Best Guide (U.S. Department of Energy, DOE G Practice #181 (Energy Facilities Contractors 450.4-1C). The Guide defines safety culture Group, 2015), other contractor best as: practices, and related literature on organizational management and culture “An organization’s values and behaviors change. modeled by its leaders and internalized by its members, which serve to make safe Because an organization’s safety culture is performance of work the overriding priority influenced by the work environment, the to protect the workers, public, and the culture may shift as the work environment environment.” changes. Ongoing internal monitoring is a means to self-identify problems and plan Attachment 10 further describes three safety specific improvement actions prior to culture focus areas that are based on a joint external assessments. DOE-EFCOG initiative that began in 2007 and included commercial nuclear industry experience and research over several

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 4

3.0 Overview “In our zeal to quantify, analyze, Although safety culture cannot be measured systematize, and proceduralize, we risk directly, there are organizational performance indicators and associated overlooking an essential truth; culture organizational behaviors, which, if properly is but a construct, a lens through which correlated and interpreted, can be used to we may notice and contemplate our provide indicators of potential weaknesses fuental humanity.” that could contribute to failure or to strengths that could be applied to other areas Culture is not a property, not a set of and initiatives. (Cole, et al 2013) The ability attributes we can manipulate, dissect and to proactively identify weak signals before they become a factor in a significant reconstruct. Culture is rather ‘the organizational event can be of great benefit. medium of lived experience’- a manifestation of relationships, The key to safety culture monitoring is to collect qualitative and quantitative data and psychological processes, and understand the relationship between human communication - a resultant not an behavior and the observed results. If the antecedent. The discussion of safety operational performance data are trending culture is but our most recent attempt to positively, one would ask, “What are the understand the human relationships with behaviors people in the organization are our technologies, how we create them, exhibiting that should be reinforced?” If the operational performance data are declining, how they in turn shape us. It is not a one would ask, “What are the undesirable linear predictable projection, rather an behaviors that should be modified?” This eternal dance of discovery and effort may provide insight on the ability of reinvention. As we seek to understand the organization to self-identify, to report, how we as technical professionals co- and to resolve problems. Learning create technical marvels to improve the opportunities may be identified through human condition, let us not forget that we issues management and/or contractor assurance systems and from external reports, design technology so it may serve us, not including DOE assessments and corporate that we may serve technology. Let us and industry evaluations. keep humanity as our focus, careful that we not reduce that which makes us human to some mechanistic model, and 4.0 Leadership Team always honor the mystery of who we are and how we together create our To successfully transform performance data into safety culture insight, an organization experiences. must have leadership that understands and appreciates the connection between a Earl Carnes, DOE Safety Culture & healthy safety culture and mission success. High Reliability Organization Advisor They must actively participate in, and (retired) consistently lead the organization toward, improved performance and safety culture.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 5

and other consequences. Leadership must oversee the development and deployment of To achieve this, organizational leaders need management systems and relentlessly drive to: to obtain accurate, timely, and continuous feedback on the health of management 1. Understand that observed safety systems. behaviors are directly influenced by the organization’s work The Senior Leadership Team (SLT) is environment. defined as the most senior management personnel onsite charged with the safe 2. Understand the value of a safety operation of the organization. For example, culture baseline assessment. the SLT could include high level functional 3. Understand how the practices of area leaders in areas such as research, their organization influence the operations, support, maintenance, and operating environment. human resources. 4. Collect representative operational The SLT reviews safety culture data and performance data characterizing key recommendations to determine if action is facets of the approved ISMS so when necessary. This input may be from a performance degrades, indicators of dedicated resource (such as a social science drift can be detected. subject matter expert), existing data 5. Compare the operational data across reporting systems (such as a company-wide the safety culture attributes to dashboard), or a Safety Culture compare performance to behaviors Improvement Team (SCIT). SLT (determine the health of the interactions should occur in a group setting organization’s safety culture). to promote reflective conversation about safety culture. The SLT should gain a 6. Directly observe the performance of thorough understanding of the work to get a sense of the work organization’s safety culture, which serves environment and related behaviors of as the basis for their decision-making. various work groups. 7. Assimilate the above items and have The SLT members should share their own open discussions about the effect of interactions with organization personnel, the results on operational success so field observations, and other individual that realistic improvement actions experiences to help the SLT understand the can be recommended. organization’s safety culture. The most valuable insight often comes from frank Leaders communicate and demonstrate their discussion of safety culture based on the commitment to safety and reliable SLT members’ observations and insights. operations through their words and their actions. Without leadership’s commitment, a The SLT’s periodic review of safety culture healthy safety culture is not possible. should be documented. Follow-up actions Leadership must ensure the management should be tracked (e.g., through an issues systems and procedures provide the required management system or other means). level of safety, security, and quality while Strengths and improvement opportunities simultaneously avoiding failure mechanisms should be communicated to the organization

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 6

to promote desired behaviors and foster 5.2 Team Composition improvement actions. SCIT members should represent a cross section of functional areas, be familiar with daily work activities, and have knowledge of 5.0 Safety Culture Safety Culture principles. The team must have a common understanding of Improvement Team organizational goals and objectives and an Organizations seeking to promote and appreciation of how safety culture improve their safety culture may establish a influences operational performance. The Safety Culture Improvement Team (SCIT). SCIT must be trusted by the workforce and A number of names may be used for this management. Team members should have team (e.g., safety culture monitoring panel, broad-based operational experience, and safety culture working group). The term some team members should have the ability SCIT is used in this document. to collect, analyze, and interpret data. It is strongly advised to train the team on the 5.1 Team Purpose content of Attachment 10 (or equivalent), The SCIT proactively monitors performance methods of changing behavior and culture and processes inputs to identify emerging and conducting culture evaluations. If an challenges and opportunities for organization is not using a SCIT, a social improvement. The level of effort and scientist/organizational development formality used to conduct culture monitoring professional may help identify and analyze and periodic reviews should be tailored to available data to infer safety culture trends. the needs of the organization and be proportional to size, budget, and mission. A complex organization with multiple high risk facilities would likely benefit from a Allow time for the SCIT team to more structured approach, whereas a smaller agree on the most important areas for organization could adopt a less formal monitoring and improvement and watch approach. membership for attrition and fatigue. Likewise, management commitment is necessary to ensure SCIT members have There is no “one-size-fits-all” adequate time allocated to monitoring to monitoring and improving duties to prevent burnout. safety culture. SCITs, also called “Safety Culture Monitoring Panels” in NEI 09-07, are a best practice, but culture performance monitoring can be 5.3 Emergent Issues performed by dedicated safety culture Emergent issues may arise between personnel. Other existing avenues may meetings of the SCIT. These could be be used for monitoring or trending, such externally or internally generated issues that as a Trending team. Organizations indicate dissatisfaction with, for example, should perform their own evaluation of the organization’s safety focus, whether a SCIT is the best option for responsiveness, corrective action program, their own monitoring processes. or treatment of personnel. The SCIT ensures that such issues are brought to the attention of the SLT and the organization’s applicable

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 7

internal processes (e.g., differing professional opinions). 5.4 Team Processes The SCIT assesses and seeks to improve safety culture with the goal of improving mission performance and reducing risk. To accomplish this it: • Analyzes data to determine performance areas to focus on (e.g., “What” is not working to expectations of published safety management systems). • Uses walkarounds and two-way communication to directly observe behaviors related to performance issues (the “why” behind the “what”). • Compares observed behaviors to desired behaviors (as defined by safety culture behavior attributes). • Identifies gaps to recommend improvements in the work environment to improve behaviors. • Makes recommendations to the SLT and agrees on actions. • Reviews the effectiveness of work environment changes toward improving the culture. • Communicates the results of data analysis and the improvements to the workforce. Figure 1 depicts the operation of the SCIT. The exact order and who does each step is at the discretion of the organization. The reader is directed to NEI 09-07 for other options for the organization and reporting lines for the SCIT.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 8

Example Process Flow of Safety Culture Improvement Operation

Section 7.0 Assess Culture Changes and Recommend Improvements to Work Environment Section 8.0 Leadership Team & Safety Culture Safety Culture Based on walking work spaces and Safety Culture Safety Culture Improvement Section 6.0 Improvement Team two-way communication, focus Improvement Team Improvement Team assigns Culture prioritizes areas of group comes back with establish approved Team takes a focus group to Monitoring potential underlying recommended improvement and improvement collective look at validate culture/ Process Inputs culture/behavioral justifies based on change in work initiatives and all process inputs behavioral concerns environment that will result in monitor progress on concern desired culture/behavior change • Issues Management implementation • Safety Culture Performance data binned by Assessments safety culture focus area • Work issues • Assist visits • Self assessments • Employee Concerns • Differing Professional Opinions • Enterprise assessments • Organizational health indicators • Metrics Section 10.0 Section 9.0 • Etc. Leadership Team & Safety Culture Leadership Team & Safety Culture Improvement Team communicate Improvement Team evaluate results, and improvement actions and effectiveness of behavior/culture effectiveness in improving by revisiting inputs

Note: This flow chart is only example in that it identifies important logic steps the SCIT should accomplish. The exact order and who does each step is left to the discretion of the organization.

Figure 1. Example Process Flow for Safety Culture Improvement

in a continuous cycle, allowing for refinement of goals and objectives as 6.0 Building Performance understanding emerges and is applied. Indicators This section provides guidance on selecting and building indicators that show progress 6.1 Introduction toward meeting safety culture objectives. Performance measurement is an overall Qualitative and quantitative data streams can management system that is not just be used to understand the critical safety concerned with collecting data associated culture objectives of the organization. Once with a goal or standard; a strategic look at selected and built, these indicators may be data available for overall culture context is aggregated to form a portfolio of the necessary. The outcomes of a well- organization’s cultural health and provide constructed safety culture performance insight into cultural strengths and measurement system improve overall opportunities for improvement. Because organizational performance by tracking culture is a reflection of attitudes and progress of strategic organizational initiatives and tracking performance to key Assurance System and Issues Management Program behavioral attributes1. These actions occur ensuring the pedigree, validity, and fidelity of the data input into the SCIT. For DOE contractors, see DOE O 226.1B Implementation of Department of 1For purposes of this document, it is Energy Oversight Policy, Attachment 1, Contractor assumed that the organization has a Contractor Requirements Document.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 9

behaviors, it is not possible to measure Question that performance indicators will culture entirely objectively. Nevertheless, answer: Are leaders effectively ensuring that there are measurable aspects of conditions staff members have sufficient manpower, that can be trended to determine if cultural financial support, and accessibility to issues contributed to the condition. Process information and equipment to do their work weaknesses, discovered through self- with desired quality and safety? assessments, can also provide evidence of possible concerns with the safety culture. 6.3 Sources of Data Similarly, the attitudes and behaviors of What type of information do we need to organization personnel can be assessed collect and how will we collect it? The through surveys, interviews, and behavioral information used to build performance observations. indicators can come from many sources including corrective action programs, 6.2 Organizational Objectives training records, human resources activities, and Safety Culture Attributes safety culture assessments, occurrence Planning is necessary to establish a reports, workforce surveys, etc. Appendix A framework that will provide a provides examples of potential sources of comprehensive understanding of data for safety culture monitoring. performance. Meaningful indicators that Usually, the organizations or individuals that reflect progress towards organizational perform culture monitoring will not own or objectives must be carefully selected. The supply the data and information that SCIT, with input from the SLT, should supports ongoing monitoring efforts. For determine a small number of objectives most example: important to sustaining and strengthening the organization’s culture. • The Human Resources Department or the Employee Concerns Program The actions and behaviors that contribute to may supply some information, while achieving the objectives must be defined. the Safety Department may supply The safety culture attributes contained in other data. As a result, it is important DOE G 450.4-1C, Attachment 10 (or for those responsible for compiling equivalent) should be used to gain this the information to maintain good granularity. When properly constructed, the working relations with the description of the objective and associated organizations that supply the attributes will provide insight into the information, including actively questions that the performance indicators are looking for new or better sources of aimed at answering. For example: information. Organizational objective: Leaders provide support to accomplish work activities. • The Quality organization might develop a performance indicator for Attribute: Leaders ensure that sufficient tracking rates of procedure non- resources have been provided so staff can compliance. If the indicator was part perform their work with distinction. of an existing culture monitoring Resources may include manpower, financial mechanism, it would need to be support, and accessibility to information and evaluated for impact and inclusion equipment. into the aggregate monitoring indicators.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 10

Backup points of contact and succession however, little information on indicators of planning need to be established to maintain safety culture. Report number: 2010:07 repeatability and sustainability of the ISSN: 2000-0456, Indicators of safety performance indicator process, with culture – selection and utilization of leading agreement from the appropriate safety performance indicators by Reiman organizations. and Pietikäinen is a good resource for selecting and using safety culture indicators. Appendix B provides examples of potential indicators for monitoring cultural health. The NEI 09-07 model can be referred to for guidance on what The performance criteria can be determined departments within an organization may once the indicators, the source of data, and have topical data for monitoring and the monitoring method are established. This analysis. It is important to avoid can be done by establishing a goal or overwhelming the safety culture standard or by determining monitoring process with so much volume variance/tolerance bands to represent that the valuable insights are obscured. acceptable or unacceptable performance.

Note: When looking at cultural issues, more salient items might also be telling. For 6.4 Performance Indicators instance, "artifacts" of past processes and Performance indicators should support the programs can provide valuable information safety culture framework and accurately as to how well a program has been portray the organization’s performance. The incorporated into the culture of an framework typically contains both leading organization. These can, of course, be both (prospective) and lagging (retrospective) physical and internal to individuals. For indicators. Leading indicators are important instance are there posters, procedures, for providing an early warning of declining policies, or signs that are out of date, or that performance, and lagging indicators provide have been revised without removing the a description of actual performance previous version? experience. Both have advantages and disadvantages. Effective leading indicators look for missing or degraded barriers 6.5 Quantitative and Qualitative (negative) and evidence that people are demonstrating proactive thinking (positive). Methods It is generally difficult to develop effective Both qualitative and quantitative methods leading indicators that detect subtle declines are necessary to develop a rich in performance that can be easily and understanding of cultural health. These data quickly reversed. Lagging indicators, such are in numeric or narrative form. as recordable injuries, tend to be more Quantitative data (i.e., statistics) strips the standardized and can be compared across emotion from the experience while organizations more easily but are a measure qualitative data (i.e., words) bring of actual historical performance and quantitative data to life. Either or both types therefore are too late to reverse. There is a of data can be used to illustrate the picture large body of published literature describing and suggest the most promising areas for measurement of safety performance, improvement.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 11

Quantitative methods produce data that primarily provide facts or figures that look at the “who, what, where, and when”. These data generate outcomes dealing with 7.0 Assessing knowledge and comprehensions (e.g., define, classify, recall, recognize). An Performance Indicators example of a quantitative method is the and Recommending statistical analysis of survey data. Improvements Qualitative methods produce information Once performance indicators are established with more description by questioning the and data are collected, how will the data be why and how. These generate outcomes analyzed? The SCIT assesses performance dealing with , application, indicator inputs, determines the direction analysis, synthesis, or evaluation. An and magnitude of change, and makes example of a qualitative method is the recommendations accordingly. This is thematic analysis of focus group transcripts. accomplished using the process sequence Although by design, culture insight leans identified in Figure 1. Note that the primary more towards qualitative methods, the use of value is the logic steps that need to be mixed methods is valuable when analyzing accomplished, not the precise order of the cultural data. Obtaining information from steps. How, when, and by whom each logic diverse sources allows for triangulation of step is accomplished is determined by the insights to provide a richer, more informed organization based on its systems. picture of cultural strengths and areas for Step #1: SCIT receives input from the improvement. For example: SMEs from a variety of sources (Section • It is advantageous to have both 6.0). closed- and open-ended questions in a survey; these will produce both qualitative and quantitative outcomes The SMEs provide their opinion of whether and, when integrated, allow for a or not what they are seeing in performance more complete analysis of data is an issue. In some cases, performance information. data changes may indicate process problems • An organization will likely recognize and not necessarily underlying behavior or that an attrition issue is occurring culture problems. and easily able to generate Input data are categorized by culture quantitative data to describe the size attributes (e.g., personal accountability, or impact of the issue. However, the safety communications, questioning attitude, qualitative information obtained leadership accountability, respectful from exit interviews will certainly be , problem identification and of more value when trying to define resolution, etc.). This can be done by the and address contributors to attrition. SMEs collecting the data or by the SCIT. Considering both quantitative and qualitative data will likely be The culture attributes used for categorization necessary to determine the full are determined by the organization (e.g., underlying issues and path forward. ISM Safety Culture Focus Areas, Institute of (Campus Labs, 2017) Nuclear Power Operators Safety Culture

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 12

Traits, etc.). These categories will assure the there has been a change in management completeness of the data set. Data should be focus and direction for raising issues and collected to represent each culture attributes. tracking them in the Corrective Action If data for a particular culture attributes are Management system, or if there is a missing, efforts should be undertaken to challenge to a questioning attitude in the populate that area. organization. Step #2: SCIT examines the data for each When analyzing the results of an indicator organizational objective relative to the using survey or focus group information, it safety culture attribute (Section 7.0). is important to be aware that both negative and positive safety culture indicators can be When analyzing the performance data, the limited to individual departments or work SCIT should review the questions that were groups. This may be due to the influence of originally asked. Does the data answer the a particular manager, line supervisor, or question? How does performance compare strong informal leader. This means that to the goal? demographic information that can be Indicators should be monitored for change, collected without compromising including improvement or decline. If an confidentiality, is extremely important. indicator unexpectedly declines, it may Example: When performing focus groups mean that there is no data or information to and surveys with a random sampling of the track or analyze. This does not mean that organization on the subject of “teamwork culture has declined or that the indicator is and mutual respect” (Employee/Worker no longer valid. However, this should Engagement), polarized results (e.g., 80% prompt some evaluation into what has positive, 20% negative) are best interpreted changed in the process that was supplying via demographic information. In other the source data and whether anything has words, if the 20% of dissatisfied workers are changed in the . concentrated in a specific organization, this Example: When monitoring a “Questioning will have a different resolution than if the Attitude” (Organizational Learning) respondents are spread evenly throughout an indicator comprised of issues raised in the organization or represent a specific Corrective Action Management system, a demographic such as recent college dramatic increase or decrease isn’t graduates or late career professionals. necessarily caused by a culture issue. A Without demographic information, it is decrease in issues should be evaluated to difficult to parse similarities between groups verify that a chilled work environment does of respondents that provide data that are not exist, but the organization should also higher or lower than average. explore whether alternative venues for Note: When soliciting qualitative data, it is raising issues or concerns has not important to bound feedback to a certain experienced an influx. In other words, timeframe (e.g., “in the previous six determine if organizational behaviors have months”) or otherwise framed to address changed, but don’t assume they have culture, since negative experiences deteriorated. Likewise, if there is a sudden from the past may be remembered and increase in issues raised, verify whether this brought up as if they represent current is because of an increase in worker conditions. population (i.e., the number of issues has increased but issues per capita is stable), if

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 13

Step #3: SCIT determines whether or not Step #4: Owner/focus team provides improvement actions or additional recommended improvements with initiatives are needed (Section 7.0). justification (Section 7.0). After the SCIT has reviewed the input data Based on initial monitoring, the SCIT and identified strengths and potential safety prioritizes areas of potential behavior/culture culture challenges and the recommendations concerns. The organization’s performance from the focus groups, it is ready to indicators may indicate the need to dig formulate its recommendations. These deeper on a particular topic or may result in recommendations must be directed at recommendations for improvement, changing the work environment so that a including the type of performance indicators positive behavior change will result in the that should be used, how they should be affected organization. The focus groups selected, and the kind of actionable should identify the expected positive information they might be able to produce. behavior changes that will occur if the (A Guide to Safety Culture Evaluation, improvement is successful to allow later EFCOG, 2015). However, to truly effectiveness reviews. understand the organizational behavior that resulted in the performance change, focus After the above is completed, SCIT is ready group discussions need to be held with the to communicate their recommendation to the affected groups. It is at this level of SLT. This communication can be by formal understanding where true improvement report or other means. This communication actions can be developed. should include the scope of the inputs Example: It has been reported to reviewed, specific trends observed over management that certain groups are time, any adverse safety culture impacts reluctant to raise issues but this is not identified, the organizations involved, and reflected in the Corrective Action actions being taken to mitigate or address Management system (Organizational the impacts. The report to the SLT should Learning: Credibility, trust and reporting include trends or potential issues that could errors and problems). To monitor this aspect be early indications of a safety culture of Safety Culture, an option is to determine challenge or strength. The panel’s analysis whether employees have an aversion to and report should address behaviors as well raising issues, are using a different avenue to as outcomes. raise issues, etc., and gather any pertinent data from those other avenues that allow for Step #5: Recommendations get approved the raising of issues. In other words, and implemented (Section 8.0). capturing a potential chilled “pocket” vs. registering the impact the other avenue that Improvement actions should be designed was used, such as a new suggestion box using organizational change management installed in worker lunch rooms. The former techniques and processes to improve the would require a robust response from the probability of success (see Section 8.0). The organization to mitigate culture risks, while SLT and SCIT monitor progress on the the latter may be addressed simply by improvement actions. providing additional options for soliciting employee suggestions.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 14

Step #6: SCIT monitors culture/behavior culture. Through the safety culture concerns to see if improvements are monitoring process (Sections 6.0 & 7.0), effective (Section 9.0). data have been gathered that will suggest the If the analyses and assumptions about strengths and weaknesses of the relationships between variables are correct, organization. After the evaluation of these improvement initiatives should yield data, virtually all managers ask the valid positive behavior/cultural changes that have question, “so now what?” Edgar Schein’s a positive impact on performance that is model for improving organizational culture reflected in the performance indicators. (Schein, 2009) describes three elements to help answer that question. The remainder of this section discusses these three elements in Step #7: SCIT communicates results of more detail:. improvement actions and their impact on culture change (Section 10.0). 1. Determine culture strengths to build on Observed changes should be communicated 2. Work on cultural areas that inhibit to the SLT through regular meetings. The success meetings should provide the forum at which 3. Improve work environment to critical, reflective conversations about safety positively improve the culture. culture take place. Prior to the meeting input There are dozens of cultural change models material is prepared in the form of a report and methods described in business and or slides. During the meeting collective organizational development literature. If the judgements are formed about the organization subscribes to one of these significance of cultural implications. roadmaps, social scientists can help with the Specific actions, owners, and dates are transformation. Behavioral scientists can assigned when practical. In the nominal help engineers and scientists understand the process, the primary function of the SCIT is human aspects of proposed (e.g., to provide the SLT with “enriched emotions, feelings, ) and help intelligence” on the health of the safety bridge the gap between professional culture.This intelligence facilitates the work disciplines. of the SLT and enables the SLT to judge the organizational health. Thus, the SCIT is not Recognize that the track record for required to gauge the health of individual successful culture change in American safety culture attributes. However, if a organizations is poor. John P. Kotter, noted organization finds it useful for the SCIT to professor of the Harvard Business School, “grade” every attribute, the organization examined why most transformation efforts may certainly do so. fail (Kotter, 2007). He concluded that organizational change efforts failed when senior leaders made fatal errors in one or more of eight critical areas. To increase the 8.0 Safety Culture likelihood of success, he encourages Improvement Initiatives organizations to consider the following steps to transform an organization (see Figure 2). This section describes how to identify and execute improvement initiatives to enhance A good starting point for developing safety the maturity of the organization’s safety culture improvement initiatives is to be

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 15

familiar with the three safety culture focus When developing a change initiative, how to areas described in Attachment 10 (i.e., evaluate the effectiveness of the initiative Leadership; Employee/Worker engagement; must be understood. Verify that the initiative and Learning Organization). The SCIT was implemented as intended and had the should identify initiatives to improve or desired effect. sustain performance in the three focus areas. EIGHT STEPS TO TRANSFORMING YOUR ORGANIZATION (Reference: John P. Kotter, Harvard Business Review, January 2007) 1. Establishing a Sense of Urgency a. Examining market and competitive realities b. Identifying and discussing crises, potential crises, or major opportunities 2. Forming a Powerful Guiding Coalition a. Assembling a group with enough power to lead the change effort b. Encouraging the group to work together as a team 3. Creating a Vision a. Creating a vision to help direct the change effort b. Developing strategies for achieving that vision 4. Communicating the Vision a. Using every vehicle possible to communicate the new vision and strategies b. Teaching new behaviors by the example of the guiding coalition 5. Empowering Others to Act on the Vision a. Getting rid of obstacles to change b. Changing systems or structures that seriously undermine the vision c. Encouraging risk taking and nontraditional ideas, activities, and actions 6. Planning for and Creating Short-term Wins a. Planning for visible performance improvements b. Creating those improvements c. Recognizing and rewarding employees involved in the improvements 7. Consolidating Improvements and Producing Still More Changes a. Using increased credibility to change systems, structures , and policies that don’t fit vision b. Hiring, promoting, and developing employees who can implement the vision c. Reinvigorating the process with new projects, themes, and change agents 8. Institutionalizing New Approaches a. Articulating the connections between the new behaviors and corporate success b. Developing the means to ensure leadership development and succession

Figure 2. Eight Steps to Transform Your Organization

8.1 Build on Strengths positive culture attributes is a missed opportunity since these attributes It is important to recognize and reinforce contributed to the success already achieved. strengths that have been identified through Additionally, the organization’s existing monitoring the organization’s safety culture. cultural strengths can be leveraged to help It is often easy for management to focus on overcome those culture attributes that need the cultural aspects needing improvement attention. The organization can capitalize on and ignore the positive aspects. However, what it does well and encourage employee failing to emphasize and strengthen the engagement by including the workforce in

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 16

problem solving and owning the solutions. sincere “thank you” goes a long way This process rewards positive behavior and toward reinforcing positive behavior. outcomes. Successes are shared with an • Close the loop. When an emphasis on employee contributions and employee(s) reports a problem, organizational learning is promoted. consider it an opportunity for Examples of ways an organization can build improvement. Make sure the upon its strengths and engage employees to employee receives feedback on how improve mission performance include: their comments were used to improve the performance of the • Share success stories. When a organization. successful outcome is discovered in the organization, help all learn by When seeking to impact cultural issues that sharing the success story with those are inhibiting success, first look for throughout the organization who solutions inside your organization. Some could benefit from the information. parts of the organization may have already solved the problem. For example, a safety • Solicit ideas and suggestions. When culture evaluation at one research an employee is struggling with a organization revealed that there was task, solicit assistance from others widespread dissatisfaction with the work who have achieved success with the planning system. The workforce reported task to maximize organization that the system was cumbersome, difficult to learning. implement, did not enhance safety, and was • Engage impacted workers in the an impediment to conducting experiments. solution to the problem. Hold focus However, there was one exception. The groups and provide workers with the workforce within one research sub-group problem that needs to be solved. indicated that they were very satisfied with Workers most affected by the how work planning was conducted. Senior problem will typically determine management took a closer look into this better solutions to the issue based on group’s work planning methods and learned their perspectives and experience. that they had modified the planning system Worker engagement and ownership to streamline the effort and promote team allows the workers to solve the participation. Senior management asked problem rather than the solution representatives from this organization to being handed down by management. lead the effort to revamp work planning for the entire research branch of the • Reinforce positive safety culture organization. Senior leadership looked behaviors. When workers are within and discovered that they already had observed exhibiting positive safety the solution in one of their groups. culture behavior, they should be recognized by the individual who 8.2 Work on Cultural Areas that observed the positive behavior. Feedback can be provided in various Inhibit Success ways but should be sincere, specific, The senior leadership team must understand immediate, and not associated with the state of the organization’s safety culture, any other message. Oftentimes, a consider the recommendations of the SCIT, and lead the organization as appropriate

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 17

actions are taken to address the cultural Less severe or pervasive issues (e.g., faint areas that are inhibiting success. The actions signals) can be addressed through other must come from the highest levels of means. For example, feedback from the management to demonstrate ownership and workforce sometimes indicates that leadership. The expectation should be that managers are not visible in the workplace the improvement actions/recommendations and are believed to be disconnected from the are opportunities to influence culture and are reality of life on the shop floor or in the field not expected to be a panacea that solves (See Attachment 10, DOE ISM Guide, cultural problems instantly. 2011). In one organization, senior leadership mandated the managers spend more time in The first step is to understand the severity the field and developed a reporting system and extent of the weaknesses. Weaknesses to track their performance. Managers that have high consequences (e.g., dutifully complied and logged entries into workforce fear of retaliation for raising the system every month. An analysis of two safety concerns) and are pervasive should be years of data showed that for more than half addressed through the formal corrective of the entries made by managers, there was actions system. These types of cultural no evidence that managers actually left their problems have legal as well as operational desk and more than 60% of the time there implications and require a rigorous effort to was no evidence that they actually engaged ensure that effective changes are (i.e., had a conversation) with any members implemented. Accountability tools should be of the workforce. Senior management had used and periodic feedback should be created a system to increase management obtained from individuals that have action time in the field but failed to provide items to help them stay on track. If they are managers with clear expectations and overwhelmed, determine what can be done training on how to effectively engage with to help them get back on track. the workforce when spending time in the When developing initiatives, solicit input field. The organization should encourage the from the affected organizations. If art of conversation and train, equip, and feasible, the affected organization should coach managers how to listen and engage have the opportunity to devise a response for the workforce in regular meaningful senior leadership approval. By allowing the conversations. affected organization to develop their response, the organization may assume more 8.3 Improve Work Environment ownership in solving their problems and An organization’s leadership must recognize improving the potential for success. the effect the work environment has on safety culture. A few examples of factors Be specific when applying potential that can affect the work environment solutions. If only one sub-organization is include: experiencing a problem, avoid forcing the • Availability of resources (e.g., solution on the entire organization. Applying people, tools, access to information, a broad-brush approach when a more etc.) focused solution is necessary is often viewed unfavorably by the workforce. One size does • Reliability and usability of not necessarily fit all. management systems

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 18

• Physical work location (e.g., affected water flow whenever water lighting, heat, ergonomics, safety, was used. Management followed up etc.) with the worker and indicated they appreciated the individual bringing it • Age and maintenance of facilities to their attention and they were going (e.g., timely repairs, preventive to determine appropriate maintenance, etc.) improvement actions. • Interpersonal behavior (e.g., how • Failure to take action on a reported employees treat each other, respect, safety concern. A worker reported honesty, etc.) repeatedly through the proper • Support, encouragement, and channels that the lighting was out in recognition from leaders (e.g., a building he/she works in regularly. reaction to reported problems, The lighting was not repaired, which response to human error, fair led to worker frustration and performance appraisals, consistent eventually resulted in a significant disciplinary actions, etc.) event since the lighting impacted the worker’s ability to do his/her job. • Management responsiveness to improving the work environment • Failure to provide adequate when workers identify concerns resources. Management’s (e.g., ignore, timeliness of response, expectation is that workers use knee communication/feedback, etc.) protection when involved in tasks requiring prolonged kneeling. Below are a few positive and negative Workers obtain their knee protection examples of the potential impact the work from the tool crib. The tool crib is environment can have on an organization’s out of knee protection. Therefore, safety culture: workers either do not use knee • Prioritize and fund safety protection, or improvise and come up improvements. It was recognized with alternate knee protection that is that the organization had limited not as effective or may cause other funding for making safety safety . improvements. Management allocated funding and established a cross-organizational team of first line 9.0 Evaluating Effectiveness workers to identify and prioritize a list of safety improvements. Several of Change Initiatives safety improvements were When developing a change initiative, the implemented including painting, method for evaluating the effectiveness of building maintenance, and installing the initiative must be considered before the sidewalks. These improvements initiative is implemented. When made a significant difference in the management invests in safety culture morale of the workers. initiatives, they deserve an answer to the • Feedback for reporting a safety issue. fundamental question, “did the initiative A worker notified management that have the desired impact?” If the answer is piping produced water hammer and no, the initiative should be modified or

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 19

discontinued. If the answer is yes, the • Are other metrics trending in the initiative should be recognized as a success desired direction? story. This fundamental question must be Examples of data that can be used to answered to help demonstrate the value of evaluate the impact of safety culture safety culture to the organization. initiatives include, but are not limited to: When evaluating the effectiveness of change Systems, structures and components initiatives, the SCIT should make evidence- based decisions on what initiatives should be • Ratio of preventive and corrective continued, which should be modified, and maintenance which should be discontinued. • Percentage of safety critical When developing the evaluation plan for an equipment that fails inspection / initiative, a graded approach should be used testing based on the urgency and seriousness of the Past process safety performance problem. For example, an initiative to improve worker willingness to report safety • Availability of safety systems concerns may need to be evaluated sooner • Number of safety critical equipment rather than waiting for the next safety that fail to operate as designed culture assessment or later than an eminent safety risk, such as scaffolding that lacks fall Human factors protection. • Sick leave When evaluating effectiveness two things • Turnover should be measured: was the initiative implemented as intended and did it have the • Job satisfaction and work motivation intended results. Evaluating the scores from surveys “implemented as intended” simply means • Amount of procedure violations “did we execute the things we said we would in our initiative?” Many • Root causes of events dealing with organizations develop terrific initiatives but human behavior then fail to implement them as intended. Past organizational safety performance This requires that activities be measured and tracked so that the results can be reported. • Recurrence of incidents with similar root causes In contrast, evaluating the intended results of the initiative involves verifying that a • Backlog of corrective actions change in safety culture or behaviors has occurred. • Are the workforce perceptions of 10.0 Safety Culture safety culture improving (based on survey and focus group data)? Communication • Is there consistent safety Communication should be occurring performance across the organization? throughout the safety culture monitoring and • Are stakeholders satisfied with the improvement process. Communication from organization’s safety performance? the SCIT is not designed to replace primary

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 20

communication between management and management’s safety culture vision for the the workforce. Communication from the organization and how it relates to current SCIT should complement primary conditions and demonstrates management communication channels. This section commitment to bring together the vision and discusses communication between the SLT conditions. The plan should be flexible and and the SCIT, internal organizational up-to-date with organizational culture and communication, and external site conditions. A typical safety culture communication. communication plan is framed by asking questions such as: Safety culture communications should provide continuity from previous messages. • What is the message? For example, after the results of a safety • What is the value of the message? culture evaluation have been communicated, follow-up actions to address comments • Who are the audiences? show that management values input from the • Who will be delivering the message? workforce. • After appropriate consideration as to the When will the message be delivered? impact, safety culture communications • What is the medium for delivery? should be integrated into other, existing operational feedback mechanisms (e.g., 10.1 SCIT Communication contractor assurance system, causal analysis, There should be a formal connection lessons learned, and safety shares) to between the SCIT and SLT to facilitate provide an organizational behavior communication and decision-making. The component. SCIT should invest the time to ensure the Communicating the results of safety culture SLT sees the value of cultural insight so that improvement efforts help everyone embrace the SLT owns/values the process and results. the safety culture initiatives and desired A key value to emphasize is avoiding culture of the organization during the complacency leading to a major event. conduct of their day-to-day work. Communication helps engage the workforce Clear roles must be established for both the in the safety culture monitoring process and SCIT and SLT regarding safety culture contributes to organizational learning. communication. At least one member should participate on both the SCIT and SLT to Communication should include celebrations provide continuity and promote efficiency in of success as well as thoughtful communication. This person should provide communication of challenges. Over time, the SLT with insight to into how SCIT frequent, timely, honest, and transparent results were formulated and answer SCIT communication helps build trust and mutual questions related to the SLT. The person respect between management, the should understand the connection between workforce, and stakeholders. Safety culture safety culture and successful mission communication should rely on both formal accomplishment and be knowledgeable and informal feedback systems. about how the SCIT achieves those results A safety culture communication plan helps (i.e., understand the results and provide structure so that messages achieves communicate the recommendations of the the intended result. It is a chance to connect SCIT to improve the chance of mission

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 21

success). In addition, SLT participation in draw conclusions from incomplete or SCIT activities sends a message that the incorrect data will be challenged. SLT is committed to, and supportive of, • Future Actions – When those activities. management understands the current When communicating with the SLT and status, where does that lead? Be other senior managers, use the prepared to discuss not only the communication framework identified current information, but also the earlier: recommended next steps. How will problems be addressed, and how will • Identify the audience – How much opportunities be exploited? When does management know about the consuming management’s time and topic? Some managers will be very attention, recognize the value of their familiar with the material being time and tell them what they can do covered, while others may have just to help. basic background knowledge and There are a variety of ways to present the need more context. What areas are SCIT information to senior management. particularly relevant to them? Some examples include: • Message – What specific message is • to be communicated to management? Dashboards – A dashboard is a data visualization tool that displays What are the most important points the current status of selected metrics and what is the message? It is and key performance indicators important to be direct and efficient in (KPIs). Dashboards consolidate and presenting the information; senior arrange numbers, metrics, and other management has limited time and a information on a single screen. wide range of issues demanding their Dashboards may be general or attention. Stay focused on the core tailored for specific roles, and can point(s). display metrics from a single point • Context – Why does the information of view or from a wider perspective. matter to management? What is the Optimally, dashboards can pull real- significance and context? Show how time data to maintain a snapshot of the information relates to the most current information, and organizational goals and historical can be customizable to the exact performance. What are the needs of the customer. influencing factors? What are the • potential risks, and what Scorecards – A performance scorecard is a graphical opportunities are being presented? representation of an organization’s • Quality of Information – What is progress toward some specified goal the certainty of the data and or goals. Both dashboards and conclusions? When presenting to scorecards measure performance senior management, data must be against certain metrics and KPIs, accurate and precise. How the but while a dashboard indicates the information gathered and what was status at a specific point of time, a the potential flaws or gaps? Data can scorecard shows progress over time. be ambiguous, and any attempt to

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 22

Scorecards can be tailored to the stimulate conversation between managers needs of the customer, and can be and members of the workforce. Often, utilized to show trends and to managers need training and coaching so they identify short and long term effects can have meaningful engagements with the of organizational changes. workforce and to create a climate where person-to-person communication openly • Reports – A report can provide occurs. details on the data sources evaluated during a specific period and It is important that senior management resulting analysis that substantiate communicate to the entire organization any subsequent actions and initiatives. planned adjustments necessary to address The reports can also outline planned safety culture areas needing improvement to actions for the next period. enhance mission effectiveness. This communication must be structured so that all

personnel understand the message. SCIT members serve a dual communication Management must clearly state an expected role. The initial communication role is to action, deliverable, and the intended impact communicate to senior management/ of the action. leadership the results of the safety culture A key benefit of monitoring safety culture is monitoring process. The second to stimulate two-way communication communication role is to provide clarity to between management and the workforce. the members’ home organizations First communicate what the monitoring is concerning the results. telling the organization and second, identify 10.2 Organizational the specific behaviors that lead to successes and challenges. Communication Communication vehicles such as e-mails, Organizations may desire more or less posters and newsletters should be used to formality and control of communication ensure rapid and wide distribution of safety with the workforce. The tools used to culture results and meaning to the communicate can vary based on the maturity organization, as well as other of the organizational culture. Some communication methods such as hard messages may be tailored to specific sub- copies, use of multiple languages, videos, organizations and may not be applicable to briefings, etc. If the facility has a the entire organization. Communication communication organization, coordinating should provide a balance between messaging can be done to promote efficient celebrating successes and opportunities for delivery and message cohesion. A good improvement. working relationship with the communication organization can also keep It is important to recognize direct the SCIT in the loop if other organizations interpersonal communication between start to issue competing messages. management and the workforce. Routine informal conversations between managers 10.3 External Communication and the workforce are an important method Stakeholders are parties that have a vested of communication. Management interest in the successful accomplishment of observations in the field are intended to the mission. Therefore, they have an interest

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 23

in safety culture because it can impact • Performance Evaluation mission success. The list of stakeholders Management Plan varies from organization to organization but • EFCOG meetings typically includes: • Community safety fairs/forums • Customers • The community surrounding the site • Corporate entities 11.0 Effectiveness of the • Subcontractors SCIT • Elected officials In addition to striving to improve the organization’s safety culture, efforts should • Other contractors and entities (share be undertaken to continually improve the best practices) SCIT to make it more effective and to • Regulators continue to add value to the organization. The first step in the improvement process is • Organized labor organizations to evaluate the SCIT. Examples of process Communication with stakeholders evaluation measures include, but are not constitutes external communication and limited to: therefore requires a different protocol and • If the SCIT planned to meet bi- degree of rigor. Coordination with your weekly or monthly, did that actually organization’s Public Affairs Office (or occur? equivalent) is essential. Requirements may vary depending on where you are within the • If the SCIT planned to produce DOE enterprise (e.g., National Nuclear monthly safety culture articles for Security Administration vs. Office of the company newsletter, did that Science). External communication tends to happen? be more formal, requiring review and • Did the SCIT develop and faithfully release by the technical information office execute their communication plan? and authorized derivative classifiers/review officials. • Did the SCIT get the data needed to measure safety culture progress? Examples of how safety culture monitoring • Did members of the SCIT show up at results and associated improvements are the meetings? communicated to stakeholders include: • Did the workforce participate in • Safety culture sustainability plans surveys and focus groups at an submitted to DOE acceptable rate? • ISMS declaration (see DOE O 450.2 • Was two-way communication used paragraph 4.c.) to discover issues requiring • Informal safety culture forums at improvement? large sites where contractors can • Did management/leadership commit share information with each other the resources to support the SCIT? and with DOE representatives

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 24

• Did the affected organization 13.0 References implement with the intent to get the desired results of the improvement Campus Labs. Baseline Help Center. actions? Quantitative vs. Qualitative Measures. http://baselinesupport.campuslabs.com/hc/en • Did other supporting organizations -us/articles/204304995-Quantitative-vs- deliver the resources requested by Qualitative-Measures). Accessed January the SCIT? 10, 2017. Based on the insight gained by answering Cole, Kerstan, Susan Stevens-Adams, and the above questions, the SCIT should meet Caren Wenner, A Literature Review of with the SLT to make the necessary Safety Culture. Sandia National modifications and to provide positive Laboratories. Sandia Report SAND2013- feedback on those aspects of the SCIT 2754, March 2013. which are working well. Energy Facility Contractors Group, Integrated Safety Management Working 12.0 Records Group. A Guide to Safety Culture The SCIT may generate some records in the Evaluation. 2015. course of their activities. These may include: Energy Facility Contractors Group, Safety Culture Working Group. Best Practice #181. • Meeting agendas 2015. • Meeting minutes Institute of Nuclear Power Operations. • Action item lists Traits of a Healthy Nuclear Safety Culture, • Evidence of activities (e.g., copies of Revision 1. INPO 12-012. April 2013. communication pieces) International Atomic Energy Agency. Safety • Progress Reports Culture: A Report by the International • Annual evaluation reports Nuclear Safety Advisory Group. Safety • Safety Culture Program Plans Series No. 75-INSAG-4. 1991. • Etc. Kotter, John P. Leading Change: Why These records should be maintained Transformation Efforts Fail. Harvard following quality and records management Business Review. January, 2007. system requirements for records retention. National Aeronautics and Space Administration. Columbia Accident Investigation Board, Report Volume 1. Care should be given to determine August 2003. the sensitivity of records, especially in the case of qualitative data that includes National Transportation Safety Board. (or implies) identifiers for sources of Collision of Two Washington Metropolitan feedback in focus groups, interviews, or Area Transit Authority Metrorail Trains surveys. Additionally, the self- Near Fort Totten Station. NTSB Report No. identification of issues or challenges may RAR-10-12, 7/27/10. be deemed sensitive by the SLT, and Nuclear Energy Institute. Fostering a appropriate precautions should be taken Healthy Nuclear Safety Culture. NEI 09-07, to label and protect records. Revision 1. May 2014.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 25

Schien, Edgar. (2009) The Corporate Culture Survival Guide. Jossey-Bass, John Wiley & Sons. U.S. Chemical Safety and Investigation Board, Investigation Report – Refinery Explosion and Fire March 23, 2005. Report No. 2005-04-I-TX, March 2007. U.S. Department of Energy. Federal Line Management Oversight of DOE Nuclear Facilities. DOE G 226.1-2A. April 14, 2014. U.S. Department of Energy. Integrated Safety Management System Guide. DOE G 450.4-1C. Attachment 10. Safety Culture Focus Areas and Associated Attributes. September 29, 2011. U.S. Department of Energy. Integrated Safety Management. DOE O 450.2 Chg 1 (MinChg). April 25, 2011.

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 26

Appendix A Potential Sources of Data for Safety Culture Monitoring

Regulatory Communication and other External Sources of Feedback

1. Defense Nuclear Facilities Safety Board inspections and evaluations 2. DOE Facility Representative Reports 3. DOE Office of Nuclear Safety and Environmental Assessments, Office of Nuclear Assurance inspections and observations 4. External assessments 5. Occurrence Reporting Processing System (ORPS) 6. Industry evaluations and benchmarking 7. External safety culture evaluations

Internal Sources of Feedback 1. Safety culture self-assessments, surveys, and evaluations 2. Other self-assessments 3. Management/leadership observations 4. Quality surveillances 5. Operating experience 6. Performance trends and metrics 7. Contractor assurance information

Input from the Workforce 1. Employee Concerns Program 2. Incentives, awards, and recognition programs 3. Safety leadership input 4. Employee engagement survey results 5 Feedback at plan of day, pre and post-job briefs, workgroup meetings

Safety Culture Review Meetings 1. Safety Culture Monitoring Panel/Safety Culture Improvement Team 2. Site Leadership Team 3. Executive Management/Leadership Team

Rev. 0 April 2017 Guide to Monitoring & Improving Safety Culture Page 27

Appendix B Potential Indicators for Monitoring Cultural Health

• Time to resolve issues raised (regardless of the type of issue) • Ratio of severe to minor issues that are being raised • Tracking suggestions for improvements • Schedule slip for performing internal self-assessments • “Noteworthy” practices identified by assessments, compared to deficiencies or weaknesses • Survey participation rate • Communications (the messages, the frequency, the effectiveness, etc.) • Count/celebrate successes • Staff perception (survey)

o Reporting of deviations, worries and own mistakes is encouraged by the management o Feedback is provided to personnel on near-misses and incidents o Work climate supports team work and knowledge sharing o Safety is a clearly recognized value at the organization • The discrepancy between formal rules and actual work • Adequacy of system for familiarization and on-boarding of new personnel • Analysis of common safety related findings (trends, root causes, changes, variety of corrective actions, generalizability to other components / equipment) from events and near misses

Rev. 0 April 2017