John Advised of Work That He Had Completed and Had Come to the Initial Conclusion That

RAeS HFG EMSG Compliance in Maintenance Project

Royal Aeronautical Society (RAeS), Human Factors Committee (HFG) Engineering and Maintenance Standing Group (EMSG)

Safety Culture in Aviation Maintenance. Non-Compliance with Maintenance Procedures 1. Introduction and Background Aircraft maintenance within public transport operations is subject to many challenges, typically these include: time constraints, cost constraints, competition, staffing, training, management of technical data and required resources. One of the key issues affecting safety in aviation maintenance was considered by the EMSG to be the “Safety Culture” that exists in a company. Corporate culture is a potentially huge topic, involving many contributing elements. A culture of compliance is one that supports an organisation’s safety achievement to desired levels; conversely one of non-compliance is one that threatens it, in particular the potential for human error to occur. One of the key elements identified by the EMSG through maintenance error investigation as a causal factor is the apparent level of non-compliance with maintenance procedures and their usability. Maintenance procedures in themselves are also a vast topic, which would require a very complex study, and larger than the EMSG had the resources to support. Therefore, to try to get a better understanding of the extent that non-compliance affects safety the committee decided to study compliance with one particular aspect of inspection, using the first level inspection as a baseline example. The ‘Daily’ check was chosen for study being a check typically carried out every 24 to 48 hours. It is also the first level of inspection that requires a release to service. Committee members considered that there was no obvious recorded safety issue highlighted from the accomplishment of Daily checks, therefore the level of compliance might shed some light on the decision making process of maintenance personnel when carrying out inspections. The Daily check had additional value, as it’s a manageable chunk of maintenance activity suitable for this kind of study. This microcosm of the maintenance workplace might address some of the issues surrounding compliance and whether the decision making process was linked to whether an item of work is feasible, practical or indeed perceived to be warranted. The selection of tasks for compliance or otherwise should not be a decision for the maintenance engineer but it is clear that they frequently have to make such decisions. If reasons for non-compliance with procedures can be reduced, better managed, or even eliminated, it would strengthen the case for compliance with good, workable procedures. This in turn, should promote better safety practices in the organisation and improve the company’s safety culture. We strongly believe it is important to stop bad norms that we believe already exist and/or are developing. 2. Aims The studies primary aims were considered to be:  To determine the nature and extent of compliance with procedures within Daily inspections.

025f379950703c13e5d3f13edec059f8.doc Revision E 1 RAeS HFG EMSG Compliance in Maintenance Project

 To understand the maintenance engineer decision-making process where compliance or non-compliance was achieved and to understand if this was random or systematic within shifts, the Company or more widely in the aviation industry.  To improve the management of compliant practice in aviation maintenance organisations. In the generation of this report, there was no intention to show licensed engineers as being irresponsible or deliberate rule breakers. Nor was the intention to suggest that the answer to the problem is simply to enforce dogged compliance with every procedure, as they currently exist. It was however, the aim to raise the profile of a perceived problem within the maintenance environment. To take the opportunity to express the concerns that many have within the industryproblems maintenance engineers face. They are faced with balancing the multiple praircraft ntenance today,wilst trying to maintain comptems in aircraft, with limited resources 3. Methodology To determine the degree of compliance with Daily inspection, procedures in several UK maintenance organisations were reviewed. EMSG committee members, who were employees of the respective airlines, achieved this within their own organisations. EMSG members looked at examples of compliance with Daily checks within their own organisations using a process of unobtrusive observation similar to that used in Line Oriented Safety Assessment (LOSA) assessments. This was carried out as an independent overview of the task, without interference, although we also sought feedback from those engineers doing the task. The results were pooled to de-identify the organisations involved. It is also a fact that inevitably some of the findings are more supported by opinion than fact, but that does not detract from the information contained herein. 4. General Findings The review of compliance with the “Daily” inspection followed a standard methodology of observation and discussion of those who routinely do the task

 All of the organisations contributing were operating large commercial airliners in scheduled, or chartered public transport operations.  Each airline had documented checklists available for their staff to use when carrying out the Daily inspection.  Each checklist was based on the aircraft manufacturers maintenance recommendations, but had been enhanced by the airline based on their operational knowledge to add to the requirement of the “Daily”.  Although checklists were available in most cases the experienced engineers carrying out the dailies task did not see the need to use them.  The engineers observed in this review were considered to be average, or better experienced people who were well trained, capable and importantly committed to the task of safety within aircraft engineering and their Company’s thrust for safe operations.  None of the observed inspections and the associated actions resulted in any safety outcomes. 5. Indicative Findings

Although not necessarily the same in all companies, or even between individual Engineers,

the following listed findings are indicative of the maintenance environment we currently operate within, and to better understand the working environment it was necessary to record some generalisations and assumptions. Also a number of common themes were identified, which suggest “norms” in the aviation maintenance industry within the UK and potentially exactly the same issues exist many other countries.

 Generally is was advised that the engineers would look at the checklist from time to time as a refresher, but it was suggested that the routine use of the checklist was only necessary whilst experience was built on a given aircraft type.  There was virtually no systematic quality assurance assessment of the Daily Inspection and how it was achieved. Albeit, all, or part, of the “Daily inspection” could be assessed during the routine quality processes in place.  Compliance audits called for in EASA Part 145-A-6 (c), were often not effectively carried out, and in any case, compliance with the requirements of a “Daily” was not seen as key target for such checks.  The engineers routinely made judgements of what not to cover in the “Daily” based on their individual experiences, but this did not follow a systematic decision making process.  Whenever the engineers undertook non-standard actions, they did so with the best intentions, they believed they were meeting their Company’s needs. The engineers, being practical, when dealing with the day-to-day adversities tend to think more about the task and its completion and not the level of compliance. However, they do recognise risks and do not deliberately take what they perceive to be risks to get the job done.  The amount of detail covered in the “Daily” could vary, this was largely said to be dependant on secondary tasking for the engineer on the shift. This additional tasking included the amount of defect rectification that the engineer might be required to support on the aircraft, or the numbers of certifying staff available on the shift to complete the work. There were notable variations from the intended requirements; the findings for these and our views on the relative risks are reported in the detailed analysis below. 6. Analysis of Specific Elements Each element of the Daily was reviewed and of these those with observed variance or concerns have been identified, commented on, and assessed herein. Herein, each task element is commented on and assessed for impact. The observations are based on the notes taken from the task assessors, and the comments are views of EMSG members that relate to the task element.

Task Element Observed Standard Potential Risk Level

6.1 Flight deck checks Variable Low

Observations: Generally the high level of achievement is commensurate with the perceived importance. The working environment is comfortable and the tasking varied and in some part interesting. When necessary, handy kits are available e.g. filaments. In some cases, the cockpit checks were not done to an effective standard, citing two primary reasons.

- The ability to access the flight deck readily, particularly if the aircraft is on a turn-round, and - Many tasks are duplicated and done by both the flight crew and engineers. Defects were considered likely to be reported as and when they were identified, (e.g. fire system squib tests) therefore it is perceived that this is not a strong driver for the engineer to carry out these checks. Cockpit checks appear to be an area where the engineer’s workload might be reduced. Particularly if there was not enough time to do everything without incurring technical delays.

EMSG Comments: Although assessed as low risk some issues such as flight management and auto pilot functionality might not be discovered until the aircraft is airborne if the designated checks are not carried out during the Daily inspection.

6.2 Wheels, Tyre & Brake checks Good Medium

Observations: Tyre checks were generally done to a high standard. Tyre checks were viewed as fundamental, and appeared to be done as a matter of course. These checks were usually the first thing looked at on arrival and are easy to access. It was also understood that tyres are susceptible to damage during landing and taxiing and therefore need to be routinely checked, this was reasonably systematic across all the companies maintenance departments. Issues like checking the tyre pressure and especially where fairly low readings in the acceptable band had been noted on previous sectors meant that tyres are almost always checked. However, it was noted that there was an element of sampling and as a negative; in poor weather this tasking was not as well achieved. Some checks such as the wheel brake check were noted as being done as a visual check with brake off, which was not in accordance with the requirements, but the engineers claimed that they could tell the wear status just as well in this way, and although it is non-compliant, it is seen to work.

EMSG Comments: Medium risk, shortfalls in checking tyres and tyre pressures could result in high profile delays if the crew refused to accept it, or the aircraft suffered a puncture during taxiing, or take-off runs. Similarly brake condition checks could lead to serious implications of overruns if not done correctly, but this was not seen as having the same level of importance in risk terms, although clearly that is not necessarily the case.

6.3 Engine checks Good High

Observations: All engineers considered engine checks as critical hence a high level of compliance. Primary areas considered were FOD damage through the engine, oil levels and fuel or oil leaks. Recording oil uplifts was not systematic.

EMSG Comments: Potentially this can be high risk, especially if checks are not done, but this is well understood, and no evidence of poor compliance was noted.

6.4 Visual external checks Variable Medium

Observations: Most exterior checks are done in a general sense as part of the walk-around, but although the engineers are trained to be observant, this task lacks a clear focus. There are some known no-go items and these were seen to be systematically checked (e.g. jiffy drain leaks, dry bay leaks etc.) the biggest problem, in particular on very large aircraft, is accessibility, exacerbated by the fact that not all areas can be clearly seen from the ground. Tasks, such as checking cowlings etc. can be highly variable, some doing a general check, whereas others carry out more specific checks of particular areas e.g. engine clamshell doors. The working environment is not good in winter months, especially in bad weather, the tasking in these areas can be dirty (e.g. in wheel-wells etc.) and visibility can also be poor. Checks of the external lights had a low level of compliance because it was reasoned that the filaments can fail at any time. It was also proposed that some engineers might do the exterior light checks by observation as the aircraft taxied-in. One problem that is particularly relevant to exterior area inspections is the order in which the checklists call off the items to be inspected, they tend to order these by ATA chapter rather that to reflect the flow of work as the engineer moves around the aircraft.

EMSG Comments: The risk is medium, as the walk round for the exterior areas is a general visual for structural damage and integrity. The effective completion of external checks is an area where non-compliant methodologies can be commonplace and at the same time remain undetected or indeed unlikely to raise concerns with the supervisors or managers as it would be seen as implementing a logical workflow, this might not be their view if an incident occurs. The practice of non-systematic checks is a potential problem that warrants effective company and regulatory control.

6.5 Cargo hold checks Variable Medium

Observations: In many cases little is achieved in the way of inspection in the cargo bay areas, this is often not seen by the engineers as an area of significant risk and the area is very busy in the periods prior to or after any flight with luggage and freight handling underway, which makes access a problem. Also it may be assumed that the loadmaster and his team will report any problems they notice, but this is not a viable practice. Although the integrity of the cargo doors and surrounding areas is demonstrably important and has been causal in several high profile accidents and incidents, in some cases there was little concern shown for this area during inspections being undertaken.

EMSG Comments: The risk is medium to high, again an area where structural damage to the airframe, cargo doors, and fire integrity of the baggage hold side walls and door frames, seals etc are critical to the aircraft, but are not seen as an area of high concern by the maintenance staff.

6.6 Fuel/water checks Poor Low

Observations: The quality of modern jet fuel and the standards in place in general within the oil companies providing the fuel results in only a minimal number problems occurring. Whilst this is a positive situation, its potential benefit is eroded by a lack of to do the task correctly, which is rarely availabse checks a norm amongst engineers that water/contamination checks, as part of the Daily inspection, are largely unnecessary. It is not considered viable in the real world to consider delaying an aircraft to allow time for settling after refuelling before doing the water drain check. Feedback received was based around the high reliability of the aircraft and the fuel provided. Whereas, the problem relates to the quality of fuel uplifted, and the ability of the fuel in the aircraft to condense water held in the fuel in suspension and to condense moisture out of the air. It was further reported that engineers had found that one particular manufacturer’s aircraft was more prone to water build up in the fuel cells than in other types, and in these cases the fuel drain sample checks were more likely to be done. Secondary in the concern, but none-the-less significant if allowed to develop, is fungicidal contamination of the fuel system typically Cladosporium Resinae. Fungicidal contamination can be particularly prevalent where water settles in fuel tanks and allowed to stay there.

EMSG Comments: The risk is low - purely due to the high levels of reliability of the fuel at the point of delivery. The widely held belief that water drain checks are unnecessary, but this is fundamentally erroneous however, the low incidence of problems perpetuates it.

6.7 Cabin checks Poor Low/Medium

Observations: Minimal checking is done in the cabin, time and accessibility is the primary reason given. The checking of items such as: seat serviceability and availability of a serviceable life jacket per seat were seen as not being feasible, or even reasonable to expect this to be done on large aircraft, and at best some sampling is done. As part of checking the aircraft cabin some elements are virtually never done, typically the engineers do not often check cabin lights, albeit that they do rectify any reported defects in the cabin as required. The cabin is a busy area during the period pre and post flight and others (cleaning, galley servicing, cabin preparation and possibly the cabin crew may already be on the aircraft) all need access in the cabin at the same time It was also felt that if there were defects in the cabin then these would have been reported through the cabin logs from the previous flight, thus in reality leaving the cabin serviceability to the customer to report the faults. The cleaners check the toilets and seats and similarly the cabin crews also check the seats, cabin area and galley this includes checking life jackets in some part as is called for in JAR-Ops 1. Finally the other reason for not carrying out systematic cabin checks is there physically is not enough time available to do all the checks, and the cabin area is low risk and an area seen to be capable of withstanding minimised checks.

EMSG Comments: The risks to the aircraft are low, but the risk to individual passengers can be significant. There is a high level of assumption that someone else will do the checking for passenger safety equipment, however, in many cases it is the cabin crew or passenger who finds many of the problems with his seat and equipment.

6.8 Stand-by power checks & Poor Medium/High Specific aircraft system checks

Observations: Some complex tasks undertaken during the completion of the Daily inspection (e.g. standby power checks/warning lights checks) can require several attempts to carry the task out robustly. This may be driven by aircraft type specific requirements, or the ergonomics may not be good (i.e. in one aircraft type it requires the engineer to lean over to hold a test switch down, which results in some of the lights to be checked, being outside of the field of vision). Other cases were cited in which the checklist procedure was inappropriate, or wrong, which renders the task unachievable, and therefore the task went undone. Other engineers had modified this task to achieve the perceived end result, thereby completing the task in a non-compliant manner without the controls of a Company

endorsed procedure, albeit that the approved procedure is flawed, Other feedback highlighted that it was possible, whilst checking the standby power on one specific aircraft type it can cause an unplanned system shutdown. Potentially this can lead to engineers being nervous of carrying out the task and in some cases not doing it for that reason.

EMSG Comments: Tasking that is impossible to achieve, or is problem bearing for the engineer are susceptible to non-compliant actions, or not being done at all. It is therefore essential that the line management overseeing such tasks learn of which tasks fall into this category and feed this back to the OEM for corrective action. 7. Other Considerations

Other considerations that were offered in comment or are supporting issues worthy of inclusion are also included in the analysis but are recorded under a separate heading herein.

7.1 Corporate Culture Poor High

Observations: The corporate culture of the companies involved was representative of those typical in European aviation operations, seeking good safe practices and supporting the notion of safety and continuous improvement. However, generally there is clear evidence that there is a difference between what is stated as being done and what actually happens. Although this non-compliant practice may be unwanted, it is in reality condoned and even encouraged. The primary reason for this relates to the resources available to support the tasking if it were to be done correctly in line with the documented instructions. The maintenance organisations’ culture should be the driver for stopping non- compliant practice, but conversely it often perpetuates the self-applied changes taken by engineers to get the job done. There appears to be a flight-line culture, where experience based discretionary violations are considered to be acceptable with the justification of getting the job done. Within engineering there is intent for general compliance with instructions and maintenance engineering believe that they are compliant. However, there is a difference between intent and achievement, but this is not a malicious act. This can vary company to company and also between individual engineers, depending on their experience, and whether they believe the task adds value. Experienced engineers were found to add additional items to the checks that they have learnt might cause problems and potential delays. However, they rarely communicate these perceived improvements to allow the potential changes to be validated and checklists to be amended for the benefit of all. There can be a tendency to not check items because the problems are perceived to only occur rarely. Fuel system water drain checks would be an example, where in the dailies observed these were omitted.

There is a belief that external/environmental factors should not however stop you doing the task properly. It is also a commonly held view that many tasks can be done from memory without reference to the manuals, but this offers no guarantee that the task will be achieved correctly.

EMSG Comments: The culture of the maintenance organisation should be driven by the compan corporate culture, which makes this a leadership and direction issue for the top management as they establish the culture and standards that the company should be following. The senior management should also be actively involved in human factors implementation and training in the organisation through a focussed business case for improvement. It is also important to consider that where a culture of non-compliance is condoned, even if that is unintentional on managements’ behalf, it is likely to engender a belief that that is what is wanted from the engineers. Engineers that get into the habit of violating procedures on Daily checks might apply the same maxims elsewhere, which could result in routine violation and potential in more safety critical areas.

7.2 Management and Control Poor High

Observations: The management and control of work is influenced by line managers, corporate leadership and their attitudes towards the work in hand. A norm for Maintenance Managers is to be results driven and often by the nature of the job they are distanced from the workforce, in particular those doing the line maintenance such as the Daily inspections. The use of checklists is indicated as the way any task is completed, but at the same time their use is probably assessed as unnecessary, not enforced, or actively encouraged. Even in companies that are using the Daily inspection worksheets for the sign off, the company do not require engineers to stamp up the sheets whilst doing the task, only after its completion, so its worth as a control might be challenged. Supervision is one of the safety controls provided for the control of work, but in today’s airline business, virtually no supervision is done in the case of line maintenance. This is hardly surprising as the aircraft are likely to be parked across a large area of an airport, and the practicalities of providing active supervision limit this as a safety control. It is also true that most supervisors would not expect to need to supervise their certifying staff during the completion of Daily inspections; nevertheless as an industry we claim it as an active safety control even if it is a largely hollow claim. However, one area supervision should routinely be addressing is that there are often requirements for the engineers to sign for work they did not, or could not, see carried out, taking the work of subordinates on trust.

EMSG Comments: It is widely believed that if all Daily inspections were done with checklists, as the

tasks were designed to be done by the aircraft manufacturers, it would generate delays to scheduled aircraft operations due to the time taken to do the tasks called out. The numbers of qualified engineers available to do the task is the limiting factor. The disappointing issue is that as an industry, the operators, the regulators and the engineers themselves, continue to convince themselves that work is systematically achieved and managers routinely say “all maintenance is done in accordance with the maintenance manuals, implying that the checklists are used. From a management and control point of view, we will not be able to improve this situation until we are ready to face workplace reality.

7.3 Regulations Variable Low

Observations: The regulations pertaining to maintenance and compliance are basic and clear. The engineers however do not find following the regulations easy, in as much as the requirements and the resources often don’t match. The aviation industry over the years has evolved, and at the same time there has been a series of cut backs in numbers of certifying staff. This has been a combination of shortfalls in suitable qualified staff and the financial pressures we face in these challenging times. At the same time the amount of public transport passenger flights has increased, this all puts pressure on compliance with regulations, especially if parts are not seen as been necessary. As an example it is accepted that the regulations exist and that they expect the use of checklists, but that requirement is not systematically followed. Additionally the new regulations will limit certification tasks such as the Daily to EASA Part 66 licence holders for the type to certify the aircraft, this is bringing serious limitations to the industry. Some engineers considered that maintenance procedures and inspections are often raised for the one reason, to satisfy the regulator. This might be the case and at the time that might be necessary, but the regulator and the operator should work together to try to ensure that procedures are only raised when no other method of control is possible. When retroactive actions are taken to solve the problem, all the tasks and inspections added to the checklists should be periodically reviewed and if possible removed.

EMSG Comments: The regulations must set the standards that we need for safety and efficiency, but the same time they must also reflect the reality of what is achievable. The concern is that regulations as written call for a standard of procedural compliance that is not reflected in the workplace and that within the requirements as stated, there may be room to better reflect the workplace and at the same time introduce improvements. As an example if simple tasks are actually completed as memory items, one approach might be to require memory items to be accepted if supported by training and routine testing, much as is already accepted practice in flight operations.

7.4 Resources Variable High

Observations: The time available for a Daily inspection is generally the minimum needed to complete, assuming no diversions or defects need to be addressed by the tasked engineer. The time to complete the task properly is a resource issue that has already been highlighted and can be a causal factor in driving non-compliant maintenance activity. It is expected that the certifying engineer will call for assistance if he could not complete the task in the time available. However, this was not always case, and some engineers think this unrealistic, as often there is not anybody spare to assist. Delaying a flight for late completion of the Daily inspection would not be welcomed unless completely unavoidable. It was accepted that omitting the less important tasks in the checklist was an option and one regularly applied when time was tight. Generally, the engineers used the time available, and without needing to think about it, adjusted the time taken to complete the Daily Manpower availability is another resource that influenced the level of compliant practice. In some cases insufficient numbers are available because of off-base maintenance requirements, sickness, training and leave. The location of the aircraft on the airfield is also a problem that adds pressure on the resources. Remotely located aircraft present more problems particularly if transport was not dedicated and available to the line engineer, this could also limit access to ground support equipment that might influence compliance.

EMSG Comments: The issue of resources is perhaps one of the more significant concerns we face today; there has been and in many areas still is a shortfall of experienced engineers worldwide an this includes the UK. The age profile demographics of engineers shows that the retirement rate is continuing to exceed the development rate of trainees. The traditional sources supplying trained and capable technicians who can rapidly bconverted into licensed engineers hasbut dried up. There remains a significant ge of alicensed engineers and although there are signs of improving demographics for A and C engineers. However, it appears that the cyclic the industry has been experiencing has delayed theconsequences of shortfalls in licensed engineer numbers and experience levels.

7.5 Training Variable Medium

Observations: Training is important to establish the standards that are required by the operator to meet the OEMs intent, but it is not the norm to give specific training or recurrent training for compliant completion of the Daily inspection.

EMSG Comments: Some of the engineers suggested that included in the recurrent type training given, it would be useful to include procedural compliance training. It was also suggested that more emphasis is needed on recurrent training based on lessons learned from incidents. One way that we might usefully address the procedural non-compliance, would be to give recurrent training on the doing the Daily inspections to establish the level of compliance sought. Additionally, do the supervising engineers have the skills to assess compliance and the reasons why it is not occurring. Those airlines that give supervisory training in their overall training package, should ensure that the supervisors are adequately equipped to guide and lead in procedural compliance, and those that do not train supervisors in the skill set they need, they should consider introducing such a programme. Should we be giving training to staff against fixed syllabuses or should we have a more flexible approach that dealt with the needs of the individual for the specific job, that focuses on the potential problem areas of that jobs tasking. Training is already a high cost to companies, it is therefore important to ensure that the training given provides the maximum benefit to serve the staff’s needs and company requirements.

7.6 Competence Variable Low

Observations: All companies ensured that the staff doing the Daily inspections were adequately qualified and trained on the aircraft types they worked on. However, with regard to tasks such as the Daily inspection, few do recurrent training and testing to assess the quality of the work being done. Inevitably there are skill variations between engineers and variation in the standards achieved in the completion of the task by individuals. It is also natural that different engineers will play to the strengths of their particular skill set.

EMSG Comments: Competence assurance is a corporate responsibility, as well as a regulatory requirement, and as such has the potential for variable standards or differing approaches should be addressed both, by the management and, all levels of supervision involved in line maintenance.

7.7 Error Tolerance Variable High

Observations: Each work process should be designed to be error tolerant and by design should be a workable approach that can be relied on for systematic use by the staff. As already described in this document, in many airlines the Daily inspection is not an error

tolerant system. Checklists can be difficult to use, as they usually do not promote a job flow, they can be proliferated with tasks that are perhaps not really practical to do, or may no longer need doing, and generally are not user friendly. If none of this were true it is likely that the Daily inspection checklists would get better usage in the workplace. Some of the tasks called out in the Daily that are assessed as being of low importance, although not omitted, but tend to be sampled rather than 100% checked. In other cases there was no evidence that these tasks are ever done. Additionally most engineers recognised that there was a need for short cuts in the workplace to get the work done, and do not consider that these are not always all bad however, they are not necessarily able to assess the impact of non compliant actions. One engineer suggested that as part of this study, that we should also consider why we are performing a Daily inspection and is it really adding value as we do it today. Perhaps this question should be taken outside of the current legislative requirements in order to examine the real value of the Daily as a stand-alone element.

EMSG Comments: It is important to identifying working practices and systems that lack error tolerance. We already know that if error can occur it will, and therefore need to identify where error might occur and focus on setting controls in place to minimise the chances of the occurrence and to contain the potential outcomes. Systems of working established in a company should as far as is possible, be error tolerant. One clear problem with working from memory, with little or no training, and no testing, is the fallibility of memory and therefore a high likelihood of errors occurring. If we within aircraft engineering accept that the Daily inspection is usually done as a memory task, and take little or no action to assess and control this process we must also accept that we will suffer undesirable outcomes, which although unlikely, they potentially could lead to accidents or serious incidents.

7.8 Checklist usability & content. Variable Medium

Observations: The checklist layout for the Daily inspection in most cases is not laid out to make ergonomic sense, or in fact to be user friendly. Most engineers felt the checklists need improvement in structure and usability. The poor structure of the tasking may be one reason for the lack of support for their use by the engineers. Part of this problem as already stated is the lack of a standardised workflow in the way the engineer moves around the aircraft. Another reason given for not using the checklist was their practicality out on the ramp, at night in the dark, whilst the engineer’s hands are at least in part occupied with torch and tools used in inspecting the aircraft. Some manufactures checklists were said to contain “seemingly endless” lists of cross-references to the maintenance manual, which although useful for research when needed, adds no value to the user

carrying out the task on the line. In some checklists the wording is unnecessary complex or vague and may not match up with the actual intent of the task. In some cases the engineer needs some knowledge and interpretation to understand what he is supposed to be checking for and what he should expect to see. Engineers also reported that Daily inspection checklist tasks may have originally been called-out for reasons that are now no longer relevant, which have now been superseded, or modified out. However, little effort was being made to remove redundant tasking from the checklists. Similarly, some of the requirements in the checklists are unreasonable; they can’t be done or would take too long to do (lifejacket checks would be an example) or where compliance with the requirement would conflict with local regulations (such as running engines during the early hours of the morning). Conversely it had also been observed that surprisingly some tasks have been removed from the checklists, where they were clear it still needed inspecting, they felt in such cases that they would still inspect what they believed to be the essential items, all of which adds to undermining the compliant use of the checklist.

EMSG Comments: As part of the aircraft Type Certification process an initial estimate of the requirement is carried out. In many modern aircraft this is based on MSG 3 analysis principles determining the inspection requirements. The OEM documents all of the manufactures maintenance requirements for the aircraft type, this includes the safety and maintenance requirements. Workflow is not necessarily a driver for the OEM in the development of the checklist, indeed sometimes they are developed along an ATA chaptering structure. If a company were to review the contents of Daily inspection over a number of years, it is worth including a review of the tasks that are added versus those that are rescinded. Te general perception is that the Daily grows considerably over the life of an aircraft and perhaps many of the tasks no longer add any benefit.

There is a requirement for checklists and tasking to be designed with human factors issues taken into account, but there are insufficient guidelines on the sorts of things to look out for in checklist design.

In some airlines an internal Maintenance Review Board assesses what is required in the maintenance programme and this includes the Daily inspection. Such reviews, based and understanding of the requirements and the experience gained from SBs, ADs, and such learning as component failure rates or areas of wear enables them to vary the maintenance programme, which is subsequently has approved. This process could, if not done already, be used to streamline the Daily inspection checklist to improve workflow and process; this might encourage more frequent usage. As part of any effort to improve the Daily inspection checklist, the staff should be encouraged to feedback what they perceive is wrong with, or difficult to achieve, in the checklist.

Similarly in the quest for improvement and achieving industry best practice, it will be necessary for the learning each company gets to share this with operators of the same

aircraft types, albeit it is recognised that this has commercial implications it would be more effective if the industry tried to address some of these problems together.

7.9 Workflow Variable Medium

Observations: Poor workflow has been cited several times in this report as one of the reasons the engineers prefer not to work directly from the checklist. This can result in the aircraft being circumnavigated by the engineer numerous times whilst carrying out the tasks. This is said to be largely due to the checklists not being logically laid out to support workflow by zones as opposed to ATA chapter requirements.

EMSG Comments: Clearly if we try to improve the checklists one key area that needs to be thought through is the workflow pattern to make the visit to the aircraft as efficient as is possible.

7.10 Working from Memory Variable Medium

Observations: There was an accepted tendency within aircraft maintenance staff for tasks such as the Daily inspection to be done purely from memory and to subsequently clear the task with a single signature. Although some companies do use the checklist for the subsequent sign-off it is the norm that this is done after the fact. The less experienced staff may use checklists, but this also means that they take longer to complete the tasks and in a small number cases this was seen to reflect adversely on the perception of individuals assessed performance. Most experienced engineers agreed that they amend the checklist tasking in their minds without needing to refer to the document, and based on their experience adding items that they believe should be checked, they also tend to subtract items that they think add no value.

EMSG Comments: Everybody’s memory is fallible and it is highly likely that engineers working from memory will result in variations of the tasks completion. It is also the case that if training or occasional us does not refresh the memory, over a period of time the practices employed will become norms and there will be a belief that the tasking will be being achieved in a complaint manner and not recognising the potential problems. The aircraft operators management are ultimately responsible for the standard of the work on the aircraft and if they know and accept that tasks are being done from memory then they are condoning the process. It is a corporate responsibility to ensure the working standards and where necessary provide controls and interventions to manage the task effectively and minimise the risks of errors or being made with the

potential consequences of unserviceabilities, delays, incidents or worse. 8. Conclusion On balance this paper identifies that although there is a problem with the way line maintenance tasking is being achieved, in particular the Daily inspection, the overall risk of an aircraft accident caused by this is probably low. The assessment did identify that the engineers worked hard to complete the task as well as could be done and that largely all the items to be checked that were perceived as important were well covered. Notwithstanding, the report also identifies there is a problem with the amount of non procedural working that is done, at least at the Daily Inspection level and this could possibly be widely spread but that was not assessed. What was also understood from this assessment was that very little is is being done to manage non-compliant application effectively. We also believed that a dogged approach of repeating the mantra that all work is done in accordance with procedures will not improve the situation. Also enforcement of use of the checklist as it stands is not a realistic response to the problem. It would however be useful if the aircraft maintenance organisations could get a measure of their levels of non-compliance through compliance monitoring this task and perhaps others. This should be done not to prove individual engineers wrong, but to understand the way that the task is actually achieved and to what level there really is a problem. The results if assessed as need resolution, should lead to the development of improved ways of working that are controllable and viable within the existing manning and tasking. This might include:  Reducing the number of tasks to be covered to the essential minimums  Accepting some tasks are memory items, but in that case, they must be part of initial and recurrent training and, in the future, becoming part of testing process to ensure a continuing acceptable standard is achieved.  Improved checklists that reflect the workflow needs of the staff doing the Daily.  Improved checklists that are reviewed to reduce them to containing the information required to complete the task at the time.  Consideration of reallocation of some of the tasking, e.g. such as the cabin checks being moved to the cabin crew.  Improving the usability of the checklists in the operational environment, such as lamination to weatherproof them.  Development of an aide memoir pocket sized flip type checklist to help with improving memory based tasking.  Enhancing the requirements of audit and compliance monitoring to ensure the organisation has a clear view of the level of task compliance and the competencies of the individuals involved.  The allocation of more resources (time, staff, equipment) might become an outcome, albeit not one that is sought, but if realism dictates additional resources are needed, then that is what is required.

025f379950703c13e5d3f13edec059f8.doc Revision E 16