Handouts (Ho) & Practice Exercises (Pe)

Practice Exercise Version: 3.02 Version Date: 09/30/2016

HANDOUTS (HO) & PRACTICE EXERCISES (PE)

Table of Contents:

Final Practical Exercise PE-01-PE...... 3 Internal Audit Form Finding HO-01-PE...... 11 Internal Audit Form No Finding HO-02-PE...... 13 Internal Audit Form Repeat Finding HO-03-PE...... 15

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 1 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016

This page intentionally left blank

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 2 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016 FINAL PRACTICAL EXERCISE PE-01-PE

As we have mentioned, Green Heron Airlines has a well-established CASS.

However, Green Heron Airlines Flight 1400 experienced an in-flight engine fire and landing gear failure. The crew conducted a go-around and used emergency procedures to extend the landing gear. No injuries were incurred, but the plane suffered substantial damage.

As the PMI, you have received information concerning the incident. You have:  Summary information concerning the scenario  Excerpt from the Green Heron CASS Manual  Three audit reports

Be able to answer the following questions and please reference where you found your answers.

Part I What went wrong? 1. In terms of performance and effectiveness, what was the failure within maintenance program?

 Performance

 Effectiveness

2. Why did the CASS fail?

Part II What next?

3. Where should changes be made in the CASS?

4. What do you think the next steps should be?

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 5 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016 Scenario On September 28, 2007, about 1313 central daylight time, XXXX Airlines flight 1400, experienced an in-flight engine fire during departure climb from Lambert-St. Louis International Airport (STL), St. Louis, Missouri. During the return to STL, the nose landing gear failed to extend, and the flight crew executed a go-around, during which the crew extended the nose gear using the emergency procedure. The flight crew conducted an emergency landing, and the 2 flight crewmembers, 3 flight attendants, and 138 passengers deplaned on the runway. No occupant injuries were reported, but the airplane sustained substantial damage from the fire.

The National Transportation Safety Board determines that the probable cause of this accident was XXXX Airlines’ maintenance personnel’s use of an inappropriate manual engine-start procedure, which led to the uncommanded opening of the left engine air turbine starter valve (ATSV), and a subsequent left engine fire, which was prolonged by the flight crew’s interruption of an emergency checklist to perform nonessential tasks.

The XXXX Airlines MD-80 Maintenance Procedures Manual contained one approved manual engine- start procedure, which stated that maintenance personnel must open the ATSV using an approved, specialized wrench to turn the wrenching flats on the upper end of the butterfly valve shaft and request that the flight crew activate the engine-start switch. The procedure further instructs maintenance personnel to close the ATSV using the wrenching flats and verify that the ATSV is closed.

During post accident interviews, XXXX Airlines maintenance personnel indicated that the approved procedure was very time consuming and could take about 20 to 40 minutes to perform because the required specialized wrench was not part of the standard tool kit and so had to be found; then, the cowl latches and lower door had to be opened, the engine start sequence performed, and the lower door closed. They stated that, instead of using the approved procedure, they usually chose to use a prying device to reach, depress and hold down the ATSV’s manual override button, which is accessed through a small panel located on the forward lower cowl door.

According to the Boeing AMM, when using the ATSV manual override button to perform a manual engine start, the button should be pushed using only thumb or finger pressure. Using a prying device to push the manual override button could put too much force on the internal pin, causing it to buckle. Boeing addressed this problem in December 1996 by issuing AOL 9-2549 to inform operators that using a levering tool to depress the manual override button could deform the internal pin. Further, Boeing revised the manual engine-start procedure contained in its AMM to include a caution to only use finger pressure to push the manual override button to avoid damaging the ATSV.

XXXX Airlines’ manual engine-start procedures are more conservative than Boeing’s because they do not allow the use of the manual override button procedure when performing a manual engine start. XXXX Airlines’ only approved manual engine-start procedure requires that a special wrench be used to turn the wrenching flats on the ATSV butterfly valve. However, as described previously, some XXXX Airlines maintenance personnel sometimes used the (unapproved) override button procedure when attempting manual engine starts because it was quicker, more practical, and easier.

According to XXXX Airlines, mechanics can submit a paper or electronic copy of a Form E63, “Request for Services,” to request a change to the maintenance procedures. Guidance regarding 25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 6 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016 the form is contained in the general procedures manual, which describes procedures to be followed by all company maintenance and inspection personnel. XXXX Airlines stated that, in 2008, it received 7,283 requests for maintenance changes. Although the company does not track the percentage of maintenance requests that result in maintenance manual changes, it estimated that about 75 percent of all manual changes have resulted from such requests. A review of the forms revealed no evidence of that a request to change the manual engine-start procedures had been submitted.

Manufacturer Manual Engine-Start Procedures

The Boeing MD-80 Aircraft Maintenance Manual (AMM), dated June 15, 2005, contains two approved manual engine start methods. One method, which is similar to the approved XXXX Airlines procedure, involves manually opening the ATSV using a special wrench to turn the wrenching flats on the upper end of the butterfly valve shaft. A notch across the flats points to the words “Open” or “Closed” on the switch cover to indicate the butterfly valve position. The other method involves depressing the manual override button, which accomplishes the same function as normal electrical activation of the ATSV.

On December 16, 1997, Boeing issued All Operators Letter (AOL) 9-2549, applicable to all DC-9 and MD-80 airplanes, cautioning against the use of a tool to depress the manual override button. The AOL was the result of a December 1996 event in which an operator experienced an uncommanded ATSV-Open indication during climb out. An engine teardown inspection found that the manual override button’s internal pin was bent and that the button was stuck in the depressed (override) position, which allowed the valve to open uncommanded. The AOL informed operators that using a screwdriver or similar levering tool to depress the manual override button could deform the manual override button’s internal pin.

In keeping with the AOL, Boeing added the following caution to its AMM.

Caution Use only hand pressure to depress override button. Use of screwdriver or other type of prying device to depress override button can deform slender pin mechanism inside valve. A deformed override button pin can hold solenoid switcher ball off its seat which allows valve to open uncommanded when air pressure is available to engine start valve. If undetected or uncorrected, this condition will result in significant damage to engine starter.

MD-80 Air Turbine Starter Valve Discrepancy History

A review of XXXX Airlines’ ATSV-related maintenance troubleshooting procedures found no specific written guidance relating to a failed ATSV or ATSV-air filter. A review of the accident airplane’s maintenance logbooks dated from September 1 to September 27, 2007, indicated that the ATSV- air filter had been removed and replaced on the airplane on September 17. During post accident interviews, the mechanic who wrote in the maintenance logbook that the ATSV-air filter had been replaced during the September 17, 2008, maintenance work stated that he actually removed it, blew air through it, and then reinstalled it on the airplane. He stated that he did not correct the logbook entry. Other maintenance personnel indicated that they also used this “blowing” technique to check the ATSV-air filter. The engine-start switch had been changed on September 19; and the

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 7 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016 ATSV had been replaced six times from September 16 to September 27, 2007 (the same period that the reported engine start problems occurred). According to the off-duty company pilot, on the flight before the accident flight, the ATSV partially opened on the first manual start attempt, and the engine started normally on the second manual start attempt. He also stated that, although the ATSV had been replaced, its replacement had not corrected the start problem. According to XXXX Airlines, Boeing, no previous malfunctions related to the air filters had been reported before the accident, and none of these organizations required that the filters be tracked.

Since the cause of the engine no-start condition was incorrectly diagnosed, the left engine ATSV had been replaced six times during the 12-day period before the accident to address these engine no-start events. Because of the intermittent nature of the failure, maintenance personnel found that replacing the ATSV appeared to allow the engine start system to work properly in all but one case; therefore, they cleared the logbook discrepancy and took the ATSV off of MEL status for the times that the ATSV replacement appeared to work. In the case that the ATSV replacement did not work, it remained on the MEL because the ATSV-air filter would not allow air to pass through and no filters were in stock. Neither Boeing, nor XXXX Airlines had previously encountered a failure within the ATSV-air filter and, therefore, no procedures on how to address such a failure existed.

The logbook review also revealed that the ATSV operation was deferred and put on the MEL four times. The deferred status was cancelled three times after maintenance was performed (ATSV changed), an operational check was made, and the automatic start sequence was deemed satisfactory.

During each shift at XXXX Airlines’ Technical Services, Maintenance Operations Control division, located in Palm Coast Florida, two technical services personnel are assigned to review and act on “alert” items reported by line maintenance personnel. An alert is triggered automatically within XXXX Airlines’ computer maintenance system to flag repeat discrepancies, deferred items placed on minimum equipment list (MEL) status, and other safety-related reports from line mechanics. Alerts are reviewed by technical services personnel, who offer troubleshooting guidance to maintenance personnel and track fleet wide maintenance issues. A review of the alert items for the accident airplane from September 16 to September 27, 2007, revealed three alerts related to the left engine ATSV not opening.

 The first alert occurred on September 16 and included the maintenance discrepancy and the action taken.

 The second alert occurred on September 17 and included the maintenance discrepancy, the action taken, and Technical Services’ comments in the form of an action to be taken (ATBT), recommending that maintenance personnel troubleshoot the wiring. The ATSV maintenance operation was deferred and the ATSV put on the MEL until the airplane arrived at the next maintenance base, at which time, the ATSV was replaced. However, because the air filter in the replacement ATSV would not allow air to pass through to the line, and because no air filters were in stock, the ATSV remained on the MEL. On September 18, maintenance personnel reported that the reported maintenance discrepancy “could not be duplicated” and removed the ATSV from the MEL.

 The third alert occurred on September 27 and included the maintenance discrepancy, the action taken, and an ATBT recommending that maintenance personnel troubleshoot the wiring.

A review of XXXX Airlines’ maintenance write-ups for reported engine start problems on the MD-80 fleet from September 16 thru September 28, 2007, revealed 27 additional engine start-related maintenance items. Of these 27 items, 18 were deferred and put on the MEL. All of the ATSV- related maintenance write-ups indicated that the issue was resolved. It was noted for each of the 27 cases that, once the start problem was resolved, no repeat discrepancies occurred within the next several flights. None of the corrective actions noted in the maintenance write-ups indicated replacing the ATSV-air filter.

Manufacturer Maintenance and Inspection Procedures

The XXXX Airlines maintenance program required ATSV-air filter elements to be cleaned at every C check. The ATSV-air filter cleaning procedure in effect at the time of the accident, which was in accordance with the procedures on XXXX Airlines Maintenance Work Card 7751, dated March 16, 2006, required opening the air filter, cleaning the filter element with solvent, drying it with compressed air, reassembling the air filter, and reinstalling it on the airplane. The procedure also stated that the screen fitting was to be removed and the metal fitting checked for flatness and general condition. A review of XXX Airlines’ maintenance manuals revealed that no detailed visual inspections of the filter element itself nor specific time-in-service inspections or replacements of the ATSV-air filters were required and that the filters were not tracked.

The accident ATSV-air filter manufacturer, outlines cleaning and visual inspection, and replacement procedures for the ATSV-air filter in its component maintenance manual (CMM), dated May 15, 2000. The CMM cleaning procedures state that the ATSV-air filter should be cleaned using cleaning solvent and a soft bristle brush and that compressed air should be blown through the air outlets to remove debris. The CMM visual inspection procedure states that the ATSV-air filter should be examined under bright light using 5- to 7-power magnification to detect surface flaws. The CMM further states that all of the ATSV-air filter parts should be visually checked for cleanliness, cracks, corrosion, deterioration, and obvious signs of damage and that the threaded parts should be checked for crossed or damaged threads. The CMM states that, if strand damage exceeds 50 percent of one strand, the ATSV-air filter should be replaced. However, the CMM does not specify the intervals at which these cleanings, inspections, or replacements should occur.

Figure 3. Schematic of the major components of the ATSV. (Not to scale.)

Additional documentation:

QA Audits observing incorrect manual start procedure and Alert reports for repeat write-ups

A review of QA reports revealed instances of use of the unapproved override button procedure when attempting manual engine starts. The corrective actions documented on the QA report were unclear.

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 10 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016 INTERNAL AUDIT FORM FINDING HO-01-PE

Maintenance Quality Assurance Audit

Your department Audit Date: 10/12/06 Line Maintenance Date of Last Audit E-Mail:

This Quality Assurance audit report summarizes the recent audit that was performed at your facility/department. An audit is only a SAMPLING examination to verify compliance with a specification or requirement. It is not an in-depth inspection. Management is expected to use the audit discrepancies as indicators of possible problem areas or of inadequate procedures or controls. Management should conduct its own in-depth examination to determine the extent of problems and their cause.

This report consists of total findings, including 0 repeat finding with the breakdown of results as follows:

Number Category Discrepancy Category Description CAR ID Number 0 I CFR or a Safety of Flight Issue 1 II Company Policy or Procedural Non-Compliance 1 0 III Concerns that if uncorrected could escalate Repeat Discrepancies

Due Date(s): The due date(s) for the Corrective Action Response(s) is stated on the individual CAR form. A type written response for each discrepancy(s) must include: A. The root cause of the problem (section A) (see Note below); B. The immediate/short-term corrective action taken (section B); C. The action to prevent recurrence (long-term solution), which includes comprehensive changes implemented to prevent recurrence of this discrepancy (Section C); D. Locate and correct similar discrepancies, if they exist, in areas not audited (Section D); E. Identify the individual responsible for implementation and monitoring corrective action, to include effective and implementation dates (Section E)

Note: For any "Repeat Findings" noted within this report, provide an explanation why the corrective actions provided on the previous audit were not effective. The response can not be processed without this information. State this explanation in the Root Cause section (Section A) of the CAR form.

Within this report, we have identified discrepancies which are required for written response in Section 1 of each applicable CAR form. The following information further defines the purpose for the discrepancy category.

 Repeat Finding: Are those findings that have been documented from a previous audit and are found to exist in a substantially identical uncorrected condition during the current audit.

 Discrepancy: A conclusion, supported by objective evidence that demonstrates an instance of non-compliance with a specific standard, such as the CFRs or an FAA accepted company procedure (RSM). Response time is 14 calendar days for a Category I and 30 calendar days for Category II discrepancies. The response time begins the day after the audit outbrief.

 Concern: A condition supported by objective evidence, which may become a discrepancy if left uncorrected. Concerns should receive the same priority that discrepancies receive with the ultimate objective being to correct potential problems or trends and prevent discrepancies from occurring. Concern discrepancies require a written response within 30 calendar days.

 Observation: A noteworthy item that may address items of interest. These observations do not require a written response.

Request for Reassignment of CARS If you (the auditee) feel the CAR in question should be reassigned to another department or escalated to a director level status (due to a program breakdown or lack of policy or procedure) requests for reassignment should be made to the QA department in writing within 5 business days of receipt of the CAR. Requests for reassignment shall include a clear description of why the CAR should be considered for reassignment and a suggestion by title of where the CAR should be reassigned. Prior to granting a reassignment the QA department may have a meeting/conference call between the auditee and the individual/ department selected to receive the reassigned CAR to discuss the possible reassignment between all affected parties. The QA department will then accept or reject this request based on supporting documentation and/or evidence.

Should you need any further assistance, please call or e-mail. Thank you for your assistance and cooperation during the audit.

Maintenance Quality Assurance Audit Corrective Action Response Requested By: RZ Event: 568

Section 1 – Corrective Action Response Section (Auditee Completion) CAR Record ID: 458 CAR DUE DATE: 11/12/07 Issued to: Manager Line Maintenance Assigned to: Manager Line Maintenance Error Category: Procedures Finding Category: ( II ) Repeat Finding? Policy: AMM manual engine start procedures Discrepancy Description: Maintenance personnel using unapproved procedure to perform manual engine start.

Corrective Action Response Section A: Root Cause Assessment: Lazy mechanic

Section B: Current Corrective Action: Disciplined mechanic

Section C: Action to Prevent Recurrence: Held meeting with leads to remind them to follow procedures.

Section D: Locate and correct similar discrepancies, if they exist, in areas not audited (complete applicable info): Audit performed in other area(s) within my area/dept. No similar discrepancies discovered. Audit performed in other area(s) within my area/dept. Located and corrected similar discrepancies at or in: Section E Effective Date: 10/30/06 Responsible Individual: Implementation Date: 10/30/06

Note to the Auditee: The Corrective Action Response (CAR) can not be processed without all areas in Section 1 completed AND the applicable supporting documentation submitted with this CAR form. Failure to submit applicable supporting documentation will result in a potential rejected CAR. Section 2 - Corrective Action Review (QA Department Use only) Acceptable? (Y/N) Y Accepted By: RZ Date: 11/01/06 Verification Req’d? Onsite Req’d? Rejected By: Date: Initial? 2nd Attempt?

Review Comments:

Verification Action Items:

Verification Accepted By: JS Date: 11/4/06 Rejected By: Date:

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 12 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016 INTERNAL AUDIT FORM NO FINDING HO-02-PE

Your department Audit Date: 04/15/07 Line Maintenance Date of Last Audit 01/25/07 E-Mail:

Number Category Discrepancy Category Description CAR ID Number 0 I CFR or a Safety of Flight Issue 0 II Company Policy or Procedural Non-Compliance 0 III Concerns that if uncorrected could escalate Repeat Discrepancies

Due Date(s): The due date(s) for the Corrective Action Response(s) is stated on the individual CAR form. A type written response for each discrepancy(s) must include: F. The root cause of the problem (section A) (see Note below); G. The immediate/short-term corrective action taken (section B); H. The action to prevent recurrence (long-term solution), which includes comprehensive changes implemented to prevent recurrence of this discrepancy (Section C); I. Locate and correct similar discrepancies, if they exist, in areas not audited (Section D); J. Identify the individual responsible for implementation and monitoring corrective action, to include effective and implementation dates (Section E)

Maintenance Quality Assurance Audit Corrective Action Response Requested By: RZ Event: 580

Section 1 – Corrective Action Response Section (Auditee Completion) CAR Record ID: 475 CAR DUE DATE: 11/12/07 Issued to: Manager Line Maintenance Assigned to: Manager Line Maintenance Error Category: No Findings Finding Category: (N/A) Repeat Finding? Policy Discrepancy Description: No Findings

Corrective Action Response Section A: Root Cause Assessment:

Section B: Current Corrective Action:

Section C: Action to Prevent Recurrence:

Section D: Locate and correct similar discrepancies, if they exist, in areas not audited (complete applicable info): Audit performed in other area(s) within my area/dept. No similar discrepancies discovered. Audit performed in other area(s) within my area/dept. Located and corrected similar discrepancies at or in: Section E Effective Date: Responsible Individual: Implementation Date:

Review Comments:

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 14 of 16 Practice Exercise Version: 3.02 Version Date: 09/30/2016 INTERNAL AUDIT FORM REPEAT FINDING HO-03-PE

Your department Audit Date: 01/25/07 Line Maintenance Date of Last Audit 10/12/06 E-Mail:

Number Category Discrepancy Category Description CAR ID Number 0 I CFR or a Safety of Flight Issue 1 II Company Policy or Procedural Non-Compliance 1 0 III Concerns that if uncorrected could escalate 1 Repeat Discrepancies 1

Due Date(s): The due date(s) for the Corrective Action Response(s) is stated on the individual CAR form. A type written response for each discrepancy(s) must include: K. The root cause of the problem (section A) (see Note below); L. The immediate/short-term corrective action taken (section B); M. The action to prevent recurrence (long-term solution), which includes comprehensive changes implemented to prevent recurrence of this discrepancy (Section C); N. Locate and correct similar discrepancies, if they exist, in areas not audited (Section D); O. Identify the individual responsible for implementation and monitoring corrective action, to include effective and implementation dates (Section E)

Maintenance Quality Assurance Audit Corrective Action Response Requested By: RZ Event: 575 Section 1 – Corrective Action Response Section (Auditee Completion) CAR Record ID: 465 CAR DUE DATE: 02/24/07 Issued to: Manager Line Maintenance Assigned to: Manager Line Maintenance Error Category: Procedures Finding Category: ( II ) Repeat Finding? Policy: AMM manual engine start procedures Discrepancy Description: Maintenance personnel using unapproved procedure to perform manual engine start.

Corrective Action Response Section A: Root Cause Assessment: Proper tool not available.

Section B: Current Corrective Action: Informed mechanic where to get proper tool.

Section C: Action to Prevent Recurrence: Held meeting with leads on where tool is located.

Section D: Locate and correct similar discrepancies, if they exist, in areas not audited (complete applicable info): Audit performed in other area(s) within my area/dept. No similar discrepancies discovered. Audit performed in other area(s) within my area/dept. Located and corrected similar discrepancies at or in: Section E Effective Date: 02/12/07 Responsible Individual: Implementation Date: 02/12/07

Review Comments:

25712, Air Carrier Continuing Analysis And Surveillance System (CASS) Participant HO/PE FOR TRAINING PURPOSES ONLY Page 16 of 16