An Assessment of the Effectiveness of Public Law 111-216 in Reducing Accident Risk1

Office of Accident Investigation and Prevention Federal Aviation Administration November 9, 2012

1 The original assessment was completed by the Office of Accident Investigation and Prevention, Federal Aviation Administration, November 22, 2010. Assessment of PL 111-216

This document presents the results of a safety analysis of proposed rulemaking by the Federal Aviation Administration (FAA) on Pilot Certification and Qualification Requirements for Air Carrier Operations, pursuant to Public Law 111-216, The Fiscal Year 2010 Federal Aviation Administration Extension Act. Section 209(b) of the Act mandates that FAA convene a panel of experts to establish the best methods and required training time, plus recurrent training requirements, to ensure that flight crewmembers of Part 121 air carriers and Part 135 air carriers “master aircraft systems, maneuvers, procedures, takeoffs and landings, and crew coordination.“

Section 216 of the Act directs FAA to undertake rulemaking to ensure that Part 121 air carriers “develop and implement means and methods for ensuring that flight crewmembers have proper qualifications and experience.” Section 216 also requires the rulemaking to ensure at least the following.

Comprehensive pre-employment screening of prospective flight crewmembers, “including an assessment of the skills, aptitudes, airmanship, and suitability of each applicant” to function effectively in an air carrier environment.

All flight crewmembers must obtain an airline transport pilot (ATP) certificate and have “appropriate multi-engine aircraft experience.”

Section 217 of the Act requires rulemaking to amend minimal requirements under Part 61 of the Federal Aviation Regulations for obtaining an ATP to ensure that a pilot can “function effectively in an air carrier operational environment. Amendments to Part 61 must ensure that applicants for an ATP have received flight training, academic training, or operational experience that effectively prepares them for a multi-pilot environment, adverse weather conditions, including icing conditions, high altitude operations, and adherence “to the highest professional standards.” Finally, Section 217 retains the required minimum of 1,500 hours before a pilot can qualify for an ATP, including “sufficient flight hours” in difficult operating conditions.

Based on a detailed review of 417 accidents over 10 years, the analysis found that the rulemaking would be most effective against accidents involving turbojets in on-demand Part 135 operations and against twin-engine aircraft accidents in Part 135 commuter operations. The rulemaking would have a significant effect against Part 121 accidents, but less so than against Part 135 accidents. The rulemaking would have had little to no effect on the small number of Part 125 accidents that occurred over the 10-year study period.

Anticipated Rulemaking

An ATP license is currently required to operate as a pilot-in-command (PIC) during Part 121 operations. The Act and the rulemaking would retain the minimum of 1,500 hours to qualify for an ATP. In addition, as directed by the Act, the rulemaking will require an ATP and an aircraft type rating for all pilots in Part 121 operations, and will require that at least 50 of the minimum 1,500 hours to qualify for an ATP be in multi-engine aircraft. The rulemaking also will include a restricted-privileges ATP which allows credit for appropriate academic and classroom training and credit for military flight experience. For example, the rule may require 1,000 hours total time as a pilot for a graduate of an accredited aviation degree program, or a minimum of 750

2 Assessment of PL 111-216 hours total time for a military pilot. The restricted-privileges ATP also would have to be a minimum age of 21.

The rule also will require time in a flight simulation training device to obtain an ATP as part of the effort to ensure “sufficient flight hours” in difficult operating conditions and to incorporate the additional experience requirements defined in Section 217 of the Act.

In addition, the proposed rule will require either a straightforward minimum of 2,500 hours of pilot experience to qualify as a PIC in Part 121 operations, or the rule may alter that by requiring that a prospective PIC has at least 1,000 hours of experience in air carrier operations, plus the 1,500-hour minimum to qualify for an ATP. Though the net effect of these 2 variations on PIC eligibility is not exactly synonymous, the effect on accident reduction is essentially the same.

Safety Analysis

Analytical Approach

The analysis assesses the likely capacity of the proposed rules to reduce the total number of accidents as well as fatal accidents in the future. It will address all operations in which the PIC must have an ATP or in which the rulemaking will require an ATP of the first officer (FO). Therefore the analysis will address all Part 121 and Part 125 operations; all operations involving twin-engine aircraft as Part 135 commuters; and all on-demand Part 135 or Part 91K operations involving turbojets or other airplanes configured with 10 or more seats. The analysis is based on all accidents that meet the above criteria from fiscal year (FY) 2001 through and including FY 2010 as follows:

Part 121 Operations 354 accidents and 370 aircraft Part 125 Operations 5 accidents and 5 aircraft Part 135 Commuter 14 accidents and 14 aircraft Part 135 On-Demand 44 accidents and 44 aircraft

The preferred approach would be to assess each element of any rulemaking as a discrete item. However, the statutory directions include elements that are difficult to isolate. Where possible the elements are addressed separately, but they are not assigned discrete scores. Instead scores are assigned for the entire rulemaking as a single proposal.

Scores represent the likelihood that the proposed rules could have prevented the accident had the rules been in place at the time of each accident as a result of the rule’s capacity to have precluded or corrected a contributing condition. The rule was first scored against each accident qualitatively with the following categories of effectiveness: none; low; moderate; moderate-to- high; high; and very high. Those categories correspond to ranges in the following order: 0%, 15%, 35%, 55%, 75% and 90%.The end result was a quantified set of benefits in which, for example, a score of 35% against an accident would have reduced the risk by a theoretical .35 fatal accidents and a theoretical 35 fatalities. All scores were reviewed by a 3-person panel.

3 Assessment of PL 111-216

Note that the mere presence of some condition such as a pilot with less than the proposed flight hours did not necessarily mean that the rulemaking would have been effective against a particular accident scenario. Many accidents simply do not reflect at all on pilot performance. Where a pertinent issue was influential in an accident, its score depended on an assessment both of the degree to which the factor influenced the accident and the degree to which a new rule realistically could be expected to have changed the outcome of a particular accident, based on the details of each accident flight. Finally, as a matter of analytical principle, no accident received a score higher than 0.9 based on the assumption that we can never be certain that any intervention would eliminate all risk in a particular scenario. With these caveats, the analysis focused on the following factors:

Part 121 PIC with less than 2,500 hours Part 121 FO with less than 1,500 hours, 1,000 hours, and 750 hours Part 121 FO operating without an ATP Part 121 FO operating without a rating in the aircraft being flown Crew Resource Management Sterile Cockpit Procedures Professionalism Weather (thunderstorm, icing, turbulence, gusty crosswinds etc.) Stall-upset or unusual attitude

Part 121 operators had 354 accidents involving 370 Part 121 aircraft in the 10-year study period. The majority of the 354 Part 121 accidents involved low-level outcomes or isolated risk that did not affect most aircraft occupants. For example, turbulence accounted for 102 of the 354 accidents, all but a few of which involved single injuries, while gate-ramp accidents accounted for 88 accidents. Most of those accidents did not reflect on crew performance. Similarly, miscellaneous on-board events and deer strikes on runways accounted for another 10 accidents, and 12 more events qualified as accidents only because an injury occurred during a precautionary evacuation. These types of events account for 60 percent of the total accident set for Part 121 operators, while still others involved maintenance and other issues not related to crew performance. Many of the remainder involved relatively low-level outcomes, though with varying levels of risk, while others involved high-time PICs with high-time FOs with ATP ratings who likely would not be reached by this rulemaking. The point here is not to dismiss these events but to suggest that benefits, as measured by avoided accidents, fatalities, and serious injuries, often will be more modest than advocates might anticipate.

Data Sources and Data Quality

Accident reports and structured accident data from the National Transportation Safety Board (NTSB) were the primary data source, supplemented where necessary and possible with data from FAA. Accident data will be limited to 10 years due to the poor quality of data available in either NTSB or FAA reports on pilot experience for earlier accidents. FY 2001 was selected as the starting point because that was the point at which NTSB data on pilot flight hours began to capture co-pilots at least somewhat systematically.

4 Assessment of PL 111-216

However, several issues limit the applicability of some accidents even within this 10-year dataset. In the case of accidents involving U.S. operators in foreign countries, foreign governments do not always release the factual information, probable cause statement, and contributing factors. Information that is available through the NTSB for these accidents rarely include pilot data. Pilot data is also limited for domestic accidents that have relatively low-level outcomes, such as turbulence or ramp-gate accidents. In addition, other records randomly involve blank data cells for PIC or FO flight hours and FO certificates and type ratings.

Where any of these problems existed, the text of each NTSB accident report was reviewed, as were files in the FAA’s Office of Accident Investigation and Prevention. That exercise added or corrected data for some or all pertinent fields for more than 200 pilots, but some or all pertinent data on some pilots and causal factors is simply not available.

Useful flight-hour data was identified for 332 PICs among the 370 accident aircraft. All but 4 of the 332 PICs had more than 2,500 hours. Of the other 38 PICs, damage to 2 aircraft was found during maintenance, and the respective air carriers could not associate either accident with specific crews. Consistent with comments above about the frequent absence of flight-hour data in certain categories of accidents, 24 of the remaining 36 accidents for which PIC flight-hour data was not available did not reflect on pilot performance. Those 24 accidents involved maintenance issues, ramp events caused by ground staff, and turbulence injuries in clear air turbulence or after adequate warnings were ignored. Eight of the other 12 accidents involved large jets operating international flights and 3 involved large jets on domestic flights by major air carriers. Most or all of those captains would have had significant flight experience. Consequently, the absence of data on those PICs does not affect the findings in this report.

Useful data on flight hours or ratings was available for 326 FOs (88%). The 45 cases for which no data was available included the same 38 cases noted above. Consequently, again, the absence of data in the 45 records will not affect the findings of this report.

Results of the Analysis, Part 121 Operations

All but 4 PICs exceeded the straightforward 2,500-hour minimum. The alternate approach of requiring an ATP plus at least 1,000 hours of experience in air carrier operations likely would have affected the eligibility of a fifth PIC, whose total flight time marginally exceeded 2,500 hours. All 5 of those PICs operated with FOs who had commercial ratings rather than ATPs. Crew performance was a primary issue in 4 of those 5 accidents. The fifth accident was caused by a tug operator who struck a stationary regional jet at the gate. Though the rule would have affected the eligibility of both pilots to operate that flight, the rule would not have influenced the accident.

The proposed PIC minimum hours and the required ATP certificate to operate as an FO in Part 121 operations would have affected the eligibility of 83 accident crews in the 10-year accident set. However, crew performance was an issue in just 34 of those cases, and the outcome of some of those 34 would not have been affected by this rulemaking. For example, 2 fatal accidents in which FOs did not have ATP ratings illustrate how the rule could have affected the eligibility of the FOs without influencing the outcome: Flight 5481 (21 fatal) crashed

5 Assessment of PL 111-216 immediately after lift off from Charlotte, NC due to an improperly-rigged elevator; and Chalks Ocean Airways Flight 101, (20 fatal) crashed into the Atlantic Ocean when the wing failed in normal flight. Though both FOs would have been affected by the requirement to have an ATP, the rule would not have prevented the accidents, and they would have scored a “zero.”

Conversely, some of the other elements in the rulemaking, such as screening pilot applicants, preparing pilots better for the complex environment of air carrier operations, better training in stall recognition, and the importance of professionalism would have had some minimal chance of influencing additional accident outcomes.

The rulemaking was found to be effective in varying degrees against just 31 accidents, and in most cases the scores were “low” to “moderate-to-high” categories. There were no accidents that received scores of “very high.” Appendix 1 shows the scoring for each of the 31 Part 121 accidents against which the rulemaking received positive scores. Table 1, summarizes the scoring of the complete rulemaking against all 354 Part 121 accidents from FY 2001 through FY 2010.

Table 1: Scoring Summary in Part 121 Operations FY 2001 Through FY 2010

Qualitative Effectiveness Quantified Number of Accidents Score Score Accidents Avoided None 0 323 0 Low 0.15 17 2.55 Moderate 0.35 6 2.1 Moderate-to-High 0.55 7 3.85 High 0.75 1 0.75 Very High 0.90 0 0 Total 354 9.25

In the accidents that received a “high” score (75% reduction in the likelihood of the accident) or “moderate-to-high” (55%), crew performance was a significant role in the accident. These accidents are summarized below:

High (75%)

On October 16, 2001, an Embraer ERJ-145 operated by landed hard in Roanoke, VA. The PIC briefed a visual approach to runway 33 and indicated to the FO that a go-around was not an option due to surrounding terrain. The crew noted a quartering crosswind at 15 mph gusting to 21 mph. The FO said the approach was normal until 300 above the ground, where he called "one dot high" and Vref +5. Airspeed dropped abruptly, the stick shaker activated, and the PIC added power, but the aircraft landed hard before the engines could spool up. Damage included broken and cracked frames and stringers, popped rivets, and skin worn through in the lower aft pressure vessel, plus scraped skin on the lower

6 Assessment of PL 111-216

aft fuselage. The NTSB identified the probable cause as the PIC's failure to maintain airspeed. Factors included the failure of both pilots to follow company CRM and flight manual procedures and the PIC's improper approach briefing.

Both pilots were hired during a period of rapid expansion for the carrier and moved through what the NTSB report described as a “rushed’ upgrade program. The PIC had 2,548 hours with 200 in make-model. The commercially-rated FO had 1,850 hours with 90 hours in make-model. The PIC was hired 21 months prior to the accident and had been flying as an active flight instructor. The PIC had accrued 600 hours as an FO in the BE-1900 and qualified for upgrade to PIC in the ERJ based on seniority. The FO was hired with 400 hours, flew as an FO on the Beech 1900, then upgraded to the ERJ as an FO based on seniority.

Neither pilot had any prior experience in swept-wing aircraft nor had the carrier provided any jet indoctrination or training on swept wing aerodynamics. The rule’s emphasis on screening pilot applicants, professionalism, preparing pilots better for Part 121 operating environments, and more training in stall recognition would have likely prevented this accident.

Moderate-to-High (55%)

1. On August 13, 2004, a Convair 580 cargo flight operated by crashed on approach into Cincinnati, OH. The FO was killed, the PIC was seriously injured, and the aircraft was destroyed. During the flight from Memphis to Cincinnati, the FO was the flying pilot. Approximately 20 minutes before the accident, he noticed that there was a problem with the handling characteristics of the airplane. In a six-minute span, he commented five times that he was experiencing abnormal forces on the control yoke. He said, “Feels like I need a lot of force. It is pushing to the right for some reason. I don't know why…I don't know what's going on.” With the runway in sight, the FO again noted “What is wrong with this plane? It’s really funny. I got something all messed up here."

The PIC, who was preoccupied for most of the flight with reconciling his pre- flight weight and balance miscalculations, eventually advised the FO that they had a fuel imbalance because he, as the non-flying pilot, had erroneously left the cross-feed open. This oversight caused a prolonged and unintended fuel transfer that was prohibited in the Aircraft Flight Manual (AFM) and the company Flight Operations Manual and was placarded in the cockpit. The FO then stated that, "We're going to flame out." However, the PIC responded, "I got the cross-feed open. Just keep the power on." Despite having adequate total fuel on board, the aircraft lost power due to fuel starvation and crashed one mile short of the runway.

The PIC had 2,500 hours of flight time (not including the hours accumulated from 20 years of experience as a flight engineer on the Lockheed Electra, Douglas DC- 8, and Lockheed L-1011). The commercially rated FO had 2,488 total flight

7 Assessment of PL 111-216

hours, which included 1,564 hours as a Boeing 727 flight engineer and 924 hours as either PIC or FO of mostly single-engine airplanes. He had 3 months of experience as an air carrier pilot. The rule would have required more experience and seasoning for both pilots, especially for the FO, which might have enabled him to identify that a rather large fuel imbalance between the left and right wing fuel tanks was the cause of his lateral control problem. Additional experience may have also led the PIC to reconcile his weight and balance prior to departure, complete his checklists at the times prescribed in the operations manual, and respond with more urgency when he finally recognized the fuel imbalance.

2. On July 6, 2008 a McDonnell-Douglas (Boeing) DC-9, operated by USA Jet on a cargo flight, crashed in heavy fog while on approach in mountainous terrain to Saltillo, Mexico. The captain, who was the flying pilot, was killed, and the first officer sustained serious injuries. The flight data recorder shows the aircraft alternating above and below the glide slope and left and right of the localizer throughout the approach. The aircraft ultimately stalled and crashed 1,000 feet short and 500 feet right of the extended runway centerline.

The captain had 6,630 total flight hours and 2,111 in the DC-9. The FO on that flight was an experienced flight engineer but had just 830 flight hours and 75 hours as a part 121 pilot. The deficiencies in piloting skill, decision-making, CRM, and operations in a multi-crew environment that contributed to this accident are addressed in the proposed rulemaking. Had the first officer been more experienced and better trained, he would have been more likely to have intervened or called for the go around as the approach became increasingly unstable. Either form of intervention would have possibly averted this accident.

3. On February 10, 2001, a Beechcraft BE-1900D operated by Great Lakes Aviation landed unintentionally gear-up on runway 4R at Chicago’s O’Hare International Airport. Seven of the 15 passengers and two pilots received minor injuries, and the aircraft was substantially damaged. According to the PIC, the crew was executing the “Flaps-Up Landing Checklist” abnormal procedure because the wing flap system was inoperative. Neither the PIC nor the FO reported hearing the landing gear warning horn, and the FO indicated that, “No gear horn sounded nor did we ever think the landing gear were not down.”

When the aircraft was craned off the runway and electrical power applied, the landing gear warning horn sounded, a red light illuminated in the gear handle, and all three landing gear position lights illuminated red. The NTSB determined that the probable cause of the accident was the flight crew not lowering the landing gear or verifying its position with three separate checklists. Contributing to the accident was the flight crew’s failure to comply with the minimum equipment list placard that prohibited the silencing of the landing gear warning horn.

The PIC held an ATP certificate with 2,400 flight hours with 600 hours in the BE- 1900D, 580 of which were PIC. The FO held a commercial license with 1449

8 Assessment of PL 111-216

flight hours, 191 of which were SIC. With the additional experience required by the proposed rulemaking and the emphasis on crew coordination, leadership, and pilot professionalism, it is possible that the flight crew would have been more vigilant in their completion of checklists especially when flying with inoperative equipment.

4. On June 20, 2007, a Beechcraft BE-1900D operated by Great Lakes Aviation landed long and bounced in visual conditions at Laramie, WY. The crew tried to slow the aircraft but turned onto a taxiway at high speed, where the right propeller struck an electrical box. A propeller blade then broke off and struck the right side of the fuselage. The PIC later noted that the approach was about 14 knots above the targeted speed, and the airplane was one dot high on the glide slope. No injuries occurred to the 2 pilots and 9 passengers, but the aircraft was substantially damaged. The NTSB determined the probable cause of the accident was the PIC’s improper decision making, his misjudgment of speed and distance, and his failure to go around during an unstable approach. Contributing to the accident was improper crew coordination, and the FO's failure to intervene during the unstable approach and landing.

The PIC had 2,250 hours of pilot experience and the commercially-rated FO had 774 hours. The rule would have required more experience and seasoning for both pilots in an air carrier environment which may well have led the PIC to go around upon recognizing an unstable approach or might have compelled the FO to intervene.

5. On January 6, 2003, an Embraer EMB-145LR operated by ExpressJet Airlines was substantially damaged when it overran the end of a snow-covered runway during landing in Cleveland, OH. The captain, who was the flying pilot, allowed the aircraft to float excessively and was unable to stop the aircraft on the remaining runway after touchdown. There were no reports of injuries to the crew, the flight attendant, or the passengers.

The flight left the departure airport with a computerized flight plan, weather, and the Notices to Airman (NOTAMs) for their route of flight. While being vectored for the approach, the controller informed the flight crew that the runway visual range (RVR) was had decreased from 4,500 feet to 4,000 feet, which was the minimum listed on the approach plate used by the flight crew. However, the FAA had released a NOTAM a month prior which had raised the landing minimums to 5,000 feet. This NOTAM was not received by the airline from their weather service provider and was consequently not issued to the flight crew. The approach controller was in possession of this NOTAM, but was not required to inform the flight crew that the approach was below landing minimums.

The flight crew conducted an instrument approach briefing, but did not discuss the runway conditions or the amount of runway required to land (nor was it required). They were, however, in possession of a local NOTAM that described

9 Assessment of PL 111-216

the landing runway as “covered with thin wet snow.” Landing distance charts indicate that EMB-145 flight crews should add 20 percent to the normal landing distance when using a wet runway. The captain reported that he used a rule of thumb computation to determine the landing distance because he did not have enough time to ask the first officer to compute it.

The NTSB determined the probable cause of the accident was the captain's failure to attain a proper touchdown on runway, and his subsequent failure to perform a go-around, both of which resulted in a runway overrun. Factors were the company's inadequate dispatch procedures with their failure to provide current NOTAMs, and the snow covered runway.

The captain held an Airline Transport Pilot certificate with 3,765 hours and 661 in the EMB-145. The first officer held a commercial pilot certificate with 2,550 hours and 804 in make and model. While there were several contributing factors to this accident beyond the flight crew’s control, they would have benefitted from enhanced CRM and a better understanding of transport airplane performance (specifically contaminated runway considerations for takeoff and landing). Had they been exposed to the rigors of the training outlined in the proposed rulemaking they would have been more likely to consider all factors in their approach planning and more likely to call for and execute a go around.

6. On August 14, 2008, the tail section of an Embraer EMB-145LR operated by struck the runway during landing in St. Louis, MO. There were no reported injuries to the crew or the passengers, and the aircraft was substantially damaged.

According to the accident report, the first officer indicated during the en-route portion of the flight that his landings with the flaps set at 45-degrees were, “a little extreme…a little non-standard, but way more fun.” As the aircraft entered the landing flare, approximately 34 feet above the touchdown zone, the flight data recorder indicated that the pitch angle during the landing flare increased to 13 degrees and the angle of attack to 17 degrees. Moments later the stick shaker/stall warning system activated at which time the tail section impacted the runway.

The NTSB determined that the probable cause of the accident was the FO’s excessive angle of attack during the landing flare resulting in abnormal runway contact. Both pilots held ATP certificates. The captain had 13,000 total hours and 4,000 in the EMB-145. The first officer had 1,800 total hours and 1,000 in type. Despite his certification and hours, the first officer’s statements reflect a lack of understanding of aerodynamics, stalls, and professionalism that is expected of air carrier pilots. His lack of flying skills and professionalism could have been offset by a captain who heeded the warnings and exercised the kind of enhanced leadership and CRM skills outlined in the proposed rulemaking. This additional training and experience would have increased the performance of both the captain and the first officer and might have helped to prevent this accident.

10 Assessment of PL 111-216

7. On November 20, 2000, an ATR-72 operated by Atlantic Southeast Airlines (ASA) Inc., encountered moderate turbulence while maneuvering for landing at 10,000 feet above mean sea level. The fasten seat belt sign was “ON,” and the two flight attendants were preparing the cabin for landing. During the encounter, one of the two flight attendants was seriously injured. No other injuries were reported by the flight crew, the other flight attendant, or the 36 passengers on board.

Prior to the flight, hazardous weather advisories (SIGMET and AIRMET) were issued calling for large areas of moderate turbulence along the route of flight. The captain was in possession of these weather reports and was required by company policy to brief the entire flight crew on the expected weather along the route and any forecasts of turbulence. According to the accident report, however, the captain briefed only one of the two flight attendants at which time he related that, “it would be a little rough getting out of Atlanta.”

The captain held an ATP certificate with 4,234 flight hours and 2,510 in the ATR- 72, and the first officer held a commercial pilot’s license. The NTSB determined that the probable cause of this accident was an in flight encounter with forecasted turbulence which resulted in serious injury to a flight attendant. Contributing to the accident was the captain’s inadequate briefing to the cabin crew, which was required by the airline’s Standard Practice manual. Had the captain and the first officer been exposed to the enhanced meteorology, CRM, and leadership training of the proposed rulemaking, they likely would have taken the forecasted turbulence listed in the hazardous weather reports more seriously and ensured that both flight attendants were briefed accordingly prior to the flight. These actions could have prevented the injuries sustained by the flight attendant.

The remaining 23 accidents in the 10-year accident set against which the rulemaking scored positively were limited to “low” and “moderate” scores. They included 2 fatal accidents (Colgan in Buffalo, NY and Corporate Airlines in Kirksville, MO), plus 3 serious injuries and hull losses to 3 turboprops and 1 cargo jet. The two fatal accidents received a score of “moderate” (35% reduction in the likelihood of the accident) and are summarized below:

1. On February 12, 2009, a Bombardier DHC-8-400, operated by , Inc., was on an instrument approach to Buffalo-Niagara International Airport, New York, when it crashed into a residence about five miles short of the runway after the PIC lost control of the aircraft killing the PIC, FO, two flight attendants, and all 45 passengers. One person on the ground was also killed, and the airplane was destroyed.

The NTSB determined that the probable cause of the accident was the PIC’s “inappropriate response to activation of the stick shaker, which led to an aerodynamic stall from which the airplane did not recover.” The PIC’s response was inappropriate because he pulled back on the control column rather than pushing it forward to reduce the angle of attack. As a result, the airplane’s pitch

11 Assessment of PL 111-216

increased and its airspeed decreased resulting in the stall. One contributing factor was both pilots’ failure to monitor airspeed via their primary flight displays (PFDs), and thus their failure to recognize the impending stick shaker onset as airspeed fell and pitch increased. The NTSB noted that the “failure of both pilots to detect this situation was the result of a significant breakdown in their monitoring responsibilities and workload management.” The PIC’s poor response suggests he was surprised by stick shaker activation. Another contributing factor was the crew’s lack of professionalism in failing to follow FAA regulations for sterile cockpit rules below 10,000 feet.

The crash might have been prevented by this rule’s emphasis on foundational training in pilot monitoring responsibilities, in particular, as well as foundational training in CRM, stall recognition and recovery, multi-crew operations, and professionalism.

2. On October 19, 2004, a BAE Systems BAE-J3201, operated by Corporate Airlines struck trees on final approach and crashed short of the runway in Kirksville, Missouri. The PIC, SIC, and 11 of the 13 passengers were killed. The two surviving passengers received serious injuries, and the aircraft was destroyed.

The airplane struck trees short of the runway threshold after the flight crew continued an unstabilized approach below the minimum descent altitude (MDA) without adequate visual reference of the runway environment. The crew failed to follow FAA regulations for non-precision approach procedures and failed to implement proper division of duties between the flying pilot and the monitoring pilot. The NTSB determined from the cockpit voice recorder (CVR) that both pilots were looking outside the cockpit window for the runway when they reached the MDA, which meant that no one was monitoring the airplane’s altitude. It was also determined that the crew engaged in unprofessional, non-essential conversation during the approach, which was a violation of regulations that prohibit such activity during flight below 10,000 feet.

The NTSB determined that the probable cause of the accident was the pilots’ failure to follow established procedures, properly conduct a non-precision approach, and their failure to adhere to established division of duties between the flying and non-flying pilot. Contributing factors in the accident were: the pilots’ failure to make standard callouts; failure to establish a professional demeanor and adhere to sterile cockpit rules; and fatigue.

The captain held an ATP certificate with 4,234 total flight hours and 2,510 in the BAE-J3201. The co-pilot held a commercial pilot certificate with 2,856 total flight hours and 107 hours in the BAE-J3201. Had the crew benefitted from the rulemaking’s emphasis on foundational training in multi-crew operations CRM, energy management, knowledge-based decision-making, leadership, and professionalism, it is possible that the accident might have been prevented.

12 Assessment of PL 111-216

In sum, the rulemaking would require a minimum of 2,500 hours for Part 121 captains, an ATP for Part 121 FOs, and an emphasis on screening pilot applicants, preparing pilots better for Part 121 operating environments, professionalism, and more training in stall recognition. These provisions would have had some degree of influence on 31 of the 354 accidents in the Part 121 data set (8.8%), but likely would have reduced the risk of accidents by a theoretical score of just 9.25 accidents among 354 accidents over 10 years (2.6%). The rulemaking also would have averted a theoretical 23.1 fatalities (5.3%), 3.7 serious injuries (1.9%), and 11 minor injuries (2.9%) over the 10-year accident set.

Part 125 Operations.

A PIC on a Part 125 flight must have an ATP. The rulemaking would not require FOs to have an ATP. Therefore, the net effect of the rulemaking would be limited to the various emphasis areas that the rule specifies.

Part 125 operators had 5 accidents in the 10-year study period. They resulted in 4 hull losses, 1 fatality and 1 serious injury. All 5 captains had total flight time in excess of the proposed 1,500- hour minimum to obtain an ATP. Four of the 5 captains had more than 9,000 hours.

Two of the four hull losses were caused by uncontrollable engine fires in flight, and a third was caused by an engine failure that prevented the prop from feathering. One of these accidents led to an emergency landing and post-crash fire on snow-covered tundra in , while the other two led to intentional ditching. In the fourth hull-loss accident, the crew undershot the runway while landing to deliver fuel at a remote mining site in Alaska. The DC-4 had been modified with a raised cockpit above fuselage to accommodate an upward swinging, modified nose door, which “presented a different sight picture” than a normal DC-4. The FO had only a commercial certificate but the captain, who was the pilot flying, had nearly 22,000 hours and 13,600 in make- model. The rulemaking would not have influenced these scenarios.

The fifth accident involved wing-to-wing contact with a parked aircraft on a general aviation ramp in Newark, NJ. The NTSB cited the Part 125 crew’s visual misjudgment of clearing distance between aircraft. The accident report does not identify which pilot was flying the Part 125 Boeing 737-700, but the PIC had nearly 13,000 hours and 1,245 in type, and the FO held an ATP. The rulemaking is unlikely to affect visual perception of wing clearance. In sum, the proposed rulemaking would have had no effect on the 5 Part 125 accidents over the past 10 years.

Part 135 Commuters.

For Part 135 commuters, the rulemaking would affect only the requirements to obtain an ATP for pilots-in-command of multi-engine airplanes. Part 135 commuters had 14 accidents involving multi-engine airplanes over the 10-year study period. They resulted in 1 fatal accident with 2 fatalities, plus 4 hull losses. Just 2 of the 14 crews included FO’s, one of whom was an ATP. Total flight hours for these PICs ranged from 2,942 to nearly 25,000 hours.

13 Assessment of PL 111-216

Of the 14 accidents in the past 10-year dataset involving multi-engine commuter aircraft, 3 were caused by faulty maintenance and a fourth accident was caused by unforecast and unexpected turbulence, which was the only accident in which an FO was flying. Among the remaining 10 accidents, the broader elements in the rulemaking may have influenced 8 accidents, which included 1 hull loss and 7 aircraft with substantial damage. However, the rule scored highly against just one accident. The emphasis on professionalism, particularly the need to complete checklists and to follow procedures, plus the emphasis on weather were the primary sources of effectiveness in these accidents.

Overall, the rulemaking would have averted a theoretical 1.8 of the 14 accidents in 10 years. It also would have avoided a theoretical 0.35 fatalities and 0.45 serious injuries. Appendix 2 shows the scoring for the 8 commuter accidents against which the rulemaking received a positive score.

Part 135 On-Demand

In principle, the rulemaking would affect all PICs operating on-demand Part 135 services involving either turbojet airplanes or airplanes configured with 10 or more passenger seats. However, the latter category was not involved in any on-demand Part 135 accidents in the 10- year study period. Consequently, the on-demand accident set was limited to 41 of 44 accidents involving turbojet airplanes. (Three of the accidents occurred outside the United States and accident data was unavailable. Useful data was available on 38 of the 41 PICs and 37 of the 41 FOs.

As with other segments of commercial aviation outlined above, many accidents did not reflect on crew performance. The rulemaking received positive scores against 19 of the 44 accidents. There were 8 accidents that received a score of “moderate” (35%) and 11 accidents that received a score of “low” (15%). Scores were mostly a result of the targeted emphasis on professionalism, weather, and the value of recognizing go-around gates during unstable approaches (airmanship). The most significant accident to receive a “moderate” score involved an approach-to-landing accident in Aspen, CO. The accident is summarized below:

On March 29, 2001, N303GA, a Gulfstream III operated as a part 135 on-demand flight by the Avjet Corporation, crashed while on final approach into Aspen, CO. The airplane crashed 2,400 feet short of the runway threshold after the flight crew continued an unstabilized approach below the minimum descent altitude without having adequate visual reference to the runway. The two pilots, one flight attendant, and all 15 passengers were killed, and the aircraft was destroyed.

The flight was late departing Los Angeles, CA and was in jeopardy of missing the noise restriction curfew that was in effect at the time of the accident. The weather brief which the co-pilot received during preflight as well as the weather obtained en-route indicated that a visual approach would likely be available at the destination. When the aircraft neared Aspen, however, ATC informed the flight crew that three of the preceding arrivals had executed missed approaches due to deteriorating visibility at the end of the runway. Despite this knowledge, the first officer called for, and the captain briefed a visual approach to the landing runway.

14 Assessment of PL 111-216

According to the accident report, the flight crew made numerous procedural errors, demonstrated poor crew coordination, and the darkness and snow showers adversely affected the crew’s ability to avoid terrain. The NTSB also concluded that the presence of a passenger in the flight deck jump seat may have increased the pressure on the captain to complete the flight.

The NTSB determined that the probable cause of this accident was the pilots’ operation of the aircraft below the minimum descent altitude without having visual reference with the runway. Contributing to the accident was the vague wording of a NOTAM regarding the limitations on circling approaches at night in Aspen, the deteriorating weather, and the pressure on the captain to complete the flight.

The captain held and ATP certificate with 9,900 hours of total flight time and 1,475 in the Gulfstream III. The first officer held an ATP certificate with 5,500 total hours and 913 hours in the Gulfstream II and III. In spite of the considerable flight time and experience that both these pilots had, it is possible that the enhanced training requirements would have been effective in improving the performance and situational awareness of the flight crew during this accident.

In sum, the rulemaking would have avoided a theoretical 4.45 accidents (10.9%), 11 of a total of 32 fatalities (34.3%), and 2.85 of 11 serious injuries (25.9%). Appendix 3 shows the scoring for the 19 on-demand accidents against which the rulemaking received a positive score.

Summary

Based on a detailed review of 417 accidents over 10 years, the analysis found that the rulemaking would be most effective against accidents involving turbojets in on-demand Part 135 operations and against twin-engine aircraft accidents in Part 135 commuter operations. However, the rulemaking would have had a significant effect against only a small number of Part 121 accidents, and the rulemaking would have had no effect against the small number of Part 125 accidents that occurred over the 10-year study period.

The analysis also noted that a large percentage of accidents had no reflection on crew performance, due to the prevalence of ramp accidents caused by others, turbulence accidents that involve either passenger behavior or conditions that could not be anticipated, and several types of miscellaneous events. Nevertheless, the rulemaking could save a total of about 1.55 accidents per year, plus 3.4 fatalities and 0.7 serious injuries per year in Part 121 and Part 135 operations.

15 Assessment of PL 111-216

Appendix 1: Scoring Against 31 Part 121 Accidents in Which the Rulemaking Received Positive Scores, FY 2001 – FY 2010

High NTSB # Damage Injury Date Carrier Location Aircraft Service Fatal Score

NYC02LA013 Substantial Minor/None 10/16/2001 Mesa Roanoke, VA ERJ-145 Passenger 0 0.75

DEN07LA101 Substantial Minor/None 6/20/2007 Great Lakes Laramie, WY BE-1900D Passenger 0 0.55

DCA04MA068 Destroyed Fatal 8/13/2004 Air Tahoma Florence, KY CV-580 Cargo 1 0.55

CHI01FA084 Substantial Minor/None 2/10/2001 Great Lakes Chicago, IL BE-1900 Passenger 0 0.55

NYC03FA035 Substantial Minor/None 1/6/2003 ExpressJet Cleveland, OH ERJ-145LR Passenger 0 0.55

MIA01LA030 None Serious 11/20/2000 Atlantic Southeast Ashville, NC ATR-72 Passenger 0 0.55

DCA08RA077 Destroyed Fatal 7/6/2008 USA Jet Saltillo, Mexico DC-9-15F Cargo 1 0.55

DFW08CA215 Substantial Minor/None 8/14/2008 Trans States St. Louis, MO ERJ-145LR Passenger 0 0.55 Custom Air FTW04LA225 Substantial Minor/None 8/30/2004 Transport El Paso, TX B727 Cargo 0 0.35

DCA04MA045 Substantial Minor/None 5/9/2004 American Eagle San Juan, PR ATR-72 Passenger 0 0.35 Frontier Flying ANC01FA011 Substantial Minor/None 10/22/2000 Service Bethel, AK BE-1900 Passenger 0 0.35 Phonpei, DCA08WA071 Substantial Minor/None 5/16/2008 Asia Pacific Airlines Micronesia B727-200F Cargo 0 0.35

DCA09MA027 Destroyed Fatal 2/12/2009 Colgan Air Buffalo, NY DHC-Q400 Passenger 50 0.35

DCA05MA004 Destroyed Fatal 10/19/2004 Corporate Airlines Kirksville, MO BAE-32 Passenger 13 0.35

DCA07FA037 Substantial Minor/None 4/12/2007 Pinnacle Traverse City, MI CL-600-2B19 Passenger 0 0.15

NYC06LA191 Substantial Minor/None 7/24/2006 American Eagle Newark, NJ ERJ-135LR Passenger 0 0.15 Fort Lauderdale, DCA05MA099 Substantial Minor/None 9/18/2005 Spirit FL A321-231 Passenger 0 0.15

NYC05FA094 Substantial Minor/None 6/7/2005 UPS Louisville, KY MD-11F Cargo 0 0.15

DCA08FA018 Substantial Minor/None 12/16/2007 Providence, RI CL600-2B19 Passenger 0 0.15

DCA04MA082 Substantial Minor/None 9/19/2004 FedEx Memphis, TN MD-11 Cargo 0 0.15

DCA09FA048 Substantial Serious 5/6/2009 World Airways Baltimore, MD DC-10-30 Passenger 0 0.15

FTW03MA160 Substantial Minor/None 5/24/2003 Southwest Amarillo, TX B737-300 Passenger 0 0.15

CHI02LA170 Substantial Minor/None 6/16/2002 Vanguard Kansas City, MO MD-82 Passenger 0 0.15

LAX04LA050 Substantial Minor/None 11/14/2003 United San Francisco, CA B747-400 Passenger 0 0.15

CEN10LA363 None Serious 6/28/2010 American Eagle Longview, TX ERJ-145LR Passenger 0 0.15

DCA05MA071 None Serious 5/31/2005 Air Wisconsin Chicago, IL CL-600-2B19 Passenger 0 0.15

DFW05LA112 None Serious 4/29/2005 Southwest Little Rock, AR B737-700 Passenger 0 0.15

ERA09LA488 None Serious 8/27/2009 Airtran Inverness, FL B717-200 Passenger 0 0.15

CHI02LA111 None Serious 4/18/2002 ATA Springfield, MO B757 Passenger 0 0.15

CEN09MA142 Destroyed Serious 1/27/2009 Empire Lubbock, TX ATR-42-300 Cargo 0 0.15

DCA04WA043 Destroyed Minor/None 4/28/2004 Bogota, Colombia DC-10-30F Cargo 0 0.15

16 Assessment of PL 111-216

Appendix 2: Effectiveness of the Rulemaking Against 8 Part 135 Commuter Accidents in Which the Rulemaking Received Positive Scores, FY 2001 – FY 2010

Date Operator Damage Fatal Serious Location Aircraft Service Event Score Lost Control on 4/4/2004 Peninsula Airways Substantial 0 0 Unalaska, AK G-21 Commuter Landing 0.75 Warbelows Air 3/13/2002 Ventures Substantial 0 0 Fairbanks, AK PA-31 Commuter Landed Gear Up 0.15

8/24/2002 Cape Smythe Air Substantial 0 0 Nuiqsut, AK PA-31 Commuter Landed Gear Up 0.15 Treasure Cay, Ditched; Engine & 7/13/2003 Substantial 2 0 Bahamas CE-402C Commuter Pilot Response 0.15

9/17/2004 Alaska Substantial 0 0 Telida, AK PA-31-350 Commuter Landed Gear Up 0.15 Frontier Flying 2/19/2009 Service Substantial 0 1 Nome, AK PA-31-350 Commuter CFIT, Flat Terrain 0.15

2/1/2010 Substantial 0 0 Watertown, NY CE-402C Commuter Overran on Landing 0.15

3/15/2010 Destroyed 0 2 Kodiak, AK BN-21-21 Commuter LOC on Climbout 0.15

Appendix 3: Effectiveness of the Rulemaking Against 19 Part 135 On-Demand Accidents in Which the Rulemaking Received Positive Scores, FY 2001 – FY 2010

Sco Date Operator Damage Fatal Serious Location Aircraft Service Event re

12/20/2000 Dallas Substantial 0 0 Jackson, WY HS-125-700 Passenger App & Landing 0.15 Gulfstream 3/29/2001 Avjet Destroyed 18 0 Aspen, CO III Passenger CFIT 0.35

8/28/2001 Grand Aire Express Substantial 0 0 Detroit Falcon Cargo RE-Landing 0.35 Corporate Flight 3/25/2002 Management Substantial 0 0 Anderson, IN MU-300 Passenger RE – Landing 0.15 Corporate Flight 8/13/2002 International Destroyed 0 0 Big Bear City, CA CE-S550 Passenger RE – Landing 0.35 Care Flight 8/30/2002 International Destroyed 1 4 Lexington, KY Lear 25C Medical RE – Landing 0.15

4/8/2003 Grand Aire Express Destroyed 0 2 St Louis, MO Falcon Cargo Ditched, Fuel 0.35 West Coast 11/11/2003 Charters Substantial 0 0 Wheeling, IL CE-560XL Passenger RTO Overrun 0.15 Aztec Captital 2/20/2004 Corporation Substantial 0 1 Ft. Lauderdale Lear 25B Passenger RE – Landing 0.35

3/19/2004 Airnet Systems Substantial 0 0 Utica, NY Lear 35A Passenger Hard Landing 0.15 Air Castle 11/28/2004 Corporation Destroyed 3 3 Montrose, CO CL-600 Passenger LOC – Climbout 0.35 Aspen Base 7/15/2005 Operations Substantial 0 1 Eagle, CO Lear 35A Passenger Hard Landing 0.15 Netjets as 1/5/2006 Executive Substantial 0 0 Woodruff, WI CE-560 Passenger RE – Landing 0.15 International Jet 6/2/2006 Charter Destroyed 2 0 Groton, CT Lear 35A Passenger CFIT – Water 0.35

1/10/2007 Airnet Systems Substantial 0 0 Columbus, OH Lear 35 Cargo LOC – Flight 0.15

3/17/2007 Air Trek Substantial 0 0 Beverly, MA CE-500 Med App & Landing 0.15

8/23/2007 Jet Share US Substantial 0 0 Westhampton, NY Lear 60 Passenger App & Landing 0.15

7/31/2008 East Coast Jets Destroyed 8 0 Owatonna, MN HS-125-800 Passenger App & Landing 0.35

2/24/2009 Bankair Substantial 0 0 Kansas City, MO Lear 35A Cargo App & Landing 0.15

17