II. AIRCRAFT PERFORMANCE

This section is intended to provide a review of CMR 3272’s flight profile based upon DFDR information, identify and discuss previous EMB-120 ice induced roll upsets and the similarities between them, and discuss EMB-120 handling qualities through certification requirements and flight test results.

Conclusions: . FAR 23 (Air Taxi Operations) contains more conservative roll control stall requirements than FAR 25 (Transport Category Aircraft Operations). . Initial high angle-of-attack, dry-air flight testing conducted on the EMB-120 aircraft identified the potential for uncontrollable roll-offs and high roll rates. i . SLD, quarter-round testing conducted in September of 1996 identified potential autopilot Inadequacies, high roll rates and high control column and control wheel forces. . A review of the previous EMB-120 ice induced roll upsets by the FAA in 1996 identified significant handling quality issues for the EMB-120 aircraft. . A comparison of the CMR 3272 accident profile and the other EMB-120 ice induced roll upsets show signifcant similarities in aircraft handling and performance. . A review of both the CMR 3272 and WestAir 7233 aircraft response to engine torque indicates that both aircraft were experiencing high drag at the time of the upsets. . The flightcrew of CMR 3272 was conducting their flight in accordance with ATC clearances and company guidance. . As the aircraft reached its target airspeed of 150 knots (as directed by ATC), signitIcant engine torque inputs by the flightcrew did not have any acceleration effect on the aircraft’s airspeed. . The autopilot was unable to overcome the left roll rate the aircraft was experiencing during the turn to a heading of 090”. . As the autopilot disconnected, the aircraft experienced significant roll and pitch excursions. . The angle-of-attack where the upset occurred was below the threshold of the Stall Avoidance System certificated and installed on the accident aircraft. . The results of Embraer simulator studies in January 1998 showed that the timing of engine torque inputs was critical in avoiding an upset. . The results of Embraer simulator studies in January 1998 showed that the use of the autopilot was a critical factor in causing an upset.

NOTE: Several of the documents referenced within this section are Proprietary in nature and are paraphrased. The documents have been requested by the Aircraft Performance Group and remain in the possession of the NTSB Aircraft Performance Group Chairman.

7 A. CMR 3272FDR / FLIGHT PROFILE REVIEW

Information from the flight data recorder (FDR) and cockpit voice recorder (CVR) indicates that as the airplane was descending from an altitude of 6,000’ to 4,000’ with the autopilot engaged and wing flaps zero, air traffic control (ATC) instructed the Comair 3272 flightcrew to ‘I.__turn right heading one eight zero, reduce speed lo onefive zero”. The FDR shows the aircraft rolled out on the assigned 180” heading as it was descending through 4,100 feet. ATC then instructed the flight to ‘I... turn left heading zero nine zero-plan a vector across the localizer”. Comair 3212 then began the left turn and leveled off at 4,000 feet with the airspeed stabilizing at 150 KIAS per the ATC clearance. Within one second of the airspeed reaching 150 KIAS (-8 seconds prior to the upset) the FDR shows engine torque steadily increasing to 100% at the time of the upset.

It can be assumed that, to achieve the left turn to the 090” heading as instructed by ATC, the flightcrew entered 090” in the heading selector of the autopilot. The autopilot then initiated a left wing down (LWD) input to a maximum target of 25”angle of bank. As the bank (roll) angle reached approximately 20” LWD, the FDR shows the c&trol wheel moving in an opposite direction to command right wing down (RWD) in an attempt to control the LWD roll rate. The left bank angle gradually increased beyond the autopilot target of 25 degrees LWD while the pilots began commanding a torque increase of over 90 percent. The RWD autopilot inputs continued to increase. FDR data also shows the autopilot inputting airplane nose-up pitch trim during the turn, although the pitch attitude of the aircraft remained at approximately 3” nose-up while airspeed gradually reduced to 147 KIAS and altitude remained at 4.000’.

As the bank angle reached 45” LWD, the autopilot bank angle limit was exceeded and the autopilot disconnected. The stick shaker activated momentarily while both engine torque parameters recorded readings well over 100 percent. Prior to the autopilot disconnect, the control wheel was deflected about 20” to the right (approximately half travel). After the autopilot disconnected, the control wheel abruptly deflected to at least 20” to the left, and the aircraft rolled abruptly from 45” LWD to 140’ LWD in approximately two seconds. At the same time, pitch attitude rapidly decreasedfrom a 3” nose-up attitude to 50” nose-down attitude. There was only a momentary sound of stick shaker activation at the time of the upset, and there were no aural indications of aircraft buffet prior to the upset. At the time of the upset, the airspeed was approximately 147 knots. After the initial upset, engine torque is immediately reduced to a level consistent with flight idle. . During the recovery attempt, the airplane experienced large oscillations in roll attitude and pitch oscillations between 20 degreesnose-down and 80 degrees nose-down until it impacted the ground in a steep nose-down attitude approximately 17 seconds after the initial upset.

B. EMB-120 HANDLING OUALITIES

1. Initial Dry-Air/High Angle-of-Attack Flight Testing

During the Aircraft Performance Group meeting conducted in Brazil in January of 1998, the group reviewed volumes of documents relating to the handling qualities of the EMB-120 aircraft. The testing addressedin these documents varied from original high angle-of-attack (AOA) dry-air testing

8 to the most recent Supercooled Large Droplet (SLD) testing in 1995/6. From as early as the original high AOA testing, the aircraft appearedto exhibit controllability problems at high AOA’s’. As the aircraft approached approximately 18” AOA, it had a tendency to roll-off rapidly to the left. In some instances, test pilots.described high roll rates and uncontrollable roll-offs.

It is necessary to point out that these tests were conducted on a clean aircraft (i.e. no ice contamination). It must also be highlighted that, with ice contamination on the airframe (in this case, specifically the wings), the wing can stall at a significantly lower AOA than the stall AOA for a clean wing.

These high roll rates and uncontrollable roll-offs were the reason for the introduction of a stall identification system (stick shaker and stick pusher) on the EMB- 120 aircraft. This stall i identification system is designed to activate at a much lower AOA than those identified during the high AOA, clean aircraft testing. The system is intended to activate well ahead of any roll-off event and provide warning to the flightcrew that the aircraft is approaching an aerodynamic stall. The EMB-120 stick shaker and stick pusher activation AOA’s are 10.5” and 12.2” respectively. Both of these thresholds are based on a dry, uncontaminated wing. There is no allowance or adjustment made for ice contamination of any degree.

Embraer conducted quarter round testing on an EMB-120 aircraft in September of 1995 to quantify any aircraft performance and handling qualities problems as a result of an SLD encounter. Embraer arrived at several significant conclusions*. These conclusions bear a strong resemblance to the events surrounding the CMR 3272 upset. Due to the proprietary nature of the document, several significant conclusions are paraphrasedhere:

l The EMB-120 autopilot will keep the wings level until the roll servo torque is reached. After that, the torque will remain constant and the bank angle will increase to 45’ where the autopilot will automatically disconnect due to an excessive bank angle attitude. l In the event of asymmetric ice accretions while flying, the airplane will roll fast and high stick forces will be required to recover the wings level attitude.

The significance of the above bullet items are their strong resemblance to the facts surrounding the CMR 3272 accident aircraft, considering the fact that CMR 3272 was operating for a short period of time (-30-40 seconds) in a micron range characterized by the weather group as between 30-80 microns. Although the aircraft was in a turn, the autopilot was attempting to correct the left roll rate by inputting right control wheel. When the bank angle reached the autopilot bank angle limit of 45”, & the autopilot disconnected and the aircraft rolled rapidly to the left. We have no sense as to the resulting control forces on the accident aircraft.

1 Excerpts from the original high AOA, dry-air flight test data was requested of Embraer by the NTSB Aircraft Performance Group during the January 1998 meeting in Brazil. This data remains proprietary, but should be in the possession of the Aircraft Performance Group Chairman. 2 Reference Embraer document 120.EV-165, C&trol Demadation Susceptibility Following Operation in Supercooled Large Droplet Icing Environment. This document has been requested of Embraer by the NTSB Aircraft Performance Group Chairman and should be in the possession of the Aircraft Performance Group Chaimun.

9 2. Stall Behavior - Design Considerations

Section 6 of Advisory Circular (AC) 25-7, the Flight Test Guide for Transport Category Airplanes, provides guidance material for the testing of an aircraft’s stall characteristics. Paragraph b (3)(ii) of this AC describes one of four acceptable means for the recognition of the stall as “an uncommanded, distinctiveand easily recognizablenose down pitch that cannotbe readily arrested. Thisnose down pitch may be accompaniedby a rolling motion that is not immediatelycontrollable, provided that the rolling motion complieswith FAR 25.203(b)or (c), as appropriate.” One would expect and hope that the rolling motion requirements for Transport Category Aircraft (FAR 25) would be, if not consistent with, more conservative and restrictive than that of Air Taxi Operations (FAR 23). This is not the case.

Federal Aviation Regulation (FAR) 25 deals specifically with Transport Category Aircraft. Section i 203(b) of FAR 25, as referenced in AC 25-7 states that,

“For level wing stalls, the roll occurring betweenthe stall and the completionof the recoverymay not exceedapproximately 20 degrees.”

FAR 23, however, deals specifically with helicopter and air taxi operations only. Section 201(d) of FAR 23, which also deals with rolling motion compliance, provides a more conservative number than FAR 25. This section states that,

“During the entry into and the recoveryfrom the maneuver,it must be possibleto prevent more than 15 degreesof roll or yaw by the normal use of controls.”

In the case of turning stalls however, FAR 25 allows for considerably greater roll excursions. FAR 25.203(c) states that,

‘For turningflight stalls, the action of the airplaneafter the stall may not be so violent or extremeas to makeit d@icult, with normal piloting skill, to effect a prompt recover?,and to regain control of the airplane. Themaximum bank angle that occurs during the recovery may not exceed- (I) Approximately60 degreesin the original direction of the turn, or 30 degreesin the oppositedirection, for decelerationrates up to I knot per second;and a (2) Approximately90 degreesin the original direction of the turn, or 60 degreesin the oppositedirection, for decelerationrates in excessof I knotper second.”

FAR 23.203 provides essentially the same requirements.

In order to satisfy these requirements, it is important to design the wing in a way so as to control the propagation of the stall. A number of design techniques can be utilized to insure that the wing stalls at the root first with the stalled condition then propagates outboard. Since the primary roll control surfaces, the ailerons, are typically located towards the tip of the wing, insuring that the stall develops at the root first also insures mat stall recognition will occur before roll control is lost.

IO The methods employed to force stall development at the root first include washout, in which the wing structure is slightly twisted to insure a higher angle-of-attack at the root. Other alternatives include leading edge radius and/or airfoil section changes between the root and the tip. However, any design features which involve complex changes to the wing geometry present manufacturing challenges, increased manufacturing costs, and possibly bring with them increased specific fuel consumption if it becomes necessary to use a less-than-optimum airfoil geometry in order to satisfy the requirements.

A January 1996 document produced by Flight Safety Foundatioi?, stated that

“Twist or washout helps to ensure that the symmetric stall starts inboard, and spread progressively, so that roll control is not lost. Greater ice accretion has probably occurred at the tip, leaving it more impaired aerodynamically than the inboard wing section. Stall, instead of starting inboard, may start at the tip. Because the tip section may have a sharper nose radius and probably has a shorter chord, it is a more efficient ice collector. ”

By washing out the wing, manufacturers lessen the angle of incidence as the wing progresses further out the span. By design, the outboard portion of the wing should be.affected by a stall last in the stall sequence. In this configuration, an aircraft wing stalls from the aft inboard section of the wing first then moves forward and outward. As this is true, aileron effectiveness can be maintained through the slowest of airspeeds into the stall. In the event that the tip stalls first, aileron effectiveness is compromised due to the detached airflow ahead of and over the aileron, making roll control extremely difficult, or in most cases, impossible until the airflow can be reattached.

The Aircraft Performance Group, subsequentto the January meeting in Brazil, posed the question of washout effects designed into the EMB-120 by Embraer. The answer came back to the group from Embraer that there are no washout effects designed into the EMB-120 aircraft. There is a 2” angle of incidence between the wing and the fuselage at the wing root and at the wing tip.

There is an alternative available to the designer. If the airplane cannot meet the roll requirements at and immediately after the stall, then stall recognition can be advanced by the use of a “stall identification device that is a strong and effective deterrent tofarther speed reduction.” Typically, this is an audible alert (stick shaker). The audible alert is intended to activate at a much lower angle- of-attack than aerodynamic stall would normally occur on a clean wing, giving the flightcrew a clear indication that a stall is impending and providing them sufficient time to execute a stall avoidance maneuver.

In the event that the stall deepens in spite of the audible alert, a stick pusher system is employed. The stick pusher system applies a nose down force to the controls at a predetermined angle-of- attack. The stick pusher can meet the requirements of stall identification defined in Advisory Circular 25-7. By advancing the activation point of the stick pusher to an angle-of-attack lower than actual stall, the designer can achieve stall identification while roll control is retained, thus meeting the requirements for roll control as defined in 25.203.

3 Flight Safety Digest, Pilots Can Minimize the Likelihood of Roll Upset in Severe Icing, Flight Safety Foundation, January 1996, John P. Dow Sr., U.S. Federal Aviation Administration.

11 It should be noted that the installation of a stick pusher system on an aircraft is the “last resort” for the manufacturer. This type of stall avoidance system will be incorporated in the event that that particular aircraft’s stall characteristics and handling qualities are so severe as to not meet the requirements of stall identification and aircraft control. The incorporation of stick pusher system requires the manufacturer to meet additional restrictions within the FAR’s, thus costing the manufacturer additional time and money.

The stall characteristics of an ice-contaminated wing may be entirely different from those of the clean wing. If wing geometry features have been used to improve clean-wing stall characteristics, these features may also improve the stall characteristics of a contaminated wing, although it would I be very difficult to quantify such effects.

On the other hand, stall warning and/or stall identification devices are scheduled to operate at specific angles-of-attack. If the ice contaminated wing stalls at an angle-of-attack lower than that for which the scheduled warning or identification is intended, then the pilot loses the advantage of this information and its purpose is defeated. While some manufacturers have opted to reschedule the activation points for stall warning and/or identification devices when ice protection systems are operated, this is not presently required by Part 25.

ALPA is participating in ARAC working groups that are considering changes to Part 25 that would require stall warning/identification systems to be rescheduled in icing conditions. ALPA would like to point out, however, that the regulatory changes presently being considered would not adjust stall warning/identification systems for conditions outside of Appendix C. Furthermore, the changes are not intended to be retroactive. Thus they will not improve existing airplanes such as the EMB-120, but only future type certificates.

Regardlessof how well it is designed, any artificial stall warning/identification system must operate based on a set of assumptions. One such assumption is that the ice protection system (B’S) has been operated correctly. Presently, the typical operating procedure for a pneumatic de-ice boot system re- quires that ?4 to W’ (up to 1” in some cases) of ice be allowed to accrete prior to operating the boots. This is done for two reasons. First, the boot sheds ice more efficiently when used in this manner. Second, there is a concern among many in the industry that ice bridging may result if the boots are operated too soon. Ice bridging is a phenomenon in which ice accretes on an inflated boot, becoming strong enough to remain in place when the boot deflates to its resting position. In such a case, the a boots become ineffective, since they are then expanding within a chamber made by the ice.

Ice bridging has been questioned more and more rigorously in recent years. It is likely that it does . not exist in the case of modem de-ice boots with high inflation pressures and rapid cycle times. However, it may well have existed in the past, particularly in the case of older, slower cycle boots.

In any event, if an artificial stall warning/identification system is installed, the conditions required for its correct operation must be met. If, in the case of ice contamination, proper operation of the ice protection system is required in order to attain correct response of the stall warning/identification system(s), then the flight crew must be provided with a means to insure that the ice protection system is operated properly.

12 C. PREVIOUS EMEb120ICE INDUCED ROLL UPSETS

Years after the initial high AOA, dry-air testing, the aircraft’s history of documented roll-off events began. The first documented event occurred in 1989 over Klamath Falls, Oregon. Many of these events lacked any substantive documentation, and the NTSB investigated only one. In an FAA presentation given to all EMB-120 operators in November of 1995, one particular slide entitled Event History of the EMB- 120 stated that:

“While large ice features may be associated with severe consequences, even small amounts of ice on critical surfaces may be sufficient to cause significant degradation in performance and handling characteristics without natural warning such as high drag or buffet in advance of stall. J,

The FAA conducted two separatereviews of these “Airplane roll upsets” in which it appeared all upsets reviewed were all icing induced. The initial FAA review, dated January 26, 19964,was conducted by the FAA Small Aircraft Directorate as part of Phase Il of the FAA’s three phase icing program implemented after the ATR-72 accident over Roselawn, Indiana. The information contained in this document was presented at the All-Operators meeting, which was held at FAA Headquarters in November of 1995. The FAA and Embraer gave the presentation. The circumstances and details of each event were addressedin some detail at that meeting. The text contained in the document is fairly detailed and self-explanatory. As you can see, this document remained an FAA internal “draft”. It was not distributed to any of the attendees of the All- Operators meeting.

This initial FAA review identified three issues relating to the EMB-120 in icing conditions:

. A history of roll upsetevents;

l High roll control force characteristics that were identijiedin the screeningprogram conductedas part of the FAA’s overall actionsfollowing the ATR-72 accidentof October 31,1994; and l Evidenceof possibleuncleared and undetectedice on the tailplane..

More importantly, the findings resulting from the initial FAA review of the EMB-120 roll upsets revealed that:

l Aside from the ATR-42/72 and the MU-2, the EMB-120 had the “highestnumber of reported lossof control” events,

l The EMB-120 has demonstrated in service “unexpectedrapid onsetof unusuallyhigh drag with ice accretionvisible but not consideredsigni$cant enough by the crew to warrant operationof the deicingboots”, . “Total or partial wing stall resultingin roll excursionsin icing conditions”, . “The 160 kias recommendedholding speed may not provide adequatemargin abovestall...”

4 Statement of Issues Regarding the EMBRAER Model 120 Airplane Roll Upset Events, FAA Draft, January 26, 1996.

13 . “Buffet on&t with certain kinds of ice accretion may not be present in advance of stall and that the stall protection system may not provide suflcient margin above contaminated wing stall for certain probable icing conditions. ”

l The autopilot does not “provide suflcient characteristics to provide time for the pilot to react...toprevent roll upset”, . “a roll characteristic associated with ice that appears to be caused by a dz@erent mechanism than the one associated with the Roselawn ATR-72 accident,” . “...the EMB-120 airplane with certain kinds of ice accretion may not provide an adequate stall margin for airline pilots of average alerhzess, skill or strength”.

Immediately after the Comair 3272 accident, several parties requested a copy of the January 1996 FAA document. Upon receiving the request, the FAA Aircraft Certification Office (ACO) in Atlanta, convened a meeting between the Atlanta ACO, personnel of FAA Headquarters, the National Transportation Safety Board (NTSB), the Air Line Pilots Association (ALPA) and several airline representatives.

One purpose of the meeting was to address the request by several parties for the January 1996 document and present this incident information to the attendees. At this meeting, the second document found in this section, dated March 13, 19975,was distributed, presented, and discussed. This presentation was given jointly by Embraer and the Atlanta ACO.

The conclusions of the second FAA EMI- incident review are significantly different than the first. They make no reference to any handling quality difficulties with the EMB-20 aircraft. In fact, the only proposed corrective conclusions identified were:

. “...a more precise methodfor identzfiing the needfor activation of the de-ice system”, . mandatory activation of the boots with thejirst indication of ice...

l minimum maneuver speed,jlaps up, during cruise and descent, or revised stall prevention system activation schedules in icing conditions, should be considered. ”

In the Preliminarv Conclusions section where possible corrective actions are presented, none of the conclusions addressedthe handling qualities of the aircraft. However, the Summarv of Events does identify five oft the events as “uncommanded roll upsets”. P D. EMBRAER FLIGHT SIMULATOR STUDY

Beginning immediately after the CMR 3272 accident, and at the request of the NTSB Aircraft Performance Group, Embraer began to conduct a flight simulation study. They immediately began to review the flight data recorder information from the accident aircraft in an attempt to identify any aerodynamic data bank modifications that could be made to their simulator to replicate the accident aircraft performance as seen on the flight data recorder. The results of Embraer’s studies have been provided to the Aircraft Performance Group on a regular basis.

5 EMI- Icing EventDiscussion, FAA/CTA/Embraer/NTSB/ALPA/RAA, March 13, 1997,.4&nta Certification Office, Atlanta,Georgia.

14 Page 5 of Embraer document #120-AC-022 entitled Flight Test Pronosal for Plight Simulator Analvsis of the Comair 3272 Accident describes the aerodynamic data bank modifications. These modifications consisted of changes to the aircraft lift, drag, pitching moment, rolling moment and yawing moment.

When reviewing these modifications, however, one must be careful not to interpret them literally as degradations that occurred to the accident aircraft. The degradations are intended to introduce a roll rate into the simulator similar to that encountered by the accident aircraft. The Aircraft Performance Group agrees, in total, that the degradations introduced into the simulator are only one of an infinite number of possible combinations that will provide the same accident aircraft roll rate solution. Changes to the lift or drag components and corresponding changes to the “arm” to which those forces act will achieve the same results. Basically, the accident aircraft experienced a “roll rate”, a “yaw rate” and a “pitch rate” and the degradations made to the aerodynamic data bank of the simulator provided only one solution. The Aircraft Performance Group also agrees that, when applied to the simulator, the degradations introduced provide representative results.

During the flight simulator analysis conducted in January of 1998 by the Aircraft Performance Group in Brazil, approximately 35 simulator runs were made for varying upset entries. Parameters such as symmetric power application, asymmetric power application, timing of power application and autopilot usage were varied over the range of simulator runs. Several interesting results were noted:

l Symmetric power applications at or above 150 knots resulted in no upset, l Asymmetric power applications above 155 knots resulted in no upset,

l A manually flown descent (no autopilot) with power increases to maintain 150 knots resulted in no upset.

As part of the testing, several recoveries were attempted with varying degrees of success. ALPA views these results with a great amount of healthy skepticism and again cautions the reader to analyze the recovery test results carefully. The Aircraft Performance Group agrees that the results of any recoveries in the simulator should be questioned since an upset of this nature is a highly dynamic maneuver (widely varying aircraft attitudes and airspeeds). The manufacturer made it clear that the aerodynamic data bank in the simulator does not go into that degree of fidelity and valid i aircraft responsesduring these dynamic maneuvers are questionable, at best.

15 The following table provides a synopsis of the available facts for most of the EMB-120 icing upsets.

EVENT DATE ICE ICE AIRSPEE ! NOTES PROTECT10 AMOUNT D N OBSERVED WI I ACTIVATED Klamath Falls, OR June 1989 NO? Light 180 -> 160

l Max power applied

l Stick shaker as speed increasing . . +/- 30” rolls

Fort Smith, TX September 1991 No Insignificant 7 l Floor vibrations prior to upset Right bank excursion

Clermont, France November 1991 l MS decreased to 150 kts . 60” rolls

Pine Bluff, AR April 1993 No? NOW ? l Autopilot on

l No ice observed . 900 rolls

Elko, NV October 1994 NO Insignificant 150 l Autopilot on

l AK in turn

l A/c response unexpected

l 90” roll

Tallahassee, FL April 1995 Yes Trace 180 -> 140 l Noupset . Airspeed decrease

I I I I l Pitch increase Monroe, MI 1 January 1997 1 No? I unknown I 150 . AK exiting turn

(Comair 3272) l Autopilot on

l Autopilot unable to maintain bank angle . Excessive bank disconnected A&

l A/S decreased to 146 kts Sacramento, CA March 1998 Yes Light 147 (fdr) . Climbed to exit icing

(W&Air 7233) conditions P

l A/C exiting turn . Crew “felt” rumble prior to upset

l Crew disengaged AR

E. WESTAIR 7233 INCIDENT

On March 4, 1998 while enroute from Sacramento to San Francisco, an EMB-120 aircraft experienced another ice induced roll upset. While the aircraft was climbing to their assigned altitude

16