Flight Testing Angle-of-Attack Warning Combinations on Part 23 Aircraft

Jennifer Grey Geehan

Bachelor of Science Aerospace Engineering Florida Institute of Technology

A thesis submitted to the College of Engineering and Computing at Florida Institute of Technology in partial fulfillment of the requirements for the degree of:

Master of Science in Flight Test Engineering

Melbourne, Florida December 2017

Abstract

Flight Testing Angle-of-Attack Warning Combinations on Part 23 Aircraft

Jennifer Grey Geehan

Advisor: Brian A. Kish, Ph.D.

The high number of accidents involving General Aviation (GA) aircraft leads to a need for improved methods to warn pilots of high angles of attack (AOA) and impending stall. Such methods must be certified by the Federal Aviation Administration (FAA) under Federal Aviation Regulations Part 23 regarding their effectiveness in preventing departures from controlled flight. Therefore, methods of compliance must be developed to certify Part 23 aircraft equipped with AOA warning or limiting systems. The research objectives were to (a) determine methods for the evaluation of AOA warning and limiting systems and to (b) evaluate the effectiveness of baseline combinations of such systems. The warning systems included visual, aural, and haptic feedback cues. The work performed in this study was the design, implementation, and testing of an AOA indicator that intuitively displayed the aircraft’s AOA and its current flap configuration. Also tested were aural alerts that informed the pilot of both the state of the aircraft and the actions needed to prevent further energy decay. Finally, an active stick that provided haptic feedback by shaking, pushing, and stepwise increasing the stick force depending on AOA was tested. These systems were evaluated using both a traditional stall matrix approach and an innovative AOA tracking task. The test aircraft used was the Technical University of Munich’s fly-by-wire DA42.

The key results were that the stall matrix was found to be an effective means to establish compliance, while the AOA tracking task did not produce the desired repeatability and consistency and could potentially be unsafe. However, the tracking task was effective in monopolizing pilot awareness and this lack of situational awareness is believed to be a major cause in fatal GA accidents. From the visual, aural, iii and haptic feedback cues evaluated, all evaluation pilots preferred the AOA indicator developed for the project over current off-the-shelf products, the human recorded voice was the preferred aural alert, and the stick shaker was found to be very effective. The stick pusher, the only active system tested, needs further investigation for safe operation at low altitudes.

iv

Contents Introduction ...... 1 Materials and Methods ...... 11 2.1 Test Apparatus ...... 11 2.2 AOA Indicator ...... 16 2.3 Aural Cues ...... 19 2.4 The Active Stick ...... 20 2.4.1 Stick Shaker ...... 26 2.4.2 Stick Pusher ...... 26 2.5 Development of Tracking Task ...... 27 2.4 Creating the Test plan ...... 30 Results ...... 34 3.1 Data Analysis ...... 34 3.2 Results ...... 35 3.2.1 Qualitative Results ...... 38 3.2.2 Quantitative Results ...... 40 Conclusions ...... 59 4.1 Conclusions ...... 59 4.2 Future Work ...... 61 References: ...... 62 Appendices ...... 64 Appendix A: Matlab Code for Generating Plots ...... 64 Appendix B: Pilot Survey and Data ...... 69 Appendix C: Software Changes and Flight Log ...... 71 Appendix D: Stall Test Matrix ...... 72 Appendix E: Test Cards: 3 July 2017 ...... 73

v

List of Figures

Figure 1: Aircraft Upset Event Types per Flight Phase 2011-2015 [2] ...... 1 Figure 2: AOA Flow and Lift Curve [7] ...... 3 Figure 3: Probability of detection as a function of time available for search [10] ...... 5 Figure 4: Alpha Systems AOA Indicators [11] ...... 6 Figure 5: Pilot Learning Curve [10] ...... 9 Figure 6: Test Aircraft ...... 12 Figure 7: Hangar Control Room ...... 14 Figure 8: DA42 Standard Stall Speed [16] ...... 15 Figure 9: AOA Indicator Iterations ...... 18 Figure 10: AOA Indicator/EFCS Buttons ...... 19 Figure 11: Active stick force curve example (Koschlik [17]) ...... 22 Figure 12: Active Stick Installed ...... 25 Figure 13: FTE Station...... 26 Figure 14: Pitch and roll axis sum-of-sines command signals [19] ...... 27 Figure 15: Versions 1 and 2 of AOA Tracking Task ...... 28 Figure 16: Tracking Task in Cockpit ...... 29 Figure 17: Test Area [20] ...... 31 Figure 18: AOA change with Time...... 41 Figure 19: Pilot A Straight Stalls ...... 42 Figure 20: July 10 Flight 1 Pilot B Stall Matrix ...... 43 Figure 21: Tracking Task Version 1 July 5 ...... 44 Figure 22:Overall Simulated Traffic Pattern ...... 45 Figure 23: Overall Simulated Traffic Pattern ...... 46 Figure 24:Pattern 1 ...... 47 Figure 25: Pattern 1 ...... 48 Figure 26: Pattern 2 ...... 49 Figure 27: Pattern 2 ...... 49 Figure 28: Pattern 3 ...... 50 Figure 29: Pattern 3 ...... 51 Figure 30: Pattern 4 ...... 52 Figure 31: Pattern 4 ...... 53 Figure 32: Landing Data July 4 Flight 2 ...... 54 Figure 33: Landing Data July 4 Flight 2 ...... 55 Figure 34: July 7 Flight 2 Stick Pusher ...... 56 Figure 35: Stall Warning Implementation Pilot Survey ...... 58

vi

List of Abbreviations/Definitions

AFM: Airplane Flying Manual AOA: Angle of Attack AOS: Angle of Sideslip CFR: Code of Federal Regulations EASA: European Aviation Safety Administration EFCS: Experimental Flight Control System FAA: Federal Aviation Administration FAR: Federal Aviation Regulation FIT: Florida Institute of Technology FTE: Flight Test Engineer FTI: Flight Test Instrumentation GA: General Aviation GUI: Graphical User Interface ICAO: International Civil Aviation Organization KCAS: Calibrated airspeed in knots. Calibrated airspeed is the speed corrected for instrument and position error KTAS: True airspeed in knots. True airspeed is the speed of the airplane relative to the air around it and is the calibrated airspeed corrected for errors in altitude and temperature. LOC: Loss of Control NORSEE: Non-Required Safety-Enhancing Equipment FAR Part 23: Part 23 aircraft are considered to be normal, utility, and aerobatic airplanes with a maximum takeoff weight of 12500 lbs. and having 9 passengers or less. The commuter category is for multiengine airplanes with a maximum takeoff weight of 19000 lbs. PFD: Primary flight display PIO: Pilot Induced Oscillation POH: Pilot Operating Handbook

vii

TAA: Technically Advanced Aircraft TUM: Technical University of Munich VFR: Visual Flight Rules

viii

Acknowledgement

I would like to thank the following:

My advisor and committee member, Dr. Brian Kish, for his continued support throughout the process of writing and editing this thesis and for providing me with academic opportunities that allowed me to grow as a student and as an engineer. He allowed me freedom in my work, but offered directional input when deemed necessary.

His support and encouragement, were indispensable.

Dr. Ralph Kimberlin, committee member, evaluation pilot, and source of great knowledge about flight test and aviation overall. He has not only helped me to become a better engineer, but also a better and safer pilot. It has been a pleasure learning from him, and his energy and passion for the field is inspiring

Dr. Markus Wilde for his work on the project team and invaluable feedback on this thesis. His guidance throughout my time at Florida Tech has greatly contributed to my academic growth.

Dr. Stephen Cusick, for being a welcome addition to my committee, providing feedback and resources that helped develop my thesis.

Finally, I would like to acknowledge the FAA for the funding provided to complete this research, Diamond Aircraft for allowing the project team to use their facilities and providing safety pilots, the Flight Systems Dynamics Group at the

Technical University of Munich for their wonderful hospitality and making the project possible by use of their aircraft, and Mitchell Aerospace Research for his assistance in numerous aspects of the flight test campaign. ix

Section 1 Introduction

Over forty percent of fatal accidents in general aviation (GA) are attributed to loss of control in flight (LOC-I). While this percentage has steadily decreased from 60% in 2001 to currently 42%, the number amounts to 176 fatalities per year. [1] One of the reasons this number is so high is that when an aircraft is low and slow, as they are immediately after takeoff and on approach to landing, there is not sufficient time or altitude to recover. According to a study done by the European Aviation Safety Administration (EASA) on LOC-I for GA [2], the highest number of accidents occur during takeoff, followed closely by the combination of approach and landing flight phases, shown in Figure 1.

Figure 1: Aircraft Upset Event Types per Flight Phase 2011-2015 [2]

1

The Federal Aviation Administration (FAA), alongside industry partners, has been looking for a way to decrease these accident percentages further. One strategy is to make systems such as Angle-of-Attack (AOA) indicators easier to install with fewer regulations. This was done by taking the existing approval requirements and simplifying them for AOA indicators. [3] However, it is noted that these added systems must not add any more risk than they are preventing. [1]

A memorandum by the FAA, “Approval of Non-Required Angle of Attack (AOA) Indicator Systems” [4] in February of 2014, aimed to streamline implementation of AOA indicators in GA aircraft with hopes of preventing LOC accidents. Rather than going through the whole approval process for something considered a minor modification to the type design, this memorandum allowed AOA indicators to be installed by field approval or minor modification in the airplane’s maintenance records, in attempt to reduce cost, so long as it met certain criteria. This process was intended to serve as a prototype for future add-on safety system approvals.

Some of those criteria were: to have a qualitative evaluation of failure modes to ensure no adverse effect on safety of aircraft or pilot workload; the operating instructions must state the accuracy of the instrument; no conflicting information can be given in regard to the certified stall warning of the aircraft; the indicator must be standalone and cannot tap into any existing certified system of the aircraft except for electrical power needed for the display; and the indicator must be labeled “Not for use as a primary instrument for flight”. [4]

Unfortunately, this memorandum was only valid for three years following its release. For that reason, it needed to be expanded upon and updated. The result being the Policy Statement “Approval of Non-Required Safety Enhancing Equipment (NORSEE)”. [5] Equipment classified as NORSEE are meant to increase overall situational awareness by the use of non-primary aircraft systems. Items that are determined to be minor changes to the type design of the aircraft are included in this policy.

2

Federal Aviation Regulation (FAR) Part 23 aircraft are considered to be normal, utility, and aerobatic airplanes with a maximum takeoff weight of 12,500 lbs and having 9 passengers or less. Part 23 also includes a commuter category for multi-engine airplanes with a maximum takeoff weight of 19,000 lbs. and having 19 or fewer passengers. [6] The presented research, only considers non-commuter category airplanes when discussing Part 23 or GA aircraft, because that is the focus area the FAA funded Florida Institute of Technology to research.

AOA is defined as the angular difference between the oncoming airflow or wind and the direction of movement of the aircraft through the air. The reason this is so important to aircraft and their safe operation is that there is a critical AOA wherein the wing will lose lift as the flow becomes separated. Figure 2 depicts the AOA alongside a standard lift curve which shows the coefficient of lift versus the AOA. The critical AOA is the highest point on the curve where no further increase in lift is possible.

Figure 2: AOA Flow and Lift Curve [7]

Once the flow has become separated, it is no longer flowing smoothly over the control surfaces of the aircraft. Not only will the aircraft lose lift, but there is also the potential for losing controllability. Part of the definition for stall warning, according to

3 the FAA [8], is the reduction of aileron effectiveness, roll stability, and pitch authority. This can be thought of as loss of controllability. The stalling speed has different definitions, ranging from reaching the full aft stop of the control stick or yoke, to uncontrollable downward pitching motion. The FAA defines an aerodynamic stall as a “loss of lift caused by exceeding the airplane’s critical AOA” [8].

Most manufacturers have designed their airplanes favorably in this respect so that in the event of stall, the airplane can still be controlled, and departure can be averted. However, this requires the pilot to be aware of the situation. Since most GA airplanes do not have the capability of displaying AOA to the pilot, there are published stall speeds in the pilot operating handbooks. While this is a good estimate of when the aircraft will stall, an airplane can stall at any airspeed if the AOA exceeds the safe limit. Therefore, it is important to investigate informing the pilots about the current AOA to increase their knowledge of the energy state of the aircraft.

As pilot workload is a driving factor in maintaining an acceptable level of safety in aviation, it is not only important to provide the pilot with AOA information and warnings, but to also ensure that the mental workload associated with registering, processing, and understanding the information is minimized. Therefore, the additional information should be conveyed on channels that are not overburdened by the nominal operations in flight. The general means of conveying information to humans is through visual, aural, and haptic channel.

In a study done investigating haptic feedback on automotive touchscreens, when haptic feedback was added, subjective workload was reduced, and error rates improved [9]. An important finding from this study was that all participants noticed the haptic feedback during the training phase but not all could notice it during the evaluation task. This suggests that the attentional load of the participants was affected by the task and in turn, affected their ability to perceive this stimulus. Wickens [10] discusses this same phenomenon of selective or focused attention. It can be compared to tunnel vision, but involving all senses. Excessive attentional load can cause an otherwise capable pilot or

4 test participant to perform in a way that they would, under normal circumstances, deem irresponsible or dangerous.

In a world where aircraft are getting more and more “talkative” with warnings and reminders for everything from landing gear status to traffic, it is easy to understand how an aural warning for stall speed or a new sound for AOA warning could get ignored and unintentionally categorized into the nonessential background clutter noise.

The “cocktail party effect” describes the situation of being able to attend a loud party with many other people talking and having the ability to ignore the other conversations while speaking with one person. This is what Wickens describes as focusing auditory attention. One can see how in an emergency situation a pilot may become focused on one warning signal, such as low fuel, and might not hear a call out for low speed or high AOA. When time is a critical factor, as it is in aviation emergencies, this can cause rushed judgements and decrease the probability of detection of a change in critical flight systems. This is depicted graphically in Figure 3, where low search time available results in low probability of detection. [10]

Figure 3: Probability of detection as a function of time available for search [10]

Engineering psychologists have been studying how humans react to different stimuli for decades and have found that the placement of instruments in the aircraft cockpit affects their design and intended use. They have found that people are more likely to have a horizontal scan than a diagonal scan. If they can remember the position

5 of the instrument, they will not have to sample the information as often. [10] This is part of the reason why some of the more “old school” pilots prefer dial gauges over running tapes for things such as airspeed or altitude. That way they can do a quick scan, remember the relative positions of the instruments, determine rates, and have more time to either look outside of the cockpit or focus on figuring out what is wrong in the event of an anomaly or emergency.

There are currently several AOA indicators on the market with varying cost points and display options. These indicators have displays that range from digital to analog and most all of them have a green, yellow, red scale. They can be mounted in various places in the cockpit such as on the glare shield or on the instrument panel with the standard instruments for an easier visual scan of instruments.

While AOA has been used in military applications for decades, the cost to certify an AOA system for GA aircraft was prohibitive until the recently added FAA policies like NORSEE. There are many different styles of AOA indicators on the market, each with the aim to bring situational awareness to the pilot prior to a poor energy state. Figure 4 shows examples of all the different displays made by Alpha Systems. Alpha Systems is one of many companies who produce AOA indicators. It is shown here as an example because of the variety of displays that they produce. The varied displays and pilot preference are a couple of the difficulties when it comes to finding a standardized system or for training across the market. Due to pilot preference, someone might not get the full benefit out of the system because of either not understanding it or unwillingness to put in the time to get over the learning curve.

Figure 4: Alpha Systems AOA Indicators [11]

6

The AOA indicators available are considered user friendly with the installation, since they are classified as a minor modification and are not a primary flight instrument. The FAA is more lenient with the installation process, which can be noted in the aircraft’s logbook by a private pilot performing preventative maintenance or an Airframe & Powerplant mechanic. The installation does not require a supplemental type certificate. The instrument itself is a simple probe that can be mounted on a modified inspection plate under the aircraft’s wing with two air hoses that can be easily run from the wing to the instrument electronics inside of the airplane.

As mentioned earlier with aircraft becoming more “talkative”, these aircraft are also becoming more integrated with respect to displays. The FAA refers to these aircraft as Technically Advanced Aircraft (TAA). A TAA is an aircraft that has “a minimum of an IFR-certified GPS navigation system with a moving map display and an integrated autopilot” [12] with some also having weather, terrain and traffic displayed on a multi- function display. With all of these new functions, it is imperative that pilots receive proper training on how to use all the systems and develop a safe scan of the instruments to avoid fixation or omission. This more complex cockpit environment is why the FAA states that supplemental safety systems, such as AOA indicators, must have “benefits that outweigh the risks” [1]. Therefore, the FAA selected an AOA display for this research to be similar to the one used by Icon Aircraft. Details on the AOA display will be provided in the next chapter.

Although a side-stick was not used for the research presented in this thesis, an experiment conducted with a side-stick offered insight regarding flight tracking tasks aimed to raise the attentional load of the pilot, in order to test a system. For this thesis, that system would be how the pilot reacts under increased attentional load to varying sensory feedback. In the study [13] on a side-stick with force or displacement configurations, Bailey wrote about the flight test process for developing a side-stick controller for the X-20 project. The data collection process involved an evaluation problem, pilot scoring, pilot questionnaire, pilot opinion, and a ground simulator. This project also found that pilot opinion is one of the most important things that comes out

7 of a test campaign. The quantitative measurement of whether a system is effective or not is challenging as there are many different variables that depend on pilot feedback, making pilot opinion valuable despite its qualitative nature. The members of this project created a “challenging, repeatable, flight-tracking task” in order to measure pilot performance quantitatively. This task was to hold a constant 60° roll while following the cockpit prompts.

Since the aircraft from the X-20 program was fly-by-wire (FBW) and had adaptive control, the pitch axis, as seen by the pilot, was varied during the testing in order to measure the different pilot responses. Random noise was also added to simulate a real-world environment with gusts of wind. The purpose of this demanding task was to get the attentional load of the pilot so high that when any changes to the side-stick were made during this task, it could be seen in the data whether or not these changes improved operability of the aircraft or degraded it. This high attentional load is comparable to having tunnel vision but for other senses than just vision. They scored the pilot’s absolute value of attitude error in the turn.

One of the important outcomes of Bailey’s project was the adaptability of humans. Figure 5 is an example of a pilot learning curve. There is an initial learning curve that is very steep. Over the course of using the system, the operator finds little tricks or learns new techniques to increase effectiveness. Another conclusion was similar to what was found in the presented research. That is, the pilot opinion or scoring changed significantly from when on the ground doing simulations in the aircraft versus actually flying the same aircraft. Bailey came to the conclusion that a tracking task or “routine flight test procedure” was not sufficient for studying the pilot-aircraft interface due to the numerous variables changing with each pilot and each flight.

8

Figure 5: Pilot Learning Curve [10]

Many other studies have investigated the area of pilot workload with regards to different aspects of flight, such as force gradient on the stick. In Bromfield’s [14] experiment which investigated the effects on pilot mental demand with varying stick force gradient, it was found that the pilot workload significantly increased when the stick force gradient was decreased. When the stick force gradient was decreased, it effectively made the system more sensitive to inputs. This means that the pilots had to work harder to keep the same flight path as before the force gradient was changed.

It is important to note that this was a study done on GA pilots and not test pilots who have much broader experience. It was also purely simulation with no actual flights. High workload placed onto pilots sometimes tests the pilots more than the system in question. The intended effect is that the pilot is distracted, as if in an emergency or urgent situation, which cannot be accurately and safely simulated.

9

Since it is believed that one of the major causes of LOC is that the pilot becomes distracted and is not aware of the deteriorating condition of the aircraft, it is important that researchers investigate the effectiveness of different routes that the pilot’s awareness can be caught: aural, visual, and haptic. The aural route investigated in the presented research is varying computer-generated or human recorded phrases of the aircraft state and action needed by the pilot. The visual was an AOA display in the pilot’s line of sight, mounted on the glare shield. The haptic was via an active stick with force gradient, shaker, and pusher variants. It is hypothesized that by using all available sensory paths or a combination of a couple key paths, safety regarding LOC can be increased. This investigation into different sensory paths is mimicked in other aspects of aviation in the form of redundancy, which is having multiple systems that serve the same purpose such as alternate air intake, two communication and navigation radios, two magnetos, and so on.

10

Section 2

Materials and Methods

The materials and methods for the research completed in this project are outlined below. The materials consist of the test aircraft and all systems evaluated, AOA indicator, aural cues, and an active stick with force step, shaker, and pusher capabilities. The methods for the flight test campaign are also outlined in this section and include a stall matrix method, tracking tasks, and operational evaluations.

2.1 Test Apparatus The test apparatus for the flight testing was a modified FBW Diamond Aircraft DA-42 MPP (Multi-Purpose Platform) with a Garmin G1000 integrated avionics system owned by the Technical University of Munich (TUM) with the tail number OE-FSD. It was operated out of the Diamond Aircraft facility in Wiener Neustadt, Austria, at the airport with the International Civil Aviation Organization (ICAO) identifier LOAN and the surrounding airspace. The airport elevation is 896 ft [273.1 m] above mean sea level and all flights were performed under visual flight rules (VFR). Diamond safety pilots were used on all flights. The test aircraft can be seen below in Figure 6. It has a maximum airspeed of 171 KTAS and a ceiling of 18,000 ft [5486.4 m].

11

Figure 6: Test Aircraft

In addition to the standard pitot-static system of the DA-42, OE-FSD has a five- hole pitot-static nose boom capable of measuring AOA and angle of sideslip (AOS), which was used for the experimental flight control system (EFCS) input and a vane type probe mounted onto the left wing also capable of measuring AOA and AOS. This way the aircraft retains its airworthiness regardless of any changes to the EFCS. The flight test engineer (FTE) sitting in the backseat can change the EFCS input between the nose boom and the wing boom in flight. The data from both probes were recorded during flight for later analysis. The nose boom was chosen to be the AOA source for the AOA information and warning system, because it was less susceptible to errors from roll rate.

During the test flights, the aircraft operated as a normal airplane until the EFCS was turned on. This could be deactivated at any time by the safety pilot. The EFCS was only on during the maneuvers and was off during takeoff and landing. The flight crew roles and responsibilities during the test flights were as follows. The safe operation of the aircraft was dependent upon the safety pilot. He was in charge of taxiing, takeoff, speaking to air traffic control, staying within proper airspace limits, and landing. If at any time the safety pilot determined the aircraft was reaching an unsafe condition due to the evaluation pilot or the EFCS, he had the ability to quickly take over. The evaluation pilot’s role was to fly and evaluate the system while the EFCS was online.

12

This included evaluating stall matrix points, tracking tasks, and simulated patterns. The stall matrix points were done with various aural and haptic cues. The role of the FTE was to monitor the overall health of the EFCS during flight and if any change was requested by the evaluation pilot, such as active stick force or tracking task number, that change could be done directly from the FTE control station. The experience of the evaluation pilots ranged from newly certified pilots with low time to test pilots in excess of 8500 flight hours.

For aircraft-in-the-loop simulation, OE-FSD can be connected to a ground station running a detailed system dynamics model and interacting directly with the aircraft displays and interfaces. The forces placed on the active stick or the display of the indicators can be set prior to flight. These items could also be changed in flight, which meant that numerous configurations of the EFCS could be tested on one flight rather than one for each change. This ability to perform fewer flights to obtain the same amount of data saved time and cost.

Another functionality of the test bed was the ability to rapidly change or update the software. The code could easily be changed and compiled while on lunch break, tested on the aircraft using the ground station, then tested in flight. When in flight, the aircraft could be monitored or even controlled from the ground station via use of a datalink. The control room in the hangar proved to be a useful tool when the data link was operational, as the flight could be monitored in real time and the test team could communicate with each other if any questions came up.

13

Figure 7: Hangar Control Room Figure 7 shows the control room during one of the test flights. The cockpit camera can be seen, so the pilot actions can be monitored in real time as well as how the aircraft reacts to their actions. There is also a map with the aircraft location and a simulated view of the aircraft. The EFCS can also be monitored or manipulated from the ground to change parameters, such as when or how much the stick pusher activates.

As mentioned previously, stalling speed has different definitions, ranging from reaching the full aft stop of the control stick or yoke, to uncontrollable downward pitching motion. Now another definition of stall, not often used in small GA airplanes, is introduced as the “Downward pitching motion that results from the activation of a device (for example, stick pusher)” [15]. This is applicable since a stick pusher was used in this project. Figure 8 shows a stall speed chart from a generic DA42 Airplane Flying Manual (AFM) [16]. Each aircraft design has an AFM, and each individual airplane has a pilots operating handbook that will include differences in weight and balance for features specific to that aircraft such as air conditioning, autopilot, etc. In the case of the test aircraft, there was much more instrumentation installed for the EFCS, which changed the weight and balance and thus the stalling speeds. In the airplane’s AFM, the stall speeds are listed in both indicated and calibrated airspeed for different weights and

14 configurations, such as bank angle and flap/gear positions. It is noted that these are airspeeds for the most forward center of gravity and with the power idle.

Figure 8: DA42 Standard Stall Speed [16]

Although the speeds are listed for different weights and configurations, the actual stall speeds in the different configurations were determined by stalling the airplane in flight while recording the calibrated airspeed and the AOA in degrees. OE- FSD has more hardware installed than a production aircraft such as the above AFM stall speeds represent. Therefore, the stall speeds were tested in this fashion. It was also done in order to create a normalized AOA scale. This was crucial for the development of the AOA indicator. Stalls were done in the following three configurations; flaps up, flaps for approach, and flaps for landing. The stationary AOA values were also identified during straight and level flight to use as a comparison against these configuration changes. Table 1 shows AOA and calibrated airspeed in knots (KCAS) data from a preliminary flight to determine stall speeds for OE-FSD and to use as an initial attempt at normalizing the AOA values.

15

Table 1: AOA vs CAS values from Preliminary Flight

Flaps Up Flaps Approach Flaps Landing KCAS Actual AOA KCAS Actual AOA KCAS Actual AOA 151 -0.4° 126 -3.4° 94 -2.6° 134 0.4° 106 -1.8° 91 -2° 116 1.7° 99 -0.9° 84 -0.8° 109 2.7° 88 1.1° 88 1° 99 4.2° 83 2.4° 70 4.2° 95 4.5° 78 4.3° 68 6° 90 6.1° 74 5.6° 62 10° 85 7.1° 73 6.4° 80 8.2° 68.5 7.4° 76 8.9° 64 10° 70 12°

2.2 AOA Indicator The AOA values used in the AOA indicating and warning systems were normalized so that the indicator/warning logic and interface could work with any aircraft. To provide sufficient information for flight safety without overwhelming the pilot, the AOA Indicator was designed to display three levels in red, yellow and green to represent different ranges of AOA. Table 2 shows the final range of values for these levels of AOA for different aircraft configurations.

Table 2: AOA Normalization

Normalized Flaps UP Flaps APP Flaps LDG 1 14° 10° 9.2° 0.9 12° 8.7° 7.1° 0.8 10° 7.4° 6° 0.7 8.2° 4.3° 2.9° 0.6 6.9° 2.3° -0.1° 0.5 6° 0.8° -1.6°

As discussed in the previous sub-section, OE-FSD was stalled while recording the calibrated airspeed and AOA in the flaps; up, approach, and landing settings. These values were then plotted against each other along with the stall warning horn activation 16 to look for trends in the data. It was decided that the red range should start prior to the aircrafts lift detector inducing a stall warning or continuous sound in the cockpit, to give advanced energy state awareness to the pilot and to avoid nuisance of the experimental system activating at the same time as the certified system on the airplane. The red range of normalized AOA starting at 0.9, starts at an airspeed 2 KCAS below to the stall warning of the airplane itself. It is noted that this was determined to be an acceptable buffer in rate of AOA change prior to the stall warning and is not based solely on airspeed.

The normalized AOA value of 1 was set to when the actual stall was expected to occur. The top of the green range at 0.6 was set to coincide with the flaps landing configuration final approach airspeed as stated in the AFM. In the flaps up configuration, the 0.6 normalized value was chosen to coincide with the cruise climb speed stated in the AFM. The flaps approach value was chosen to be a point midway between the values decided for flaps up and flaps landing.

After the ranges had been determined, they were programmed in the AOA indicator. The display went through several iterations to make it the most efficient and easiest to understand by the pilot. Figure 9 shows three of the iterations of the indicator, with its maturity growing from left to right. In determining what the display should look like, market research was conducted on various commercially available, off-the-shelf AOA indicators. It was found that the calibration on the indicators was done such that the bottom of what is equivalent to the green arc in the custom display would be an AOA value of 0°. This is similar to what the final iteration of the custom indicator displays with the lowest value shown being the horizontal or “zero” axis. It is not uncommon to have slightly negative AOA values, but to get a larger negative AOA value such that stall occurs is not probable unless in aerobatic flight which is not of concern for this research. Thus, the negative AOA values can be ignored or “cut off” from the indicator.

17

Figure 9: AOA Indicator Iterations

In the earliest iteration of AOA indicator (left), the range of AOA values goes below the horizontal axis. The pointer is an airfoil shape, which could give the pilot the impression of having a negative AOA value and cause them to react inappropriately by pulling up to get back to a “normal” AOA. By making the horizontal line a hard stop of the green range, the red portion was shifted up, resulting in a vertical representation of the wing at maximum AOA. While this is unrealistic, it was deemed to be acceptable for the purposes of AOA indication. The pilot would still make the proper corrective action, whether it was pointed almost all the way vertical or only most of the way. This was one of the technical hurdles encountered in this project. The final iteration of the display also enhanced the indicator function by changing the color of the airfoil shape with the AOA range, and by showing the deflection of the flaps in the display. This makes the display quicker for the pilot to access during a cockpit scan. The flap-indicating feature is unique to the AOA indicator developed for this project. A considered factor was that of filtering. If the data were taken straight from the probes and then directly displayed, it would be erratic, and the pilot would not be able to get an accurate reading off the instrument. For this reason, a moving average filter was implemented in the custom indicator to take out the large spikes in either positive or negative directions during operation due to transducer noise and transients, introducing a time delay of up to 0.4 s. Filtering of raw data in commercially available indicators is unknown, as that information is not publicly available. The AOA indicator display, shown in Figure 10 was designed in Vista2D by Wetzel Technology, version 3.3.20. The software allowed the user to draw shapes, then

18 assign them locations such that another software, such as Matlab which is used for the EFCS, can take or send information to them so that real time data can be observed.

The AOA indicator was designed to change the AOA ranges for the display with the aircraft configuration, as shown in Figure 10. This figure also has the buttons necessary for operation of the EFCS highlighted, which will be discussed in the development of the tracking task subsection.

Figure 10: AOA Indicator/EFCS Buttons

2.3 Aural Cues Another concept that was investigated was the use of different aural alerts. “AOA yellow” and “AOA red” were used originally, then “push” or “push push” were added to not only tell the pilot what state the aircraft was in, but what action needed to be taken. Also, worth exploring was the use of a different voice, rather than just a computer generated one. A human voice was recorded in a distressed tone with the intent to grab the attention of the pilot. This broke the monotony of the same computer-generated voice callouts in the aircraft. The idea is that the different tone and pitch will alert the pilot to 19 a change that is happening and needs to be acted upon. A similar tactic has been used in military aircraft in the past. The appeal of using “stall stall” is that there is no question as to what it is about, as there could be with just another horn or noise. Two different methods were evaluated for the AOA warning/limiting system, which were the stall matrix and the tracking tasks.

Aural cues also used the AOA normalization scale found in Table 2. The airplane itself has a stall warning. In the course of the presented research project, additional cues were added to investigate their effectiveness. Once normalized AOA entered the yellow range at AOAnormalized=0.7 the system called out “AOA yellow” then 1.65 s later “Push” was announced. When AOA reached the red range at AOAnormalized =0.8, “AOA red” was called out, then after 1.65 s, “Push” was called out followed by another “Push” 2.2 s after the first. The timing results in one fluid sentence, i.e. “AOA red, push, push”. A red range callout always overruled a yellow range callout and would interrupt it. If the pilot remained in the red or yellow range for more than 6 s, the callouts would repeat. If transitioning from red range to yellow range, the yellow callouts will only begin after 6 s, not immediately. This was done because it is assumed that if moving from the red range with high AOA values, to lower AOA values, that the intent is to get into the green range with the yellow as just a transition. If it were to callout immediately, it would cause unwanted nuisance alerts.

2.4 The Active Stick An active control stick made by Wittenstein Aerospace & Simulation was used in this project. The available travel range for the stick is ±17.8° for the pitch axis and ±18.4° for the roll axis. The stick can be trimmed to a certain point, which for this project was set at the 0° deflection for the aircraft’s elevator and rudder. The stick has the ability to change the master force curve for both the roll and pitch axes and can also have soft stops implemented. Soft stops are a step increase or decrease in the force, which in this project were used to change the stick forces for the different AOA ranges. A soft stop

20 can be pulled through by the pilot by adding more pull force with their hand on the stick, whereas a hard stop cannot be pulled through with any amount of additional force. The value of the master curve for both pitch and roll was 1N/degree [0.4 slug/rad]. There is a multiplication factor f that is used to combine with the force master curve, which changes the force per deflection value. Equation 1Error! Reference source not found. shows how f was calculated for implementation into the generated force of the active stick. The parameters in the active stick can be adapted to fit the use case and user requirements by changing variables within the multiplication factor, f.

푞̅ 푓 = (푐 + 푘 ⋅ ) (1 + 푘 ⋅ |푛 |) 0 푞 1000푁/푚2 푛푧 푧

Equation 1:Active Stick Control Force Multiplication Factor

The variables co, kq and knz were the values selected in both the pitch and roll axis, and their values are shown in Table 3. The dynamic pressure, q̅, is a variable that is taken real time from the EFCS and is gathered from the nose boom. The normal force, nz is also gathered by the EFCS in real time.

Table 3: Active Stick Multiplication Factor Variables

co_pitch 7 co_roll 1

kq_pitch 2 kq_roll 2

knz_pitch 0 knz_roll 0

Figure 11 shows an example of the master curve for the active stick showing both the soft stops and breakout. The breakout is the force required to get out of the neutral position, which again can be seen in the figure below. Soft stops were used in the yellow and red AOA ranges at a value of 20 N [4.5 lbf] each. Therefore, upon entering the yellow range, the feedback force on the stick would increase abruptly by

21

20 N [4.5 lbf]. Upon entering the red range, another step change of 20 N [4.5 lbf]. would be applied.

Figure 11: Active stick force curve example (Koschlik [17])

The haptic feedback cues of the active stick were triggered at different points in the ranges of AOA. These items are tabulated in Table 4 and were a force step in yellow, a force step in red, a stick shaker in red, and a stick pusher in red. The pusher began at a normalized AOA value of 0.9 when it remained at or above that value for 0.2 s or greater. This was done to attempt to eliminate nuisance activations. The stick shaker began at AOA value of 0.8, red force step also began at this level, but it stayed active until the value dropped below 0.75. The yellow force step began at 0.7 and came off at less than 0.65. Each of the force steps was an increase of 20 N [4.5 lbf] and the stick pusher was an increase of 140 N [31.5 lbf]. There is the possibility of a secondary stall occurring, if the pusher comes offline and the pilot keeps pulling back on the stick, but this was not tested.

22

There was also a force step added into the roll axis for anytime the roll angle became greater than 45°. Once 60° of roll was reached, the force, which had linearly increased up to that point, levels off. When returning and lowering the roll angle, it is a linear descent all the way to 30° where the force step deactivated entirely. This was done so that when taking the roll out, the pilot did not experience the same step and overcorrect the roll angle.

Table 4: Haptic Feedback Cues

Start Stop

Yellow Force Step αnorm≥0.7 αnorm<0.65

Red Force Step αnorm≥0.8 αnorm<0.75

Stick Shaker αnorm≥0.8 αnorm≥0.9

αnorm<0.8

Stick Pusher αnorm≥0.9 (0.2 s) αnorm<0.65

The active stick also provides Q-feel which scaled force with dynamic pressure. It is named after dynamic pressure which is denoted by the letter q and was used during this project to adjust the stiffness of the controls with the dynamic pressure and hence flight speed. During the initial implementation of the active stick, it was found that the q-feel felt too high at faster velocities, while it was not sufficient at lower velocities. For this reason, the multiplication factor of f was altered to adjust the q-feel.

The q-feel at low velocities was found to be insufficient, needing more force in the pitch axis. At high velocities it was too high and needed to be lowered. The q-feel in roll was also found to be approximately 10% too high, so it also needed to be altered. The pilot feedback on the steps in the force bias was that it was insufficient. The pilot stated he could feel the force bias but did not think it to be useful, since it acted almost like a mini-pusher, yet it needed to be pulled through anyway to continue to a higher 23

AOA. It was kept because the purpose of the force bias was for the pilot to notice the undesirable change in the stick. The thought process would go on to say that if the force bias was noticed, a pilot could make corrective actions prior to any LOC event.

The stick was bench tested using different forces to find appropriate force levels. Upon installing it into the aircraft, it was found that with a pilot resisting at higher force levels it would bend the floor of the aircraft slightly upward. This did not affect the structure of the aircraft, but nonetheless, the forces were adjusted so that this did not happen anymore.

When initially calibrating the stick for flight, the forces were set to what the DA test pilot thought was an adequate amount to alert the pilot of a change in the range of AOA. However, once airborne, the pilot noticed that the forces became too weak. This could have been due to the aerodynamic forces playing on the airplane or just the perception of these forces once the attentional workload was increased in the flight environment. Studies have shown this phenomenon of attentional load discussed in Section 1 of this document.

In flight, the trigger button on the back of the stick was for beginning the tracking task, while the coolie-hat could be used to select between different variations of the tracking task. The most important button on the stick was the one on the top right, which switched between autopilot and direct law. When activating the fly-by-wire system, it would start in autopilot altitude hold mode, then returns back to altitude hold mode after being deactivated. Shown in Figure 12 below is the active stick installed into the co- pilot station of OE-FSD.

24

Figure 12: Active Stick Installed

The FTE sitting in the back seat had the ability to change different parameters during flight such as turning on the shaker, or pusher at varying levels. This allowed for pre-test campaign test flights to be conducted and optimize the system. Below in Figure 13 is the FTE station inside the test aircraft. The Microsoft® Surface tablet mounted on the back of the pilot’s seat was used as the interface with the EFCS including the active stick controls. Also shown in this figure is an early version of the AOA indicator.

25

Figure 13: FTE Station

2.4.1 Stick Shaker The stick’s force pulse feature was used to simulate a stick shaker with set amplitude and frequency. Since motion of the active stick is transmitted to the control surfaces, they moved slightly during activation of the shaker. This was taken into consideration when deciding upon the amplitude and frequency values so that they would not have an adverse effect upon the aircraft dynamics.

2.4.2 Stick Pusher The stick pusher was implemented by using a soft stop adding sufficient force to the stick that the control surfaces of the aircraft pushed the nose down before the aircraft stalled. Stick pushers can be used as an essential part of safe flight to prevent stalls. Once they are activated, that is one of the definitions for stall per the FAA Advisory Circular AC 120-109A [18]. When a stick pusher is installed that is essential to safety of flight, defining the stall, it must be checked for proper operation in a preflight check per FAA Advisory Circular AC 23-8C [15].

The biggest challenge with a stick pusher is that not only must it fire at the proper time and with the right amount of force, but it also must take into account altitude. The worst-case scenario would be if flying in the traffic pattern and coming in to land and the system believes that the airplane is stalling, when in reality the pilot is only flaring 26 the aircraft to land. In this case, it could potentially cause an accident rather than prevent one.

2.5 Development of Tracking Task When developing the tracking task for this project, there were many unknowns such as what data are needed to be gathered from this task to make it work and to get something useful out of it. The appeal of a tracking task is that it should be something repeatable by any pilot and would obtain specific values and rates, which is desired when certifying a new system. It could drive costs down, if there is only one test that needs to be completed. This repeatability of obtaining values and rates was the original goal for this thesis, however, as discussed later it was found that the developed tracking task was a better suited tool for increasing pilot workload.

In a project done by Klyde, et. al. [19] an evaluation task was created in order to quantify the handling qualities of a system designed for LOC mitigation. Specifically, it looked at pilot induced oscillations (PIO) that can lead to LOC by means of an adaptive controller. Figure 14 shows the command signals designed for Klyde’s project. Both plots in the figure show varying pitch and roll commands which allow the pilot to become focused on the task and gives the potential for PIO’s and high AOA values. For this reason, a similar approach was taken for this project.

Figure 14: Pitch and roll axis sum-of-sines command signals [19]

27

Figure 15 shows two different versions of the tracking task as carried out for this project. In both figures the top chart is showing the desired AOA value. An earlier version of these tracking tasks had pitch and roll, but since this project was investigating the use of AOA, and the aircraft had the ability to easily display it, the values were changed from pitch to AOA. Version 1 was based on the task specified in the test plan found in Appendix C, but with added bank maneuvers, while version 2 added roll maneuvers both to the left and to the right as well as stalls during the rolls in order to evaluate the wingtip mounted probe, plus flap and gear configuration changes during the task in order to simulate a realistic traffic pattern at a safe altitude.

Figure 15: Versions 1 and 2 of AOA Tracking Task

In order to get the full effect of selective attention or to be fully immersed in the task in flight, one of the evaluation pilots did not practice either version of the tracking task on the ground in the aircraft in the loop simulation.

The integration of the tracking task into the graphical user interface (GUI) was originally designed such that the pre-programmed tracking task was a moving bar to 28 follow, or track, and the aircraft itself was also a moving bar that would line up with the tracking task bar when the pilot was perfectly following the task in both bank and AOA. After obtaining pilot feedback on the system, it was changed to having the airplane bar remain stationary. At this point, the difference between the two bars was a delta of error between the two points. Figure 16 shows one of the more mature iterations of the tracking task on the multi-function display. The display is shown on the second primary flight display (PFD2) in the cockpit of OE-FSD.

The tracking task was also designed in Vista2D. For example, in the tracking task there was the course bar, which had a preset track and would continue to move the same way it was programmed regardless of any inputs from the pilot. The airplane bar was set to receive input from the EFCS in order to accurately represent in real time the attitude of the airplane.

Figure 16: Tracking Task in Cockpit

29

The tracking task was operated via the evaluation pilot using the trigger button on the active stick. There were different versions of the tracking task loaded into the software that could be selected in flight. All the evaluation pilot needed to do was use the coolie-hat, or trim button, along with the trigger button to scroll through the options and select the desired test number. This can also be done by the FTE in the back seat. Both were utilized in the flight tests. The activation of the tracking task, via the active stick, could only be carried out if the EFCS was active. To do this, as seen in Figure 10, the square white button had to be pressed and the red knob pulled before the evaluation pilot could activate the active stick via the fire button. Once the system was active, then the evaluation pilot could go and begin the tracking task of their choice.

Not all aircraft have an EFCS such as the test aircraft. To be able to implement it in a tracking task format would involve added cost for either the applicant or the authority. In the case of the authority, it could potentially be a portable solution such as a tablet computer with enough capability to determine the tracking task is being followed. This tablet could be mounted like a heads up display similar to what is seen in Figure 10. There are many AOA indicators already in production that have vastly different display systems. It could be seen by them that if the authority did not choose their display method, then it would be considered wrong.

2.4 Creating the Test plan The test plan was created with intentionally broad goals to allow for flexibility during the test campaign itself. All tests were deemed to be low risk as the aircraft would stay within the normal operating envelope, and safety pilots from the airplane manufacturer would be onboard. The original stall test matrix was very large and was hence cut down to provide a realistically achievable number of test points. In certification of aircraft and their stall characteristics, the number of points can be up in the hundreds to prove safe recovery. Appendix D shows the stall test matrix that was

30 created for this project. It is comprised of 32 points with varying bank angles, bleed rates, power settings, and gear/flap configurations.

Initially FTE’s from Florida Institute of Technology (FIT) and the FAA would act as essential crew during the test flights, but upon witnessing the operational complexity of the experimental OE-FSD systems, it was determined that the project engineer who developed the system would act as the FTE on all flights. This would also allow for smooth transitions between test points and the possibility to update or change certain items, such as some of the active stick parameters in flight.

Prior to the whole team arriving in Wiener Neustadt, a few pre-test campaign flights were completed to test the active stick installation and determining the actual stall speeds and AOA values for different configurations, this is shown in Table 2. This was also done to check all updates made to the EFCS and see if there were any software errors in it before the actual testing began.

Figure 17 shows the test area’s airspace where the flights occurred. It is noted that there was another airfield 1.6 nautical miles away, which limited airspace use and caused the pilots to look out for more traffic, sometimes including gliders who did not have traffic collision avoidance systems.

Figure 17: Test Area [20]

31

There were a total of 11 flights completed during the test campaign, all in VFR conditions. Table 5 shows times of each of the flights and what the objective of the flight was. Total flight time for the project was 12 hours and 31 minutes. All six test plan objectives were completed which were:

1. Audio AOA-Warning Method of Compliance 2. Visual AOA-Warning Method of Compliance 3. Active Stick 4. Legacy Tracking Task 5. AOA Tracking Task 6. Operational Evaluation.

Table 5: Flight Log

Flight Test Campaign 07/03-07/11 No. Date Takeoff Landing Dur. Objectives 1 20170703 13:36 14:48 1:12 Straight stalls 2 20170704 8:34 9:44 1:10 Banked stalls 3 20170704 13:40 14:40 1:00 Legacy Task & Free Pattern 4 20170705 14:37 15:38 1:01 AOA Tracking Task 1 5 20170706 14:32 15:10 0:38 AOA Tracking Task 2 6 20170707 7:05 7:37 0:32 Straight Stalls & Tracking Task 1 7 20170707 7:48 8:20 0:32 Straight Stalls & Tracking Task 1 8 20170710 8:58 10:52 1:54 Matrix & Tracking Task 1 9 20170710 12:58 14:23 1:25 Matrix & Tracking Task 1 Autoland & Operational Pattern & 10 20170711 8:03 10:01 1:58 Cue Combinations Operational Pattern & Cue 11 20170711 13:04 14:13 1:09 Combinations Total Flight Time 12:31

A set of test cards created for the first flight that took place can be found in Appendix E. The cards include not only the test points to fly, but also score sheets at the end to be used for rating the systems that were evaluated.

32

In summary, pilots did not like the negative angle on the AOA indicator as it looked as though they were getting negative AOA values. The original evaluations of the pusher and shaker in flight found that they were not strong enough and the pilots requested that they both had more power added to them. The initial investigation of the AOA indicator proved that it functioned well, but a little bit too well as it fluctuated rapidly because it was reading instantaneous values. To fix this, a filter was added to the data going to the indicator such that it did not show instantaneous changes from things such as wind gusts.

33

Section 3

Results

The results obtained from the analysis of the data are a combination of numerical data and pilot opinion. Both are valuable outcomes and can be used in future exploration of the test methods used. The stall matrix points were found to be the most effective way to test the aural, visual, and haptic feedback of the designed systems.

AOA tracking tasks were found to be insufficient for a method of compliance to test NORSEE equipment such as an AOA indicator. Specific test areas included: evaluation of a test matrix designed to test functionality of an AOA-warning system and an AOA-limiting system, evaluation of a concept for haptic feedback, and evaluation of a tracking task designed to test functionality of an AOA-limiting system.

3.1 Data Analysis All the values collected from the flights were recorded into “.mat” files to be analyzed in Mathworks Matlab. All the parameters recorded were on the same fundamental time; however, since the Flight Test Instrumentation (FTI) is the first system to be activated and the rest of the systems become activated sequentially afterwards, they all have different start times. This initially caused issues with analyzing the data when trying to match up data points with time hacks from the cockpit videos. For example, on one of the flights the gear marker data suggests that the gear is in the retracted position for the first 1000 s of flight, which we know to be false because the gear was needed to taxi and take off.

Another challenge in analyzing the data was determining the time stamps of certain maneuvers or phases of flight to match up the data with the video clips since the video clips themselves were not time marked with the same fundamental time as the 34 recorded data. The audio portion of most cockpit video recordings was lost for indeterminable reasons.

3.2 Results The traditional stall matrix had points to cover 4 bank angles, 2 airspeed bleed rates, 1 altitude, 2 power settings, and 2 gear/flap settings for a total of 32 combinations. Appendix D shows these test points. Each point could be flown to stall. Set-up for each point started at 1.2 to 1.4 times the published stall speed of the aircraft. One important note of the stall matrix is that the pilot knows when the stall will occur as they are purposefully going into that condition.

The stall matrix points were used to investigate different AOA indications and warnings in varying combinations. At some points, the evaluation pilot was asked to ignore certain warnings. For example, when it was desired to bring the AOA all the way up into the red range, the pilot was told to ignore the yellow warnings and only react to the red. This was done incrementally to not overload the pilot. First the visual indicator was examined, then the aural alerts were added on, and finally the haptic feedback of force steps, shaker, and pusher were added in. Some of the downsides to using a stall test matrix such as this are that it cannot be guaranteed a specific AOA value or AOA rate will be achieved, and the outcome of the test point depends upon proper entry into the test point. Future AOA or stall protection systems may be specified to withstand a certain AOA rate. Thus, a method of compliance is needed to verify system performance.

As previously stated, a tracking task was desired due to the potential for repeatability and ability to address the variability of the entry to stall matrix points. This could also ensure that specific values of AOA and AOA rate were obtained. As described in the previous section, the AOA tracking task was carried out as planned but had some unexpected outcomes. The following findings originated from the post flight briefings for these flights. From version 1 of the tracking task, it was found to work reasonably well, but more time was needed to recover after the Red AOA maneuver to 35 allow the pilot to keep up with the task and also to prevent secondary stalls. When performing rolls to the right or left, the wingtip vane probe AOA clearly lagged behind that of the nose probe, which was much more consistent in AOA. Therefore, it was decided to use the nose probe for collecting data during any rolling maneuvers. This brings up the question of what to do with single engine airplanes that cannot mount a nose probe since that is where the propeller is located. Experimentally, a wing or nose probe is preferred, but in operation the airplane is more likely to have something similar to a pitot probe that is installed on the wing or fuselage to gather data for an AOA system.

Many GA planes have the pitot probe mounted on the left-hand side, along with the lift tab that triggers the stall warning horn. For this reason, an experimental or off the shelf pitot system for an AOA indicator would likely also be mounted on the left- hand side since that is the standard practice for many GA planes.

The task did surprise the evaluation pilot at times, which was a good outcome of a tracking task as it is a fairly accurate measure of how a pilot would react to an unexpected change in aircraft attitude even when the pilot inputs the commands themselves. After the initial flights with version 1 of the tracking task, it was decided that a configuration change should be added into the task. This would represent a more real-world situation that could cause a pilot to encounter a LOC situation, as many LOC accidents occur in or around the traffic pattern when airplanes are low and slow.

The first version of the tracking task led to the intended stalls in turns, which was one of the objectives: to get the airplane into all the AOA regions in different attitudes of flight. Due to airspace restrictions, completing the left and right hand turns for the tracking task left an overall narrow use of airspace. This, among many others, is one of the reasons why the safety pilots on board all test flights were Diamond Aircraft employees who regularly fly in this airspace and can observe before any airspaces are potentially encroached upon and make corrective action or “knock off” the test point as a last option.

36

The altitude band used during this first version of the tracking task was between 500 ft. [152.4 m] and 1,000 ft. [304.8 m]. An initial entry speeds of 100 knots indicated (KIAS) was decided upon and lead to a minimum speed of 54 KIAS at about 0.7G. It is noted that the “clean” stall speed was approximately 64 KIAS and the “dirty” stall speed with flaps and gear deployed was approximately 59-60 KIAS. Using approximately 1.7 times the stall speed, 110 KIAS was used for version 2 of the tracking task.

Version 2 of the tracking task incorporated configuration changes and slightly different roll and AOA values in the tracking task as shown in Figure 15. This version of the tracking task also incorporated feedback from the first version. The team and test aircraft allowed for a flexible test plan structure that included time for making changes, and it took a day at most to implement them.

It was found that the planned and attempted configuration changes were not possible during the task as they resulted in an AOA profile that was too steep in pitch angles and too high in airspeed for the pilot to keep up with to remain in a safe attitude in the airplane. The landing configuration resulted in too steep of pitch and secondary stalls during recovery. This was not anticipated, but due to planning of the test and training of the pilots, it was easily and safely recovered. Very high pitch up angles resulted during the AOA buildup, which was not intended to be an outcome of attempting to get into varying AOA values, but can be expected for the higher values. Overall the task was hard to track. If it was adjusted to have a higher baseline AOA to start the task, such as 0.5 instead of 0.3, then this could prevent high speeds and steep pitch angles. The wingtip vane probe again produced unsteady AOA data, as it did similarly in version 1 of the tracking task.

Both AOA tracking tasks, version 1 and 2, could have led to potentially dangerous power on stalls that could approach the airspeed limits of the aircraft for certain flap and gear configurations. It was also noted that in the flaps for landing setting, the pitch was too steep down and led to secondary stalls. It was concluded that an AOA tracking task should not include configuration changes and that only straight stalls should be used. However, there could be a different customized tracking task for 37 each configuration that could work. Determining customized tracking was outside the scope of this project. From this point, version 1 of the tracking task was used, plus a stall matrix to determine the effectiveness of the warning system. After the first test flight, the power-on test cases of 3-5 KIAS per second bleed rate were deemed to be too aggressive since they put the airplane into less than 1G conditions and were dropped for the remainder of the campaign.

Recall from Section 1, Bailey [13] attempted creating a “challenging, repeatable, flight-tracking task” in order to measure pilot performance quantitatively and found it was not sufficient due to the numerous variables. Similarly, from the testing done on the DA42, it was determined that a tracking task was not an effective way to evaluate individual systems or a combination of systems, since there are so many variables to consider. For the purposes of this research, it is unknown whether the task itself was at fault and needed a redesign or if a tracking task in general leaves little room for unknown variables such as wind gusts.

3.2.1 Qualitative Results The final version of the AOA indicator was found to be well designed by the evaluation pilots. One said: “The visual angle of attack display on the DA-42 was one of the best I have seen. Not only did it let you know what your relative angle of attack was, it also provided flap configuration. I felt that it made much more sense than any of the after-market angle of attack indicators that I have seen so far. Some of those displays border on being stupid in addition to being confusing. However, I believe all visual systems suffer from the same problem. Pilots who lose control are not looking at them which includes the airspeed indicator.” This statement was reflected in feedback from evaluation pilots as the test campaign progressed. On each of the evaluation pilots first flights they reported monitoring the indicator on all test points, but as other alerts were added in and they were evaluating aural and haptic feedback as well, they reported monitoring the visual indicator much less often. In the case of the tracking task, some

38 evaluation pilots reported not monitoring the AOA indicator at all because their attention was engrossed in the task.

As reported by the project pilots, the test aircraft was “talkative” and many found it to be distracting. One of the evaluation pilots began to ignore the aural alerts after a prolonged period because there was so much chatter going on and he was just trying to fly the airplane. The aural alerts are another factor that could use some further investigation. An evaluation pilot stated: “With exception of the “Stall - Stall” warning, the aural warnings were lost in the background plethora of warnings in today’s modern cockpit. The Garmin Systems warnings for landing gear and other warnings soon caused one to ignore the aural channel.” This led to an important finding that the subtle nuance and intonation of the voice makes a difference. Even if the aural alert occurs at the proper time, that may not be enough. While taking the timing, phrase and tone into consideration, these three variables could possibly be changed with varying AOA levels as future work. Regarding the aural alerts one of the evaluation pilots stated that “the shaker is the last warning, pusher is the protection. All the rest we don’t need”.

Regarding the pilot comments on the haptic feedback, it was found that for all pilots the stick shaker was the most effective warning method tested. In a survey given to all pilots, it was ranked as the highest in terms of points. They believed it should be awarded the highest point value in a new certification system, where certain safety items are worth certain amounts of points.

The stick shaker was found to be very effective by all evaluation pilots and even the safety pilots took note of the potential benefits of this system. “After about 5 hours of doing approaches to stalls, I found the Stick Shaker to be so effective that I would assign it a very high points value. First, it cannot be ignored. Secondly, it should be relatively easy to install in the GA fleet, including older airplanes” said another of the evaluation pilots. They found it to be more effective than others in the field they had previously encountered. The shaker is in reality, just a more compelling stall warning. This is the reason why engaging important alerts or warnings across different inputs is

39 important. Rather than just a visual, or an aural, the haptic will always let the pilot know something is not right.

The pusher was also found to work well, although it would need some minor adjustments on the force levels and the activation times. A problem with pushers is how the system will be able to determine that it is at a safe altitude to fire. As seen in the data from actual landings, for some pilots the AOA value goes into the red during the approach for landing. If the pusher were to fire this close to the ground, it could be dangerous and potentially fatal. This is the reasoning behind the “do no harm” statement, because a pilot could unintentionally add incorrectly designed safety equipment that ends up being detrimental. The presence of a system does not necessarily add safety. The complexity and cost of safely implementing a stick pusher would limit it to higher end GA planes, as the associated cost is too much for many GA airplane owners.

The force steps on the active stick had mixed feedback. When the pilots were performing operational tasks or in the tracking task, the yellow step was often not recognized. More research would be necessary to determine the proper way to implement the force steps, or if they are even needed. One of the evaluation pilots stated that “Experienced pilots may consider this feature a nuisance; however, for inexperienced pilots it may be valuable.”

3.2.2 Quantitative Results Normalized AOA It was desired to see the matchup between actual AOA and normalized AOA in degrees to ensure that the assumption of the normalized AOA being the same as, if not nearly equivalent to that of the actual AOA was accurate. This is important because if the model for normalized AOA was inaccurate, then some of the findings might be as well since normalized AOA was used throughout the project.

40

Seen below in Figure 18 is a comparison of actual AOA and normalized AOA. This is from the July 3rd test and the normalized AOA value is shown in red, while the actual is shown in blue. There are markers showing when each of the feedback cues are fired and different AOA levels. Shown are AOA yellow, the aural warning for AOA yellow, the shaker and AOA red aural warning, and finally the activation of the pusher. Clearly, the assumption holds of normalized AOA being a good substitute for actual AOA.

Figure 18: AOA change with Time

As there were two main evaluation pilots, it was natural to compare the two sets of data from the very beginning to see if the data sets correlated. The figures below show a comparison of Pilot A and Pilot B performing stalls from the stall test matrix. This was done in order to visualize the role that pilot technique played on AOA. It was desired to gather data on and compare different flight test methods, so investigating if there were any large differences between pilots due to technique was necessary. The

41 difference in pilot technique was found to be negligible for the purposes of quantitatively evaluating data. Figure 19 depicts Pilot A performing stalls from the stall matrix test points. It is taken over a 1000 s time period and displays actual AOA, normalized AOA, flap setting, and power setting for both the left and right-hand engines.

Figure 19: Pilot A Straight Stalls

Figure 20 shows Pilot B performing stalls from the stall matrix test points. It is taken over a period of 4000 s and like Figure 19 shows actual AOA, normalized AOA, flap setting, and power settings for each engine. It is noticed in both Figure 19 and Figure 20 that the power and flap settings move relatively along with the AOA values. This is to be expected, because as the pilot decreased the power, extended the flaps and pulled the plane into a stall, the AOA values rose until they reached a critical AOA value, and the airplane stalled. There are points where the normalized and actual AOA are not identical. This is noticed to have the largest difference during the same

42 approximate time as the flap setting is altered. It is recalled that the normalized AOA value changes based on the flap setting.

Figure 20: July 10 Flight 1 Pilot B Stall Matrix

Tracking Task As previously discussed, the second version of the AOA Tracking Task was discarded due to it putting the aircraft into unsafe conditions. This discarding of the task was done in flight when it was determined to be unsafe, and therefore no reasonable data can be drawn from that specific tracking task. However, the first version worked well enough that the following data could be analyzed from it. Shown in the figures below are the actual tracking task commands that the pilots were following and then the values inputted by the pilot during the tracking task in Figure 21. Recall the first version of the tracking task shown in Figure 15. The values for AOA and roll in degrees are very difficult to tell if they match up. This lines up with the pilot feedback that the tracking tasks were a poor way to measure anything other than pilot response time. However, it can be seen that there are three distinct peaks in the

43 pitch angle data presented in Figure 21 found at approximately 1900 s, 1920 s, and 1950 s. These are the pilot’s inputs to the three peaks of AOA seen in Figure 15. Clearly the normalized AOA that was gathered does not match that of the tracking task, while the pitch values more closely do. There are many variables that makeup AOA and all were not taken into account during this test campaign due to complexity and scope.

Figure 21: Tracking Task Version 1 July 5

From pilot feedback and data obtained and displayed in Figure 21 it is determined that a tracking task such as the ones designed and tested for this research are not sufficient methods of compliance. There is potential to use some form of a tracking task if it is specifically designed for a specific aircraft, however, this is not recommended as there are simpler methods to obtain the same data.

Simulated Patterns Traffic patterns were simulated at altitude to ensure safety of the occupants and the aircraft since the active stick had very few flight hours and did not yet have a system 44 installed to prevent the pusher from firing at an unsafe altitude. Since the recording system was on for the whole flight, actual data from takeoff, approach to landing, and landing was obtained. Seen below are the simulated patterns followed by actual pattern values. As discussed earlier the wingtip probe data had errors when not in straight and level flight, due to this, the nose boom data was used for not only AOA but also AOS. Figure 22 shows the values for actual AOA, normalized AOA, flap setting, pitch of aircraft and active stick pitch positions. Clearly there are four main segments to this plot. They will be individually analyzed further down. The last one which is highlighted in Figure 30 is the abused case of the simulated pattern when the pilot purposefully used the rudder incorrectly on the aircraft, termed “rudder-ing around”.

Figure 22:Overall Simulated Traffic Pattern

Seen in Figure 23 are the values for AOS and the gear indication. When the gear is retracted, the value reads 0, when it is extended the value reads 1. The AOS is of interest here as this was an “abused case” of a simulated pattern where the evaluation pilot intentionally input exaggerated rudder commands. 45

Figure 23: Overall Simulated Traffic Pattern

Figure 24 shows the first simulated pattern from this flight that was originally shown in Figure 22 and Figure 23. The flap setting portion which is found in the middle of the figure shows when and at what level the flaps are set. A value of 0 denotes having the flaps retracted, 20 is the flaps in the approach position, and 45 is flaps in the landing position. It is also noted that the normalized and actual AOA values do not perfectly line up in this example, but that they follow the same trend. The normalized AOA value changes from approximately 0.4 to 0.6 when the flaps are changed from the approach to the landing configuration. This is something seen in many GA airplanes with the addition of flaps, which causes some concern. As discussed in Section 1, the majority of LOC accidents occur in the traffic pattern and when coupled with other factors such as lack of situational awareness this AOA change brings the airplane closer to the critical AOA.

46

Figure 24:Pattern 1 These figures show a portion of interest of the overall flight as it was desired to evaluate the patterns individually as well as overall.

47

Figure 25: Pattern 1 Figure 26 shows the second pattern in this series. The normalized and actual AOA values line up much better in this pattern. The flap setting is seen to have a direct impact on the AOA values throughout the simulated pattern.

48

Figure 26: Pattern 2

Figure 27: Pattern 2

49

The third pattern from the test flight, shown in Figure 28 again shows the same parameters as the previous sections. It is a good visual of the difference in the pitch angle of the airplane and the pitch position of the active stick. At approximately 1510 s, the pitch angle of the airplane begins to increase while the pitch position of the active stick is decreasing. At the same time, the AOS is decreasing.

Figure 28: Pattern 3

50

Figure 29: Pattern 3

Figure 30 shows the most abused case of the pattern. Clearly this figure has the most abrupt changes in AOA as can be seen at approximately 1,840 s and 1,870 s. At these points the normalized AOA value reached 1. Recall that the pusher fires after having a value of 0.9 or higher for more than 0.2 s. This can be seen in the figure and it is also recognized in the third subplot that displays the pitch angle, theta.

51

Figure 30: Pattern 4 Just prior to the first large spike in AOA and pitch angle, the AOS decreases by -6°. It is noted that this is occurring just after the gear is extended and the flaps are set into the approach configuration. This is the scenario when an airplane could get into a LOC accident due to the rapid changes in AOA. Over a period of approximately 25 s, the airplane has a good energy state and it slowly achieving higher AOA values, and then it deteriorates in approximately 5 s. If the pilot does not properly react to this rapid deterioration of energy state it could lead to a fatal LOC.

52

Figure 31: Pattern 4

Landing Data Actual landing data was also acquired during the test campaign; therefore, it was important to analyze it and compare it with the simulated patterns to see if similar values were being obtained for AOA. It is noted that the values for AOA depend not only upon the environment the aircraft is moving through at that moment, but also pilot technique. Three different safety pilots flew. All safety pilots completed actual landings and between them, some had the stall warning horn going off on landing and others never did. This translates to having some of the pilots fly into the red AOA range while others stayed within the yellow or green ranges. It is important to note that in the data shown for the actual landings, the pilots were flying airspeed and not AOA. Figure 32 shows data gathered from an actual approach and landing. It has AOA, normalized AOA, flap setting, stick position and pitch angle, and altitude. Figure 33 depicts the AOS, gear indication, and airspeed. Clearly the normalized AOA reaches a value of 1 which is well into the red range for AOA. If the pusher had been online, it

53 would have activated. At this point the airplane was at approximately 450m [1476.4 ft.] pressure altitude.

Figure 32: Landing Data July 4 Flight 2 Evaluating the actual landing data are important because it proves that the assumption of the simulated pattern was an accurate one with respect to normalized AOA values and whether or not the haptic feedback would activate.

54

Figure 33: Landing Data July 4 Flight 2

As previously mentioned, pilot technique plays a large role in the AOA values on landing. In this example from July 4 Flight 2, Figure 32 and Figure 33, it is seen that the normalized AOA value reaches 1 which is in the red AOA range meaning that the aural alert would have already gone off, as well as the pusher which is fired at a normalized AOA value of 0.9. This is cause for concern because if the pusher were to fire while on the flare for landing it could just mean a hard landing, however, if this occurred on the approach to landing, it could cause the pilot to lose control of the aircraft and not have time or altitude to recover.

Stick Pusher The stick pusher was evaluated for different parameters such as starting altitude, ending altitude, actual AOA, the force input by the pilot and the pitch angles during the firing of the pusher. A parameter of particular interest is the time and altitude it took to recover immediately following the pusher’s activation. These are critical factors in recovering from an unwanted flight attitude. 55

Figure 34 shows the firing of the pusher, as seen in the 5th subplot of the figure, as well as a few other parameters of interest. The recovery took approximately 30 s and 100 m [328 ft.] of altitude. These are acceptable levels when at higher altitudes, but are cause for concern if the pilot is flying at a standard traffic pattern altitude or lower because this is approximately 1/3 of the overall altitude. And as was seen before, in actual landing data the pilot can get into the red AOA range which would cause the pusher to fire if it were online. Refinements to the pusher were desired but outside the scope of the project. The altitude loss was to be minimized and the G force value dropping below 1G was to also be minimized. It is seen that the down component of velocity drops to almost -20m/s [65.6 ft/s] directly following the firing of the pusher.

Figure 34: July 7 Flight 2 Stick Pusher

56

Pilot Survey Upon completion of the test campaign, all pilots who participated were given a survey which asked them to suggest point values from 0-100, for stall warning implementation methods. They were also asked to suggest point values from 0-40 for Enhanced Indication System Options.

The blank survey form along with data can be found in Appendix B. For the first portion of Stall Warning Implementation, all pilots found the yoke or stick vibration, also called a shaker, to be the most effective with an average of 81 and scores ranging from 60 to 100. The second highest factor was that of a synthetic voice with an average of 40 and scores ranging from 15 to 75. The lowest scoring item for Stall Warning Implementation was for a visual indication independent of pilot focus. This makes sense as in the LOC-I scenarios, the thought is that most pilots are not paying attention to their instruments in the first place and that is how they get into the situation where they are unaware of their energy state and lose control.

Figure 35 shows this graphically. It is noted that the first three warning techniques are aural alerts and have mid-ranged averages while the visual techniques have the lowest values overall. The pilot survey as well as the pilot feedback show that a haptic warning technique is the most effective and preferred by the evaluation pilots.

57

Figure 35: Stall Warning Implementation Pilot Survey

The core results found were that; the aural alerts were most effective when a phrase was used rather than a tone, the custom visual indicator was very intuitive and project pilots said it was the easiest to understand of the indicators they had seen, the haptic feedback was a good indication of something changing with the stick shaker receiving the highest ratings from pilots, a tracking task is not a sufficient means of test for a method of compliance, and the stall matrix set of points was the best way to test the aural, visual, and haptic feedback paths.

58

Section 4 Conclusions

4.1 Conclusions The purpose of this research was to explore flight-testing techniques on AOA warning combinations for Part 23 aircraft. The DA42 was an excellent test bed and allowed for rapid prototyping of versions of tracking task and parameters on the active stick, aural cues, or AOA indicator.

The matrix method of stalls was found to be the most effective in obtaining repeatable data to examine the various AOA indicating and limiting systems presented. The haptic feedback cues were by far the most effective means of alerting the pilot of an undesired condition, as they “cannot be ignored”. This is shown in pilot comments and in the data presented. In order of effectiveness and scoring from evaluation pilots, the stick shaker ranked highest, followed by the pusher, and finally the force steps in the various AOA ranges.

The data obtained showed that if not properly implemented, certain safety systems such as a stick pusher, could cause more harm than good and be detrimental to the safety of the pilot and aircraft if fired at the wrong time. It was shown that the ranges chosen for the normalized AOA would activate the pusher at unsafe altitudes on approach to landing, which could cause anywhere from a hard landing to an accident. Stick pushers have been safely implemented in commercial aviation for many years, but in GA it would be the most complex and costly solution to avoid LOC accidents. Implementing a stick shaker into GA aircraft would be less complex and costly.

Further research is required to ensure that the pusher is a safe system in all modes of flight. One potential for ensuring the pusher only fires when it is supposed to would be to tie its operation to the stall warning horn of the aircraft itself. This, however, does not address the firing at low altitudes problem. One potential for helping the low altitude 59 problem would be to tie its operation to an altimeter and once the altimeter detects an altitude of 500 ft. [152.4 m] or less, it would inhibit operation of the stick pusher. This value of course is only a suggestion based on results from this project and would need further research to ensure that is a safe altitude for all aircraft and pilots, not just OE- FSD and the evaluation pilots.

The visual feedback cues showed good results in qualitative feedback from the evaluation pilots, and the indication of flap setting was a desired trait. However, it was further shown that if a pilot has channelized attention, such as in the tracking tasks, they will not be looking at the AOA indicator mounted on the glare shield anyway, this lack of focus on cockpit instruments is one theory for what can cause a LOC accident. The aural feedback cues that had the state of the aircraft as well as what action was needed to fix that, such as “push” had good feedback from the pilots as well. Further research would be needed to determine these statements quantitatively though in the form of pilot reaction time.

It was determined that the originally designed tracking task was a poor way to evaluate an AOA indicating or limiting system as it produced undesired and sometimes unsafe results.

60

4.2 Future Work

The following items were outside the scope of this project and could be useful items to investigate from the data already collected or with minimal additional testing:

• From the existing data, the pilot reaction times should be investigated to see if using aural alerts with different phrases or tones changes those initial reaction times. Different phrases would be “AOA Yellow” which states what condition the aircraft is in, and “AOA Yellow. Push. Push” which also states what action needs to be taken by the pilot to prevent further energy state decay. Another reaction time of interest would be the reaction time to the stick shaker or pusher. • For future work using the same or similar test aircraft, it should be ensured that the video feeds share the same fundamental time with the data so that all test points and actions made in the cockpit can be precisely determined and analyzed with more accuracy. The audio on most of the videos was lost and in any future projects it should be ensured that the audio is preserved for later analysis of the flights. • If a tracking task is desired as a method of compliance, it would likely need to be either a customized task for each configuration, or perhaps something entirely different. It would be recommended that a variation of Version 1 of the Tracking Task is used along with a matrix style test series of stalls to obtain the most accurate data on the warning system or systems. • Determining how to distribute or evaluate a tracking task. Not all aircraft have an EFCS, so the implementation is another possibility for future work along with distribution. If a tracking task were to be developed that obtained desired results, how would representatives from the authority administer it? • Another line of future work would be to investigate the different combinations of visual, aural, and haptic feedback to see if there are any combinations that decrease safety rather than enhance it. Also, to determine if all the warnings are necessary or if having one negates the need for any others.

61

References: [1] “Fact Sheet – General Aviation Safety.” FAA, Federal Aviation Administration, 26 July 2017, www.faa.gov/news/fact_sheets/news_story.cfm?newsId=21274. [2] European Aviation Safety Administration. “GA LOC-I Fact Sheet”, Web. www.easa.europa.eu/easa-and-you/general-aviation/flying-safely/loss-of-control-in- approach-and-landing. [3] US Department of Transportation. Federal Aviation Administration. FAA Clears Path for Installation of Angle of Attack Indicators in Small Aircraft. Www.faa.gov. FAA, 5 Feb. 2014. Web. 11 Aug. 2017. https://www.faa.gov/news/press_releases/news_story.cfm?newsId=15714. [4] Federal Aviation Administration. Memo No: AIR100-14-110-PM01. 5 February 2014. Web. http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/eb0fac0c1641509 586257c76005e6274/$FILE/AIR100-14-110-PM01.pdf. [5] US Department of Transportation. Federal Aviation Administration. “Approval of Non-Required Safety Enhancing Equipment (NORSEE). Policy No: PS-AIR-21.8- 1602.” 31 March 2016. http://rgl.faa.gov/Regulatory_and_Guidance_Library/rgPolicy.nsf/0/1790b02f1833357 486257f9200592110/$FILE/PS-AIR-21.8-1602.pdf. [6] Federal Aviation Administration, “Electronic Code of Federal Regulations,” U.S. Government Publishing Office Federal Digital System [online], Title 14-Chapter 1- Subchapter C-Part 23-Subpart A-Section 23.3: Airplane Categories, http://ecfr.gov/ [retrieved 13 August 2017] [7] Glenn Research Center. “Inclination Effects on Lift.” NASA, NASA, www.grc.nasa.gov/www/k-12/airplane/incline.html. [8] Federal Aviation Administration (2012). Advisory Circular 120-109, Stall and Stick Pusher Training. Retrieved from https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC%20120-109.pdf [9] Pitts, Matthew J., et al. Assessing Subjective Response to Haptic Feedback in Automotive Touchscreens. AutomotiveUI, Essen, Germany, September 2009, Conference Paper. University of Warwick Coventry, United Kingdom, 2017. [10] Wickens, Christopher D, and Justin G Hollands. Engineering Psychology and Human Performance. 3rd ed., Prentice Hall, 2000, pp. 69-108. [11] Angle of Attack Indicators. Alpha Systems, 2017, www.alphasystemsaoa.com/.

62

[12] Federal Aviation Administration. (2003). General Aviation Technically Advanced Aircraft FAA-Industry Safety Study: Final Report of TAA Safety Study Team. FAA. https://www.faa.gov/training_testing/training/fits/research/media/TAA%20Final%20R eport.pdf [13] A. Bailey, Jr.. "Development of a Side-Stick Controller for Aerospace Vehicles", Testing of Manned Flight Systems Conference, Meeting Paper Archive, (1963) https://doi-org.portal.lib.fit.edu/10.2514/6.1963-1806 [14] Mike Bromfield, Guy Gratton, and Mark Young. "The Effects of Stick Force Gradient on Pilot Mental Demand", AIAA Atmospheric Flight Mechanics Conference, AIAA SciTech Forum, (AIAA 2015-0751) https://doi- org.portal.lib.fit.edu/10.2514/6.2015-0751 [15] Federal Aviation Administration (2011). Advisory Circular 23-8C, Flight Test Guide for Certification of Part 23 Airplanes. Retrieved from https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_23-8C.pdf [16] Diamond Aircraft. (2009). DA42 NG AFM: Airplane Flying Manual. Retrieved from http://support.diamond- air.at/fileadmin/uploads/files/after_sales_support/DA42_New_Generation/Airplane_Fl ight_Manual/Basic_Manual/70115e-r6-complete.pdf. [17] Koschlik, A. TUM-FAA-AOA-Memo-1, 21 April 2017. [18] Federal Aviation Administration (2015). Advisory Circular 120-109A, Stall Prevention and Recovery Training. Retrieved from https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_120-109A.pdf [19] Klyde, David H, et al. AIAA SciTech. “Flight Test Evaluation of the SAFE-Cue System for Loss of Control Mitigation” 13-17 Jan. 2014. Retrieved from https://info.aiaa.org/Regions/Western/Orange_County/ASAT%20Conference%20201 4%20Presentations/Klyde_AIAA-2014-0447_SAFE-Cue%20Flight%20Test.pdf [20] SkyVector Aeronautical Charts, skyvector.com/. [retrieved 1 December 2017]

63

Appendices Appendix A: Matlab Code for Generating Plots July 3

%July 3 %t_start = 990; %t_end = 1120; %plot_indices_wing = find((July3_2017.CUES.DSPL.t_rx_s >= t_start) & (July3_2017.CUES.DSPL.t_rx_s <= t_end)); plot_indices_boom = find((July3_2017.ADS.AOS_noseboom.t_rx_s >= t_start) & (July3_2017.ADS.AOS_noseboom.t_rx_s <= t_end)); figure(1) subplot(3,1,1) yyaxis left ylim([0 1]) plot(July3_2017.ADS.AOA_noseboom.t_rx_s,July3_2017.ADS.AOA_noseboom.alpha_deg ,'b') ylabel ('AOA in degrees') hold on yyaxis right; plot(July3_2017.CUES.DSPL.t_rx_s,July3_2017.CUES.DSPL.alpha_norm,'r') ylabel('Normalized AOA') xlim([2000 3000]) xlabel('Time in Seconds') legend('AOA in degrees','Normalized AOA') suptitle('July 3: Pilot A Stall Matrix: Straight Stalls') subplot(3,1,2) plot(July3_2017.SRF.FLAPS.t_rx_s,July3_2017.SRF.FLAPS.flaps_deg,'r') xlim([2000 3000]) xlabel('Time in seconds') ylabel('Retracted, Approach, Landing') title('Flap Setting') subplot(3,1,3) plot(July3_2017.SRF.TL_LH.t_rx_s,July3_2017.SRF.TL_LH.thrust_lever_lh_pct,'r' ) hold on; plot(July3_2017.SRF.TL_RH.t_rx_s,July3_2017.SRF.TL_RH.thrust_lever_rh_pct,'b' ) xlim([2000 3000]); xlabel('Time in Seconds'); ylabel('Power in Percentage'); ylim([0 100]) title('Power Setting'); legend('LH Engine','RH Engine');

July 4: Flight 2

%July 4 Flight 2 t_start = 3600; t_end = 3750;

64 plot_indices_boom = find((July4_2017_Flight2.ADS.AOA_noseboom.t_rx_s >= t_start) & (July4_2017_Flight2.ADS.AOA_noseboom.t_rx_s <= t_end)); figure(1) subplot(4,1,1) yyaxis left ylim([0 1]) xlim([t_start t_end]) plot(July4_2017_Flight2.ADS.AOA_noseboom.t_rx_s,July4_2017_Flight2.ADS.AOA_no seboom.alpha_deg,'b') ylabel ('AOA in degrees') hold on yyaxis right; plot(July4_2017_Flight2.CUES.DSPL.t_rx_s,July4_2017_Flight2.CUES.DSPL.alpha_n orm,'r') ylabel('Normalized AOA') %xlabel('Time in seconds') legend('AOA in degrees','Normalized AOA') %legend('Normalized \alpha','Actual \alpha') title('AOA Change With Time') grid on; subplot(4,1,2) plot(July4_2017_Flight2.SRF.FLAPS.t_rx_s,July4_2017_Flight2.SRF.FLAPS.flaps_d eg,'r') xlim([3600 3750]) %xlabel('Time in seconds') ylabel('Retracted, Approach, Landing') title('Flap Setting')

%{ subplot(4,1,3) plot(July4_2017_Flight2.ADS.AOS_noseboom.t_rx_s,July4_2017_Flight2.ADS.AOS_no seboom.beta_deg,'r') xlim([1000 2000]) xlabel('Time in seconds') ylabel('AOS in degrees') title('AOS')

%} suptitle('Landing: July 4 Flight 2')

%pitch in red, roll in blue subplot(4,1,3) plot(July4_2017_Flight2.STICK.POS.t_rx_s,July4_2017_Flight2.STICK.POS.pitch_p os_deg,'r') hold on %plot(July4_2017_Flight2.STICK.POS.t_rx_s,July4_2017_Flight2.STICK.POS.roll_p os_deg,'b') %hold on %plot(July4_2017_Flight2.STICK.NPOS.t_rx_s,July4_2017_Flight2.STICK.NPOS.roll _neutral_deg,'k') %hold on %plot(July4_2017_Flight2.STICK.NPOS.t_rx_s,July4_2017_Flight2.STICK.NPOS.pitc h_neutral_deg,'k') %hold on %need to get pitch angle into degrees theta=57.295779513*July4_2017_Flight2.INS.Theta_rad; plot(July4_2017_Flight2.INS.t_rx_s,theta,'m'); xlim([3600 3750]) 65 grid on; title('Pitch and Active Stick Pitch Axis Position') %xlabel('Time in seconds') ylabel('Pitch in Degrees') legend('Active Stick Pitch Axis Position', 'Pitch Angle Theta'); subplot(4,1,4) %pressure altitude in meters plot(July4_2017_Flight2.ADS.ALT.t_rx_s,July4_2017_Flight2.ADS.ALT.h_press_m,' b') hold on xlabel('Time in Seconds'); xlim([3500 3750]) ylabel('Altitude in meters'); title('Pressure Altitude'); %lat long info %{ figure(2); %lateral position lateral=57.295779513*July4_2017_Flight2.INS.pos_WGS84_phi_rad; plot(July4_2017_Flight2.INS.t_rx_s,lateral,'b'); figure(3); %longitudinal long=57.295779513*July4_2017_Flight2.INS.pos_WGS84_lambda_rad; plot(July4_2017_Flight2.INS.t_rx_s,long,'r'); %} figure(2) subplot(3,1,1) plot(July4_2017_Flight2.ADS.AOS_noseboom.t_rx_s,July4_2017_Flight2.ADS.AOS_no seboom.beta_deg,'r') xlim([3600 3750]) xlabel('Time in Seconds') title('AOS') ylabel('Angle in Degrees') subplot(3,1,2) suptitle('Landing: July 4 Flight 2'); plot(July4_2017_Flight2.SRF.GEAR.t_rx_s,July4_2017_Flight2.SRF.GEAR.gear_down ,'b') xlim([3600 3750]) title('gear indication') ylabel('Retracted, Extended') xlabel('Time in Seconds'); subplot(3,1,3) %calibrated airspeed (kts?) plot(July4_2017_Flight2.ADS.CAS.t_rx_s,July4_2017_Flight2.ADS.CAS.CAS_mDs,'r' ) ylabel('Airspeed in kts') xlim([3600 3750]) xlabel('Time in Seconds'); July 5 t_start=1830; t_end=2000; plot_indices_boom = find((July5_2017.ADS.AOA_noseboom.t_rx_s >= t_start) & (July5_2017.ADS.AOA_noseboom.t_rx_s <= t_end)); subplot(4,1,1) %plot(July5_2017.ADS.AOA_noseboom.t_rx_s, July5_2017.ADS.AOA_noseboom.alpha_deg, 'b'); %ylabel('AOA in Degrees') 66 hold on yyaxis right; plot(July5_2017.CUES.DSPL.t_rx_s,July5_2017.CUES.DSPL.alpha_norm,'r') ylabel('Normalized AOA') xlim([t_start t_end]) subplot(4,1,2) plot(July5_2017.TASK.ID.t_rx_s,July5_2017.TASK.ID.in_progress,'b') title('Tracking Task in Progress'); suptitle('July 5: Tracking Task Version 1: Pilot A') xlim([t_start t_end]) subplot(4,1,3) theta=57.295779513*July5_2017.INS.Theta_rad; plot(July5_2017.INS.t_rx_s,theta,'m'); ylabel('Pitch Angle in Degrees') title('Pitch Angle') xlim([t_start t_end]) subplot(4,1,4) phi=57.295779513*July5_2017.INS.Phi_rad; plot(July5_2017.INS.t_rx_s,phi,'m'); title('Roll Angle') xlim([t_start t_end]) ylabel('Roll Angle in Degrees') xlabel('Time in Seconds') July 7 Flight 2 %July7 Flight 2 t_start = 1175; t_end = 1220; plot_indices_boom = find((July7_2017_Flight2.ADS.AOA_noseboom.t_rx_s >= t_start) & (July7_2017_Flight2.ADS.AOA_noseboom.t_rx_s <= t_end)); figure(1) %altitude subplot(5,1,1) plot(July7_2017_Flight2.ADS.ALT.t_rx_s,July7_2017_Flight2.ADS.ALT.h_press_m,' r') xlim([t_start t_end]) ylabel('meters') title('Altitude in Meters') %AOA subplot(5,1,2) plot(July7_2017_Flight2.ADS.AOA_noseboom.t_rx_s, July7_2017_Flight2.ADS.AOA_noseboom.alpha_deg) hold on %plot(July7_2017_Flight2.CUES.DSPL.t_rx_s,July7_2017_Flight2.CUES.DSPL.alpha_ norm,'r') xlim([t_start t_end]) ylim([-5 20]) title('AOA') ylabel('AOA in Degrees') %Down component of velocity subplot(5,1,3) plot(July7_2017_Flight2.INS.t_rx_s,-July7_2017_Flight2.INS.vel_D_mDs,'r') xlim([t_start t_end]) ylabel('m/s') title('Down Component of Velocity') 67

%theta subplot(5,1,4) theta=57.295779513*July7_2017_Flight2.INS.Theta_rad; plot(July7_2017_Flight2.INS.t_rx_s,theta,'m'); ylabel('Pitch Angle in Degrees') title('Pitch Angle') xlim([t_start t_end]) %pusher activated subplot(5,1,5) plot(July7_2017_Flight2.CUES.FLGS.t_rx_s,July7_2017_Flight2.CUES.FLGS.pusher_ enbl,'r') xlim([t_start t_end]) xlabel('Time in Seconds') title('Pusher Activation') suptitle('Stick Pusher July 7: Flight 2') July 11 Flight 1 %July 11 Flight 1 stall matrix Pilot B t_start = 3800; t_end = 10500;

%plot_indices_wing = find((July11_2017_Flight1.CUES.DSPL.t_rx_s >= t_start) & (July11_2017_Flight1.CUES.DSPL.t_rx_s <= t_end)); %plot_indices_boom = find((July11_2017_Flight1.ADS.AOS_noseboom.t_rx_s >= t_start) & (July11_2017_Flight1.ADS.AOS_noseboom.t_rx_s <= t_end)); figure(1) yyaxis left ylim([0 1]) plot(July11_2017_Flight1.CUES.DSPL.t_rx_s,July11_2017_Flight1.CUES.DSPL.alpha _norm,'r--') ylabel ('Normalized AOA') hold on yyaxis right; plot(July11_2017_Flight1.ADS.AOA_noseboom.t_rx_s,July11_2017_Flight1.ADS.AOA_ noseboom.alpha_deg,'b-') % Use x lim to define range of time xlim([t_start t_end]) ylim([0 14]) ylabel('AOA in degrees') xlabel('Time in seconds') %legend('Normalized \alpha','Actual \alpha') title('AOA Change With Time: July 11 Flight 1') grid on;

figure(2) plot(July11_2017_Flight1.CUES.FLGS.t_rx_s,July11_2017_Flight1.CUES.FLGS.pushe r_enbl) xlim([t_start t_end]) figure(3) plot(July11_2017_Flight1.CUES.CTRL.t_rx_s,July11_2017_Flight1.CUES.CTRL.pushe r_activated) xlim([t_start t_end])

68

Appendix B: Pilot Survey and Data

Pilot Assessment of Certification Point Values for Stall Warning and Enhanced Indication System Options

Based on the experience gained during the Methods of Compliance flight test campaign, please provide the number of points each system should be awarded towards certification. 1. Stall Warning Implementation

Description Points (0 – 100)

Constant tone

Interrupted tone

Synthetic voice

Yoke or stick vibration

Visual indication independent of pilot focus

Warning in primary field of view

2. Enhanced Indication System Options

Description Points (0 – 40)

Angle of attack indication

Angle of attack indication with trend marker

Pitch limit indication displayed on attitude indication

Pitch limit indication displayed on attitude indication with trend marker

Indicated airspeed markings that change with flight condition

Indicated airspeed trend marker displayed on airspeed indication

Indicated altitude trend marker displayed on altitude indication

Flight path marker displayed on attitude indication

Warning issued when loss of control likely within trend window

69

Stall Warning Implementation Description High Low Average Constant Tone 60 5 30 Interrupted Tone 60 4 23 Synthetic Voice 75 15 40 Stick Vibration 100 60 81 Visual indication independent of pilot focus 40 10 20 Warning in primary field of view 40 7 22

Enhanced Indication System Options Description High Low Average AOA Indication 35 5 18 AOA Indication with trend marker 40 8 23 Pitch limit indication displayed on attitude indication 25 0 11 Pitch limit indication displayed on attitude indication with trend marker 30 0 17 Indicated airspeed markings that change with flight condition 25 7 15 Indicated airspeed trend marker displayed on airspeed indication 30 0 15 Indicated airspeed trend marker displayed on attitude indication 30 0 8 Flight path marker displayed on attitude indication 20 0 10 Warning issued when loss of control is likely within trend window 35 0 19

70

Appendix C: Software Changes and Flight Log Date Code Short Description Changes Version 7/3/2017 0.29.92 Ralph Kimberlin flying AOA Start Setting 7 cues / Active Stick level flight 7/4/2017 0.30.93 Ralph Kimberlin flying AOA • Audio Call Out changed to “Stall 0 cues / Active Stick configuration Stall” instead of “AOA Red Push Push” changes, banked flights • If pusher is deactivated, shaker remains activated all the time in red range • Automatic Trim up to 45° Bank instead of 20° • Landing configuration table fixed to actual landing configuration data 7/4/2017 0.31.93 Ralph Kimberlin flying AOA • Tracking Task output adapted, slower 1 cues / Active Stick patterns and variants of tracking tasks available tracking task 7/5/2017 0.32.93 Ralph Kimberlin flying AOA • Tracking Task adapted for angle of 3 cues / Active Stick tracking tasks attack tracking only • Angle of Attack source (nose boom or wing vanes) can be chosen per FTE Display 7/6/2017 0.33.93 Auto-TO and Landing • Tracking Task adapted for angle of 4 Demonstration attack tracking with configuration changes 7/6/2017 0.33.93 Test of Tracking Task with 4 Config Changes 7/7/2017 0.33.93 Brian Kish flying AOA flights 4 7/7/2017 0.33.93 Jennifer Geehan flying AOA 4 flights 7/10/2017 0.34.93 2h of Dave Sizoo flying AOA • Calspan Pitch Tracking Task re-added 6 flights • Red Audio Warnings re-changed to “AOA Red Push Push” instead of “Stall Stall” 7/10/2017 0.34.93 1,5h of Dave Webber flying 6 AOA flights 7/10/2017 0.34.93 Autoland Trial (Alignment 6 Problem) 7/11/2017 0.34.93 Autoland & Dave Sizoo AOA 6 Flight 7/11/2017 0.35.93 David Webber AOA Flight • Call Outs changed: 7 o AOA instead of AOA Yellow Push o Push Push instead of AOA Red Push Push

71

Appendix D: Stall Test Matrix

Point # Bank Bleed Rate Power Gear and Flaps 1 0° 1 kt/s Off Up 2 0° 3-5 kts/s Off Up 3 0° 1 kt/s Off Down 4 0° 3-5 kts/s Off Down 5 0° 1 kt/s On Up 6 0° 3-5 kts/s On Up 7 0° 1 kt/s On Down 8 0° 3-5 kts/s On Down 9 15° 1 kt/s Off Up 10 15° 3-5 kts/s Off Up 11 15° 1 kt/s Off Down 12 15° 3-5 kts/s Off Down 13 15° 1 kt/s On Up 14 15° 3-5 kts/s On Up 15 15° 1 kt/s On Down 16 15° 3-5 kts/s On Down 17 30° 1 kt/s Off Up 18 30° 3-5 kts/s Off Up 19 30° 1 kt/s Off Down 20 30° 3-5 kts/s Off Down 21 30° 1 kt/s On Up 22 30° 3-5 kts/s On Up 23 30° 1 kt/s On Down 24 30° 3-5 kts/s On Down 25 45° 1 kt/s Off Up 26 45° 3-5 kts/s Off Up 27 45° 1 kt/s Off Down 28 45° 3-5 kts/s Off Down 29 45° 1 kt/s On Up 30 45° 3-5 kts/s On Up 31 45° 1 kt/s On Down 32 45° 3-5 kts/s On Down

72

Appendix E: Test Cards: 3 July 2017

73

Flights executed:

• All of 1 o 1v: Looked like it should be. No unusual things on display. No upsets, no rescaling, no noticeable delay between aircraft dynamics and display. Display could be used as feedback for approaches. Audio off helped getting a good idea of what the plane felt like getting close to stall. For future flights: Test v option first, then go to all on. Ralph: Would be interesting to change CG position, but we don’t have any mass margin. • All of 3 • 5v, 5p • 7s, 7p • All of 2

74

• Power on, high bleed rate seems to be too aggressive, would cause the plane to get too much altitude and go ballistic, especially when the pusher comes on. Feedback:

• Active stick: o Feels like a natural airplane. o Force step change in yellow seems to be a good feature for inexperienced pilots, while experienced pilots would probably push through. Step change could be a little bit lighter, but was actually in a good range. With q feedback at high speed, the step change would qualitatively feel less. o Pusher has enough force and takes airplane from pilot, which is exactly what it’s supposed to do. Pilot could probably pull through it if he knew when it was going to kick on. o Flight did not have dynamic q feedback on stick. o For power off, clean configuration, low bleed rate, pusher was basically perfect. o Clean configuration, high bleed rate: pusher was okay, but force should not be increased. o Pusher should be turned off at low altitude, but that will not help against traffic pattern accidents. Pusher could be set to push less long or to a higher AoA in landing configuration. Pusher force could also be set to ramp up. But that might allow pilot to fly right through. We could also have it push surprisingly at full force and then ramp down or cut off earlier in landing configuration. o Forces felt good for the speed range. o Force increase for roll not well implemented, felt almost like pusher. Let’s turn this feature off for future flights. o Let’s not use force step as individual option. • AoA gauge: o Ralph was looking at it all the time, except once when it was in the Sun. o Change the graphics so that the minimum AoA is horizontal. • Audio: o Much aural feedback present anyways. Especially gear warning. But turning this off would also disable the elevator limitation for power-on stalls. So load of aural cues is very high and the AoA warning is hard to separate. o Audio is useful, but hard to evaluate with all the other signals in the background. “Push-push” came through clear, so did “Push”. First push came on in the middle of the yellow range, push-push at beginning of red. That’s where it should be. Clear voice command were distinguishable against background noise. Additional beeping noises would be very annoying and hard to distinguish. We could switch “AoA red” sound with a “Stall, stall” warning. • Shaker:

75

o Worked well. Frequency and amplitude gave a clear warning but were not too high. Some coupling to mechanical control system but not too aircraft dynamics or structure. o Safety pilot would have preferred higher frequency, lower amplitude. o Stick shaker and pusher do not go off at the same time. o No future shaking as AoA goes down. • Fly-By Wire: o System felt like a natural plane. So it’s no limitation for our current project. • Auto Trim System: o Turned off at a certain bank angle. o Triggered additional audio warnings. Future flights:

• Skip a and f flights. Plan for Tuesday, 07/04:

• Bank angles: 15° and 30° left and right • v, s, and p • Not 3-5 kts power on • Refly after changes: full flaps, power on and off, p test case: 3p, 7p o “Stall stall” instead of “AoA red” o Active q feedback

76

77