System Safety Evaluation, Capstone Phase I Implementation
Total Page:16
File Type:pdf, Size:1020Kb
Flight Standards Service Alaskan Region, Flight Standards Division Report to the Manager, AAL-200 System Safety Evaluation Capstone Phase 1 Implementation Yukon Kuskokwim Delta, Western Alaska System Safety and Analysis Branch, AAL-240 March 5, 2003 Page 1 of 6 Executive Summary Purpose. To evaluate Alaskan Region, Flight Standards Division effectiveness and participation in the Capstone Phase I implementation and operational approval process. Background. Phase I of the Capstone project has been operational in Western Alaska since 1998. Twenty-seven FAR Part 135 operators have been using Capstone throughout the operating area with equipment installed in approximately 180 individual aircraft. In addition there are 15 Capstone equipped aircraft operating under Part 91. Phase I installations are primarily for VFR operations. An initial purpose of Capstone was to reduce aircraft accidents and fatalities due to controlled flight into terrain (CFIT), and a review of the implementation process by Mitre® Corporation shows an overall safety improvement in the Capstone Phase I operations area. Accident statistics, however, do not show a reduction in CFIT accidents and the safety improvement in Western Alaska could not be directly attributed to Capstone. Data also suggested that not all pilots have been trained effectively in the proper application of Capstone and may not be using the equipment as designed. Capstone Phase II involves the installation of additional advanced avionics and includes plans for IFR operations in Southeast Alaska. Phase II is scheduled to be implemented during the first quarter of calendar year 2003. Equipment installation is currently underway and training program modules are being evaluated for deployment to operators. The Flight Standards Division Manager requested a systems evaluation of Capstone Phase I to determine the validity of reports that the program is not being used effectively. Lack of pilot training and non-standard use of equipment indicates that Federal Aviation Administration (FAA) Flight Standards responsibilities may not have been met during the training program and operational approval process. Conclusions. Using questionnaires to survey FAA and operator personnel involved in Capstone Phase I, the System Safety and Analysis Branch determined that many operators did not develop comprehensive policy and procedure for the use of Capstone equipment. The result was non-standard use of the equipment by company pilots. The evaluation also indicated that operator training programs were not approved in accordance with FAA directives. Recommendations. The following actions are recommended based on evaluation findings. 1. Establish national guidance for implementation and operational use of Capstone. Current guidance on technology related to the Capstone project should be reviewed and updated. 2. Establish effective liaison between Capstone Program Office and Flight Standards Division, AAL-200, which includes written procedure regarding that interface. 3. Standardize the approval process for use of the University of Alaska Anchorage Capstone training module. 4. Ensure certificate holders develop policy and procedure for use of Capstone equipment. 5. Provide mandatory Capstone program training for inspectors who are reviewing operator programs for acceptance or approval as appropriate. 6. Develop standards for using Capstone equipment during check flights. 7. Provide Capstone equipment operational technical training for inspectors who will be conducting check flights. This may require use of the regional King Air or rental aircraft properly equipped with Phase I or Phase II equipment as appropriate. 8. Ensure that the PTRS and SDR databases are used for Capstone related activities in accordance with national guidance. 9. Complete the TapRooT® analysis of causal factors identified through the SnapCharT® and Root Cause Tree® process. The process includes documenting and tracking proposed corrective actions for each causal factor as well as identifying the office and/or individual responsible for that corrective action. Page 2 of 6 Purpose and Scope. Evaluate Alaskan Region, Flight Standards Division participation in the Capstone Phase I implementation and operational approval process to determine if Flight Standards responsibilities were met. Data gathered during the evaluation will be used to enhance phase II implementation. Background. The Capstone project was initiated in 1997 by a congressional mandate to introduce advanced electronics systems to improve aviation safety, particularly in the FAR Part 135 environment. A primary input to this effort was a study of operations in Alaska (Aviation in Alaska, NTSB/SS-95/03) that recommended using global positioning and satellite-based communications systems to provide an infrastructure that better meets Alaska’s air transportation system needs. An offshoot of Vice President Al Gore’s Flight 2000 program, Capstone was designed to reduce the fatal accident rate due to navigational errors and mid-air collisions. Under Phase I of Capstone, the Yukon Kuskokwim Delta in Western Alaska was selected to test the first generation of avionics and ground based electronics to improve safety and system efficiency. The Delta is an area of extensive FAR Part 135 commuter operations originating from a central hub located at Bethel, Alaska. The operating environment is hostile most of the year and includes severe winter weather with low temperatures, icing, snow, high winds, and widespread whiteout conditions. Most operations are VFR using small, single engine aircraft with a high incidence of CFIT accidents throughout the area. The introduction of new situational awareness tools such as terrain and mid air collision warnings for VFR operations is expected to have the greatest impact in reducing accidents and improving safety. Following deployment of Phase I in Western Alaska, a second wave of Capstone, Phase II, is being implemented in Southeast Alaska, where there is an equally hostile environment, but for different reasons. This area is comprised generally of coastal mountains with numerous valleys and fjords. Like Western Alaska, local communities are served primarily by air using small, single engine aircraft also with a high incidence of CFIT accidents due to low visibility, rain, and fog. Phase II uses more sophisticated technologies and IFR en route and approach operations. After three years of Phase I operations, the Capstone Program Office (CPO) commissioned the Mitre® Corporation to collect data, review project accomplishments, and determine if primary objectives were met. In August 2002, the Mitre® Corporation published an interim report summarizing preliminary findings that indicated lack of pilot training and non-standard use of the Capstone equipment by pilots. With Phase II implementation pending in Southeast Alaska, it was decided that a complete systems evaluation of Capstone was warranted to determine if Flight Standards in the Alaskan Region was fulfilling its responsibilities, which include training program surveillance as well as operational approvals. Evaluation findings would first be applied to enhance Phase II and then be employed retroactively where appropriate in Phase I. An evaluation team was formed with specialists from the System Safety and Analysis Branch (AAL-240) as the nucleus. The team included an Anchorage Flight Standards District Office principal operations inspector experienced in Capstone Phase I, along with operations and airworthiness inspectors from the Juneau FSDO, which will be the field office responsible for operational approvals for Phase II. Additional support was available from field supervisors and managers during the actual evaluation process. Methodology. In the past several years, the FAA initiated a program to ensure that certificate holders apply the principles of risk management in their operations. This follows the concepts of system safety, which uses managerial techniques in a systematic manner to identify and control hazards. System safety has been introduced into FAR Part 121 operations through Air Transportation Oversight System (ATOS) and Certification Standardization and Evaluation Team (CSET) programs. The Alaskan Region is active in introducing system safety to local FAR Part 135 Page 3 of 6 operations and conducting a systems evaluation on Capstone provides opportunity to apply these principles to that program. The first step was to identify the purpose of the evaluation in terms of functional requirements and existing problems. Data in the Mitre® report indicated a lack of effective pilot training and non-standard use of Capstone equipment. The evaluation team believed that if these two conditions actually existed, they reflected potential failure in program oversight since Flight Standards is responsible for approving training programs and operating authorizations. A systems evaluation would determine if these responsibilities were met during Capstone implementation. The next step was to identify local stakeholders that affect Flight Standards with respect to Capstone Phase I in Alaska. Applying the principles of system safety, the team noted that the interfaces attribute points directly to where Flight Standards played an active role in the process by granting approval for certificate holders to use the equipment. The team determined that there were four key stakeholder groups located in Alaska whose direct exchange of information affected operational approval. The four stakeholders identified