An Airline Evaluation of the Facilitation Skills Assessment Tool [FSAT] As a Device for Training and Assessing Facilitators

Wright State University CORE Scholar International Symposium on Aviation International Symposium on Aviation Psychology - 2007 Psychology 2007 An Airline Evaluation of the Facilitation Skills Assessment Tool [FSAT] as a Device for Training and Assessing Facilitators MaryJo O. Smith Ph.D. Guy M. Smith Ed.D. Captain Colin Budenberg Captain John Willats Captain Pete Griffiths Follow this and additional works at: https://corescholar.libraries.wright.edu/isap_2007 Part of the Other Psychiatry and Psychology Commons Repository Citation Smith, M. O., Smith, G. M., Budenberg, C., Willats, C., & Griffiths, C. (2007). An Airlinev E aluation of the Facilitation Skills Assessment Tool [FSAT] as a Device for Training and Assessing Facilitators. 2007 International Symposium on Aviation Psychology, 664-669. https://corescholar.libraries.wright.edu/isap_2007/22 This Article is brought to you for free and open access by the International Symposium on Aviation Psychology at CORE Scholar. It has been accepted for inclusion in International Symposium on Aviation Psychology - 2007 by an authorized administrator of CORE Scholar. For more information, please contact [email protected]. AN AIRLINE EVALUATION OF THE FACILITATION SKILLS ASSESSMENT TOOL [FSAT] AS A DEVICE FOR TRAINING AND ASSESSING FACILITATORS MaryJo O. Smith, Ph.D. Ypsilon Associates Lakeville, MN, USA Guy M. Smith, Ed.D. Embry-Riddle Aeronautical University Worldwide Daytona Beach, FL, USA Captain Colin Budenberg Thomsonfly (formerly Britannia Airways) Buxton, England, UK Captain John Willats British Airways / EasyJet Basingstoke, England, UK Captain Pete Griffiths UK Civil Aviation Authority West Sussex, England, UK Researchers and airline practitioners from the USA and UK collaborated in a four-year project to develop the Facilitation Skills Assessment Tool (FSAT), an instrument to assess the skills of an airline facilitator. The final phase of the FSAT Development Project was the evaluation of FSAT at two airlines – Thomsonfly (formerly Britannia Airways) in the UK and Southwest Airlines in the USA. Statistical analysis allowed the combination of data from both airlines; these 66 assessments demonstrated that FSAT was a valid and reliable instrument for measuring facilitation skills. Facilitation Skills briefings, crew debriefings in line operations, mediated debriefs, and critical incident stress Facilitation is a discussion-based teaching process debriefings (CISD) (Dismukes & Smith, 2000). In where the instructor helps the crew self-analyze, airlines, facilitators usually come from the instructor learn from their experiences, and draw their own staff, often with the assumption that an able instructor conclusions. Facilitation is effective with adults; is also a competent facilitator. However, facilitation especially those with extensive experience, when requires skills beyond those of a traditional instructor their performance is being examined. Facilitation is who is proficient in conveying information and appropriate for professionals, particularly when their assessing performance. In facilitation, both the technical or human factors skills are under scrutiny; it information and the assessment come from the crew; draws upon their professional experience to assess the facilitator encourages the crew to learn and their own performance rather than casting them as supports their self-assessment. The acceptance of the examinees exposed to the judgments of an evaluator. FAA’s Advanced Qualification Program (AQP) and Factors, self identified by facilitation, have a much the JAA’s Alternative Training and Qualification greater acceptance level by the individual and are Programme (ATQP) by the world’s airlines has more likely to have a permanent change in intensified the need for more and better-trained performance. Primarily, facilitation requires the crew facilitators. Facilitation skills are not intuitive; to be active participants in the learning process with extensive training and practice precede qualification the expectation that they will transfer their lessons- in this demanding craft. Inevitably, the need exists to learned to their work environment (Dismukes & assess facilitation skills to assure the success of Smith, 2000). individual instructors. In addition, the airlines have a need to evaluate facilitation programs to assure Facilitation in aviation training is central to Crew their return on investment for the resources spent Resource Management (CRM) training and on facilitation. debriefings for Line Oriented Flight Training (LOFT). Facilitation has expanded to include LOFT 664 Background of FSAT any airline training and evaluation system. It was designed to fulfill a twofold purpose – training and Dismukes, Jobe, & McDonnell (1997) of the NASA assessment. As a training tool, FSAT can be used to Ames Research Center published a study of LOFT train the trainer in facilitation skills. Similar to a debriefings in which they used a rating instrument, checklist, it can be used during facilitation to cue the The Debriefing Assessment Battery (DAB). While requisite skills. As an assessment tool, it can be used conducting a field evaluation of the DAB at to measure the trainer’s facilitation skills and to Northwest Airlines, Smith and Sasso were able to provide formative feedback. reliably rate facilitation of live debriefings with the DAB (Dismukes, McDonnell, & Jobe, 2000). Besides Based on comments from the Gatwick facilitation using the DAB as an assessment tool to rate workshop (Smith & Smith, 2004), the FSAT facilitation skills, Smith and Sasso developed an Development Team defined the Aims of Facilitation instructor training curriculum that used the DAB as a (page 1 of attachment, FSAT). The Aims of training tool to develop facilitation skills. In 2000, Facilitation were established to be the foundation for Dismukes and Smith edited Facilitation and defining the requisite facilitation skills. The team Debriefing in Aviation Training and Operations, recognized that there were numerous individual which became the industry standard for teaching techniques and personal approaches to performing facilitation. The Dismukes and Smith book promoted facilitation skills; however, the successful facilitator the DAB as an effective instrument for training and should fulfill the Aims of Facilitation. assessing facilitation skills. The FSAT measures 18 facilitation skills, broadly At Gatwick, UK, Smith and Smith (2004, May) categorized into Introduction, Discussion, and conducted a facilitation workshop for airline training Closing. Although the FSAT measurement scale may managers that featured the DAB. Two realities replaced by an airline’s grading scale, the FSAT was became clear in this workshop: (a) The DAB was a designed with an easy-to-use 5-point marking scale. research tool, not particularly suitable for assessing Conforming to the Stockdill and Smith the facilitation skills of airline trainers; and (b) The discriminatory marking scale, the FSAT scale uses a industry and the Civil Aviation Authority (CAA) two-step assessment of each skill. The first step is a needed a facilitation assessment tool to quantify broad assessment of the Trainer’s skill, relating it to a facilitation skills and ultimately to accredit airline standard or a company’s expectations (Needs instructors as facilitators. After the 2004 Gatwick Development, Professional and Competent, or High conference, the authors (two airline practitioners, one Standard). The second step is a numerical refinement regulator, and two researchers) formed a team to of the assessment, comparing the skill of the rated develop a facilitation assessment instrument. The facilitator to other facilitators within the selected team determined that two tasks were required: (a) to category. Though many airline assessment systems define the Aims of Facilitation and (b) to develop an prefer a composite numerical grade, the FSAT does airline-friendly instrument to assess the skills of a not lend itself to a sum of all scores. Every facilitator. The underpinnings of the development facilitation session may not require every skill on the project were the Dismukes et al. (1997) study, the FSAT instrument; therefore, a composite or total DAB, and the Dismukes and Smith (2000) book. score should not be calculated. The overall level of a From 2004 to 2007, the team collaborated to develop facilitator’s performance should be based on the need the Aims of Facilitation and the Facilitation Skills for specific skills. Assessment Tool (FSAT). The original FSAT, a set of 17 skills, was developed Facilitation Skills Assessment Tool (FSAT) by team deliberations grounded by previous research, the lessons-learned from the Gatwick workshop, and The FSAT Development Team agreed that the facilitation training experiences. The skills were successor of the DAB should take the form of a short modified after the team solicited feedback from practical checklist in plain language for easy use. external facilitation experts to provide content Moreover, the practitioners on the team insisted that validation of the instrument. The final phase in the the facilitation skills in the instrument should FSAT Development Project was the evaluation of conform to the realities and practicalities of airline FSAT at two airlines – Thomsonfly in the UK and training. FSAT is the response to the industry’s need Southwest Airlines in the USA. Prior to the formal to quantify facilitation skills and possibly to accredit evaluation of FSAT, the team visited each host

Load more