Indications of Workload Induced Stress in Human Voice
Total Page:16
File Type:pdf, Size:1020Kb
EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EUROCONTROL EXPERIMENTAL CENTRE EVALUATION OF THE HUMAN VOICE FOR INDICATIONS OF WORKLOAD INDUCED STRESS IN THE AVIATION ENVIRONMENT EEC Note No. 18/06 INO-1 AC STSS Project Published: December 2006 The information in this document is the property of the EUROCONTROL Agency. It may not be reproduced, even in part, in any form whatsoever without the prior agreement of the Agency. This report does not necessarily reflect the ideas or official policy of the Agency. REPORT DOCUMENTATION PAGE Reference: Security Classification: EEC Note No. 18/06 Unclassified Originator: Originator (Corporate Author) Name/Location: EEC - INO EUROCONTROL Experimental Centre (INOvative Research Area) Centre de Bois des Bordes B.P.15 F - 91222 Brétigny-sur-Orge CEDEX FRANCE Telephone: +33 (0)1 69 88 75 00 Sponsor: Sponsor (Contract Authority) Name/Location: EUROCONTROL Experimental Centre EUROCONTROL Experimental Centre Centre de Bois des Bordes B.P.15 F - 91222 Brétigny-sur-Orge CEDEX FRANCE Telephone: +33 (0)1 69 88 75 00 WEB Site: www.eurocontrol.int TITLE: Evaluation of the Human Voice for Indications of Workload Induced Stress in the Aviation Environment Authors Date Pages Figures Tables Annex References M. Hagmueller, 08/2005 81; +VI 17 7 4 100 E.Rank, G. Kubin (Graz University of Technology) Project Task No. Period INO-1 AC STSS Sponsor 2005 Distribution Statement: (a) Controlled by: Vu DUONG Head of INO (b) Special Limitations: None Descriptors (keywords): Stress, workload, air traffic control, speech analysis, voice analysis, contact free, non-intrusive Abstract: This report gives a literature review of stress analysis from the human voice. While a special focus is on workload-induced stress and the special treatment of the avionic environment, a general overview of the analysis of speech under stress using a broader definition of stress is given as well. We discuss definitions of stress and workload and point out the relation between stress and human speech. We provide a summary of useful speech databases for workload induced stress analysis. A discussion on speech signal features commonly used for stress analysis is the main part of this report. Research in the effect of workload-induced stress on speech signals has become a riveting issue recently, while it is still at an early stage. Current problems are the multitude of influence factors and the speaker dependence of the effects of stress on the speech signal. As speech has the advantage of being a medium naturally used in the air traffic control environment it seems to be worthwhile to further explore this approach on a long-term basis. NOTE: The methods discussed in this report are largely based on public available research literature. The ownership of the related intellectual property rights has not been determined in this literature study and should be sought directly from the authors of the publications cited in the ’references’ sections. PAGE INTENTIONALLY LEFT BLANK Evaluation of the Human Voice for Indications of Workload Induced Stress EUROCONTROL Preface Air Traffic Control (ATC) operators on duty have a large responsibility for the life of passengers and crews in the aircraft. High vigilance and performance are continually required. Human’s features vary in time, with health situation and by external events. A scientific principle developed by the psychologists R.M. Yerkes and J.D. Dodson in 1908, known as the Yerkes-Dodson law demonstrates an empirical relation between arousal and performance. Thereafter human performance increases with cognitive arousal until a certain point. After this point further increasing arousal results in a decreasing performance. In the ATC (Air Traffic Control) task good performance can be seen as the absence of unsafe actions caused by the human. Therewith the human part of the global ATC safety becomes proportional to the performance of the ATC operator. So, for an optimal safety of the human ATC task, an operator ideally needs moderate workload. Therefore it is common ATC practice to modulate the size of a control sector during the day, depending of the traffic load. With low traffic two or more sectors may be collapsed into one control unit with one controller, while with increasing traffic this unit is split again into the smaller sectors again, with one controller each. The aim is to hold the workload for the controller continuously at a moderate level to favour due to Yerkes-Dodson’s law ATC safety. Currently a supervisor of the control room decides on the so-called re-sectorisation. His/her decision is subjective. It is based on own experience, own assessment of a moderate workload and administrative constraints. Especially the assessment of a moderate workload for an individual is very difficult as it varies strongly by environmental stimuli and state of health. A real-time tool to evaluate objectively human stress indicators under a given workload, to keep the human at the optimal performance, could help to increase ATC safety. In the medical sense workload can be seen as an external stimuli, which cause physiological stress reactions of the human body. Stress modifies the levels of specific hormones, which in turn alter heart rate, respiration, blood pressure and transpiration. So, measuring these primary or secondary stress reactions could give objective information how this individual sense this workload. This could be an objective base for supervisor’s re-sectoring decisions. Off-line and under laboratory conditions such measurements have been undertaken, but they are not adapted to field studies nor to real working conditions. Hypothetically the human voice could be used as carrier to retrieve most of human secondary stress reactions without any physical contact to the subject. Human vocal folds are directly in touch with respiration and heart activity via the blood. Consequently the voice should be slightly modified thereof and could be used as indicator of human’s stress reaction. Based on recorded ATC speech samples we tried to investigate this hypothesis. In 1998 the EUROCONTROL Experimental Centre (EEC) initiated the ‘Study into the Detection of Stress in Air Traffic Control Speech’. The study was executed by the ‘Speech Research’ group of the British ‘Defence Evaluation Research Agency (DERA) in Malvern. The study was limited to detect higher stress and doesn’t show a strong correlation. Projet INO-1 AC STSS III EUROCONTROL Evaluation of the Human Voice for Indications of Workload Induced Stress Meanwhile speech research progressed and so we revisited the issue in form of a literature study. This note reports on the study ‘Stress and Speech – State of the Art Report 2005’ of the ‘Signal and Speech Processing Laboratory’ of the Graz University of Technology. The documented research results can be seen as promising. But the study shows as well, that especially the detection of decreasing performance under low workload conditions is requiring further basic research. Other visual analysis methods, of eyes and face, are proposed. Horst Hering Innovative Research Unit IV Projet INO-1 AC STSS Evaluation of the Human Voice for Indications of Workload Induced Stress EUROCONTROL TABLE OF CONTENTS 1. Introduction . 1 1.1. Overview . 1 1.2. Air Traffic Control Environment . 1 1.3. Types of Measurement . 3 2. Definitions Concerning Speech and Stress . 4 2.1. Definition of Stress in Relation to Stress . 5 2.2. Model of Speech Production under Stress . 6 2.3. Taxonomy of Stressors . 9 2.4. Emotional Speech . 10 3. Databases with Speech under Stress . 11 3.1. ATC Apprenticeship Database . 11 3.2. SUSC-0 Database . 11 3.3. SUSAS Database . 12 3.4. DCIEM Map Task Corpus . 12 3.5. DERA Numberplate Database . 12 4. Stress Observation in Human Speech . 14 4.1. Human Stress Classification . 14 4.2. Features for Stress Analysis . 15 4.3. Workload . 20 4.4. Work Underload, Boredom, Fatigue, and Sleep Deprivation . 23 4.5. Voice and Heartbeat . 24 5. Speech Processing with Speech under Stress . 25 5.1. Automatic Speech Recognition for Speech under Stress . 25 5.2. Stressed Speech Synthesis . 25 5.3. Stressed Speaker Identification and Verification . 26 6. Summary & Conclusion . 27 6.1. Databases . 27 6.2. Classification of Speech under Workload Induced Stress . 27 6.3. Relation to Research on Emotional Speech . 27 6.4. Discussion and Recommendations . 27 References . 29 Appendix . 37 A. Stress Analysis using Non-Speech Methods . 37 A.1. Visual Analysis . 37 A.2. Cardiovascular Activity . 38 A.3. Brain Activity . 39 A.4. Others . 39 B. Acronyms . 40 C. Overview of Speech Databases . 41 D. Extended Literature References . 42 Projet INO-1 AC STSS V EUROCONTROL Evaluation of the Human Voice for Indications of Workload Induced Stress PAGE INTENTIONALLY LEFT BLANK VI Projet INO-1 AC STSS Evaluation of the Human Voice for Indications of Workload Induced Stress 1. Introduction Stress is the physiological and psychological answer of a human body to a specific workload. Stress influences human organ functions. Measuring these indications of stress is possible under laboratory conditions. Field studies under normal working conditions are not practical and some measures cannot be obtained in real-time. In the avionic domain, optimal safety is directly related to the correct level of workload for the human operator. Underload and overload increase the human error rate. This optimal workload level is different for every human being and may vary with the daily condition. Knowledge about workload induced stress of the human operators could help to keep them at a desirable level of workload for optimal safety. 1.1. Overview This study focuses on stress analysis from the human voice and presents a literature review of recent investigations on this topic. We start in this chapter with a short introduction of the air traffic control environment. In chapter 2 some relevant background knowledge on stress and speech signals is given.