Acoustical Awareness for Intelligent Robotic Action
Total Page：16
File Type：pdf, Size：1020Kb
ACOUSTICAL AWARENESS FOR INTELLIGENT ROBOTIC ACTION A Dissertation Presented to The Academic Faculty By Eric Martinson In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in Computer Science Georgia Institute of Technology December, 2007 Copyright © 2007 by Eric Martinson ACOUSTICAL AWARENESS FOR INTELLIGENT ROBOTIC ACTION Approved by: Dr. Ronald Arkin, Advisor Dr. Frank Dellaert School of Interactive Computing School of Interactive Computing Georgia Institute of Technology Georgia Institute of Technology Dr. David Anderson Dr. Thad Starner School of Electrical and Computer School of Interactive Computing Engineering Georgia Institute of Technology Georgia Institute of Technology Dr. Tucker Balch Date Approved: November 9, 2007 School of Interactive Computing Georgia Institute of Technology ACKNOWLEDGEMENTS First and foremost, I would like to thank my wife, Lilia Moshkina, for the countless hours she spent helping me out in many different ways. She was always encouraging, helped out with the robots when it was needed, was a good sounding board for refining ideas, and was an excellent editor for this dissertation. In general, Lilia was invaluable to this work, and this dissertation would likely not have been completed without her. Second, I would like to thank my advisor, Ron Arkin, along with the rest of my committee: Dave Anderson, Tucker Balch, Frank Dellaert, and Thad Starner. Ron provided me with the freedom to explore this subject and truly identify a topic of my own choosing, while the committee was enthusiastic and helpful all the way to the end. Finally, I would like to thank the people at the Naval Research Laboratory Center for Artificial Intelligence. In particular, thanks to Alan Schultz and Derek Brock for their early acceptance of this line of research. Their experience and general support was a great asset towards refining the work and getting it published. Thanks also to Ben Fransen, a fellow intern, for being a patient listener on numerous occasions throughout two summers. iii TABLE OF CONTENTS Acknowledgements __________________________________________ iii List of Tables _____________________________________________ viii List of Figures _______________________________________________ x Summary ______________________________________________ xv CHAPTER 1 - Introduction ____________________________________ 1 1.1 Terminology_____________________________________________ 3 1.2 Research Question _______________________________________ 4 1.3 Contributions____________________________________________ 6 1.4 Dissertation Overview ____________________________________ 6 CHAPTER 2 – Related Work __________________________________ 8 2.1 Sound Source Localization _________________________________ 9 2.2 Natural Language Interfaces ______________________________ 17 2.3 Audio Classification _____________________________________ 23 2.4 Audio Probing __________________________________________ 26 2.5 Sound Vocalization ______________________________________ 27 2.6 Summary of Application Domains _________________________ 29 iv CHAPTER 3 – Acoustical Awareness ___________________________ 30 3.1 Types of Awareness______________________________________ 31 3.2 Knowledge for Acoustical Awareness _______________________ 35 3.3 Mathematical Framework for Sound Propagation ____________ 42 3.4 Chapter Summary ______________________________________ 66 CHAPTER 4 – Acquiring Knowledge about the Auditory Scene _____ 69 4.1 Building Blocks _________________________________________ 70 4.2 Representations for Characterizing Sound Sources ___________ 84 4.3 Path Information _______________________________________ 141 4.4 Building Maps without Models ___________________________ 153 4.5 Chapter Summary _____________________________________ 158 CHAPTER 5 – The Autonomous Mobile Security Robot __________ 162 5.1 Related Work in Security Robotics ________________________ 163 5.2 Monitoring the Auditory Scene ___________________________ 165 5.3 Improving the Signal-To-Noise Ratio ______________________ 203 5.4 Chapter Summary _____________________________________ 230 CHAPTER 6 – The Stealthy Approach Scenario _________________ 232 6.1 How to Hide a Noisy Robot ______________________________ 235 v 6.2 Experimental Results ___________________________________ 242 6.3 Discussion of Results ____________________________________ 253 6.4 Chapter Summary _____________________________________ 265 CHAPTER 7 – Acoustical Awareness for Human-Robot Interaction 267 7.1 A Model of Human Acoustical Awareness __________________ 269 7.2 Robotic Adaptations ____________________________________ 274 7.3 Combining Types of Awareness __________________________ 285 7.4 Chapter Summary _____________________________________ 288 CHAPTER 8 – Summary and Contributions ____________________ 290 8.1 How Can Acoustical Awareness be Applied to Mobile Robotics? 291 8.2 Contributions__________________________________________ 301 8.3 Conclusion ____________________________________________ 305 Appendix A - Software Design ________________________________ 307 A.1. Hardware _____________________________________________ 307 A.2. Software Processes _____________________________________ 309 A.3. Database Design _______________________________________ 318 Appendix B - Knowledge Gathering Tools ______________________ 330 B.1. Spatial Likelihoods _____________________________________ 330 vi B.2. Auditory Evidence Grids ________________________________ 333 B.3. Directivity Models ______________________________________ 339 B.4. Mel Frequency Cepstral Coefficients ______________________ 342 B.5. Creating Direct Field Maps ______________________________ 346 B.6. Ray-Tracing for Direct and/or Reverberant Field Maps ______ 347 B.7. Sampled Data Noise Maps _______________________________ 354 Appendix C - Guiding Robotic Movement ______________________ 358 C.1. Clear-Space Map _______________________________________ 358 C.2. Patrolling the Environment ______________________________ 360 C.3. Investigation of a Sound Source __________________________ 368 Appendix D - HRI Application _______________________________ 372 D.1. Selecting Speech Volume ________________________________ 374 D.2. Pausing for Interruptions ________________________________ 375 D.3. Rotating to Face the Listener _____________________________ 376 D.4. Relocating the Robot____________________________________ 377 References _____________________________________________ 380 vii LIST OF TABLES Table 4.1. The results of all phase 1 auditory evidence experiments. ______________ 94 Table 4.2. Mean localization error when auditory evidence grids are used with data collected by a moving robot. _______________________________________ 113 Table 4.3. Mean localization and orientation error as produced by the discovery process. ______________________________________________________________ 115 Table 4.4. Mean error in identifying the direction of maximum volume, as produced by an area coverage task. ____________________________________________ 124 Table 4.5. Localization and orientation accuracy of the two source discovery process 126 Table 5.1. List of trials completed by the robot for this scenario. All used the same patrol route, but varied in the types and numbers of active sources in the environment. ___________________________________________________ 174 Table 5.2. The relative performance of using the proposed maximum likelihood approach for detecting each type of source in the environment as a new source. ______ 179 Table 5.3. Illustrates successes in detecting and localizing radios playing music, compared across different environment types. ________________________ 185 Table 5.4. Performance of both the detection and localization algorithms for environments with at least one new sound source present. ______________ 187 Table 5.5. Summary of belief states used for each patrol run through the environment. 194 Table 5.6. Results of the source change detection algorithms, compared across different numbers of changes in the environment. _____________________________ 196 Table 5.7. Average reduction in noise levels using different relocation strategies to avoid music sources __________________________________________________ 211 Table 5.8. Results of the adaptive waypoint following algorithm averaged over the entire path. __________________________________________________________ 225 Table 5.9. Results of the adaptive waypoint following algorithm for a 3x3-m2 region in front of the sound source. _________________________________________ 226 Table 6.1. Acoustic hiding results in the presence of a 67-dB radio source. _______ 246 viii Table 6.2. Acoustic hiding results in the presence of a 67-dB radio source and a loud revereberant field. ______________________________________________ 248 Table 6.3. Acoustic hiding results in the presence of a 67-dB radio source located near a wall. _________________________________________________________ 251 Table 6.4. Acoustic hiding results in the presence of a 54-dB filter source. _______ 252 ix LIST OF FIGURES Figure 3.1. Basic reactive acoustically aware system. __________________________ 32 Figure 3.2 Reactive acoustically-aware system with behavioral coordination. _______ 32 Figure 3.3 Hybrid architecture for supporting acoustical awareness. _______________ 34 Figure 3.4. 3D model of the Aware Home Laboratory used for estimating reverberation effects and general sound propagation. _______________________________ 39 Figure 3.5 Direct vs. indirect paths from source to receiver.