University of South Florida Scholar Commons Graduate Theses and Dissertations Graduate School January 2012 Overcoming Limitations of Serial Audio Search Isabela Cordeiro Ribeiro Moura Hidalgo University of South Florida, [email protected] Follow this and additional works at: http://scholarcommons.usf.edu/etd Part of the American Studies Commons, Computer Engineering Commons, and the Computer Sciences Commons Scholar Commons Citation Hidalgo, Isabela Cordeiro Ribeiro Moura, "Overcoming Limitations of Serial Audio Search" (2012). Graduate Theses and Dissertations. http://scholarcommons.usf.edu/etd/4072 This Dissertation is brought to you for free and open access by the Graduate School at Scholar Commons. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Scholar Commons. For more information, please contact [email protected]. Overcoming Limitations of Serial Audio Search by Isabela C. R. M. Hidalgo A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science and Engineering College of Engineering University of South Florida Major Professor: Dewey Rundus, Ph.D. Rafael Perez, Ph.D. Les Piegl, Ph.D. Douglas Rohrer, Ph.D. Patricia Zarate, Ph.D. Date of Approval: March 9, 2012 Keywords: Parallel Output, Simultaneous Sounds, Music Browser, Wii Remote, Human- Computer Interaction, Auditory User Interfaces Copyright © 2012, Isabela C. R. M. Hidalgo Dedication This dissertation is dedicated to my family, for their support, encouragement, inspiration, and infinite patience. Table of Contents List of Tables .................................................................................................................... iv List of Figures .................................................................................................................... v Abstract ............................................................................................................................vii Chapter 1 Introduction ..................................................................................................... 1 1.1 Motivation ..................................................................................................... 1 1.2 Approach ...................................................................................................... 3 1.3 Contributions ................................................................................................ 4 1.4 Organization of the Dissertation ................................................................... 5 Chapter 2 Background and Related Work ...................................................................... 6 2.1 Searching and Browsing ............................................................................... 6 2.2 Image and Video Search .............................................................................. 7 2.3 Audio Search ................................................................................................ 8 2.3.1 Sound Effect Classification ............................................................ 9 2.3.2 Genre Classification ..................................................................... 10 2.3.3 Recommendation Systems .......................................................... 11 2.3.4 Audio Fingerprinting ..................................................................... 12 2.3.5 Query-by-Humming ...................................................................... 12 2.3.6 Conclusion ................................................................................... 14 2.4 Audio Visualization ..................................................................................... 15 2.5 Overview of Spatial Hearing ....................................................................... 16 2.5.1 Sound Localization ....................................................................... 16 2.5.1.1 Binaural Cues ................................................................ 17 2.5.1.2 Pinna ............................................................................. 17 2.5.1.3 Localization in the Vertical Plane .................................. 18 2.5.1.4 Loudspeakers vs. Headphones ..................................... 18 2.5.2 Concurrent Presentation of Sounds ............................................. 19 2.5.3 Simultaneous Sounds on User Interfaces .................................... 20 2.5.4 Conclusion ................................................................................... 22 Chapter 3 Parallel Audio Player .................................................................................... 23 3.1 Description .................................................................................................. 23 3.2 Hardware .................................................................................................... 23 3.2.1 Loudspeakers and Infrared Cameras .......................................... 23 3.2.2 Remote Control ............................................................................ 25 3.3 Software ..................................................................................................... 26 3.3.1 The Audio Module ........................................................................ 26 3.3.2 The Wiimote Module .................................................................... 27 i 3.3.3 Operating Modes ......................................................................... 29 Chapter 4 Experimental Methods and Design .............................................................. 33 4.1 Goal ............................................................................................................ 33 4.2 Questions ................................................................................................... 33 4.3 Experiment 1: Method ................................................................................ 34 4.3.1 Participants .................................................................................. 35 4.3.2 Instruments .................................................................................. 36 4.3.2.1 Parallel Audio Player ..................................................... 36 4.3.2.2 Survey ........................................................................... 36 4.3.3 Materials ...................................................................................... 36 4.3.3.1 Musical Genre Taxonomy ............................................. 36 4.3.3.2 Audio Files .................................................................... 38 4.3.4 Procedure .................................................................................... 40 4.3.4.1 Potential Risks and Benefits to Participants .................. 40 4.3.4.2 Pre-Manipulation ........................................................... 40 4.3.4.3 Manipulation .................................................................. 41 4.3.4.4 Post-Manipulation ......................................................... 43 4.3.5 Performance Measures ................................................................ 44 4.4 Experiment 2: Method ................................................................................ 46 4.4.1 Participants .................................................................................. 47 4.4.2 Instruments .................................................................................. 47 4.4.2.1 Parallel Audio Player ..................................................... 47 4.4.2.2 Survey ........................................................................... 47 4.4.3 Materials ...................................................................................... 48 4.4.3.1 Audio Files .................................................................... 48 4.4.3.2 Musical Genre Taxonomy ............................................. 48 4.4.4 Procedure .................................................................................... 49 4.4.5 Performance Measures ................................................................ 49 Chapter 5 Results ......................................................................................................... 50 5.1 Background Information ............................................................................. 50 5.1.1 Effect Sizes .................................................................................. 51 5.1.2 Assumption of Normality .............................................................. 52 5.1.3 Order and Sequence Effects ........................................................ 52 5.2 Experiment 1: Main Effects ......................................................................... 52 5.2.1 Effect of Audio Output on Search Time ....................................... 53 5.2.2 Effect of Audio Output on Number of Steps ................................. 54 5.3 Experiment 1: Additional Effects ................................................................. 55 5.3.1 Background Information ............................................................... 55 5.3.2 Duration of Search Stages ........................................................... 55 5.3.3 Steps per Search Stage ............................................................... 56 5.3.4 Time before First Step per Search Stage .................................... 58 5.3.5 Time Listening to the Target Sample per Search Stage .............. 60 5.3.6 Number of Incomplete