DOCSLIB.ORG

The Music Emerges: a Computational Approach to the Speech-To-Song Illusion

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Music Emerges: a Computational Approach to the Speech-To-Song Illusion University of Amsterdam Masters Thesis The Music Emerges: A Computational Approach to the Speech-to-Song Illusion Author: Supervisor: Arran Lyon Dr. Makiko Sadakata A thesis submitted in partial fulfilment of the requirements for the degree of Master of Science in Computational Science in the Computational Science Lab Informatics Institute August 2020 Declaration of Authorship I, Arran Lyon, declare that this thesis, entitled ‘The Music Emerges: A Computational Approach to the Speech-to-Song Illusion’ and the work presented in it are my own. I confirm that: This work was done wholly or mainly while in candidature for a research degree at the University of Amsterdam. Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated. Where I have consulted the published work of others, this is always clearly at- tributed. Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work. I have acknowledged all main sources of help. Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed myself. Signed: Date: 10 August 2020 i UNIVERSITY OF AMSTERDAM Abstract Faculty of Science Informatics Institute Master of Science in Computational Science The Music Emerges: A Computational Approach to the Speech-to-Song Illusion by Arran Lyon The Speech-to-Song Illusion is an auditory effect whereby a perceptual transformation occurs during an exact repetitions of some short, naturally spoken phrase, where the listener begins to hear as if the speaker is singing, when initially it sounded exactly like speech. Understanding the cognitive mechanisms underlying the illusion could reveal the inner workings of speech, song and music perception. We use a collection of audio stimuli (300) related to this illusion along with a parallel set of perceptual rating data obtained from a previous study. We describe an algorithm that automatically tran- scribes the melody as a sequence of tones within the natural voice that improves upon the original formulation. We define a set of 33 melodic, rhythmic, audio and dissonance related features measured from the raw audio and the extracted tone sequence, including features previously connected to the occurrence of the illusion. New features include a metric that measures the distance of the melody to typically composed melodies (accord- ing to a Bayesian model), along with a novel set of features that captures the dynamic change of tension and release of the musical phrase based on a measure of sensory disso- nance. We then propose a suitable method to best assign a binary label (transforming or non-transforming) to the stimuli based on the continuous range of scores provided by participants of an experiment. After under going a feature selection procedure, several data classifiers (linear and non-linear support vector machines and logistic model, along with two ensembles) that predict if the stimulus will transform from the features all ob- tain balanced accuracy scores between 0.66 and 0.71, where each model used between 7 and 10 features. We confirm that stability plays an important part in the illusion, along with the closeness of the melody to typical melodies and the consonance of the ending of the melody, suggesting transforming stimuli are those with characteristics of ordinary musical phrases. Finally, 55 participants took part in a follow up validation experiment with 98 new stimuli to test if the models generalised well, and found that one model in particular (a linear support vector machine) maintained a score above baseline on the new vocal stimuli (balanced accuracy 0.65 on the new stimuli). Acknowledgements First and foremost, I would like to thank my supervisor Dr. Makiko Sadakata for in- troducing me to this curious illusion, and for all her continuous and encouraging support in this research. Next, to Bas Cornelissen for his valuable and foundational contribution for which much of this work is based on, and without this would not have been possible, and also to Gerben Groenveld for his feedback and audio materials from his own research into the illusion. A huge thanks to Prof. Henkjan Honing who has kindly contributed his time to evaluating the work, and for the valued early feedback in the process. I would like to offer a my sincerest gratitude to my friends Valentin Vogelmann, for his helpful advice on theoretic concepts, and to Mattia Capelletti for his cultural perspective. A warm thank you goes to my good friend Tanja, for her continued support over many long walks. Finally, my gratitude extends to friends Charlie, Charlotte, Efi, Gerben, David and Alice for their advice, feedback and inspiration, to Black Gold Amsterdam for the music, and to all my friends and family across the continent. iii Contents Declaration of Authorshipi Abstract ii Acknowledgements iii Contents iv List of Figures vi List of Tables vii 1 Introduction1 1.1 The Mystery of Repetition...........................1 1.2 The Transformation of Sound.........................3 1.2.1 Auditory Illusions...........................3 1.2.2 Repetition Based Illusions.......................4 1.2.3 The Speech-to-Song Illusion......................5 1.2.4 Beyond Speech and Song.......................7 1.3 Thesis Overview................................7 2 Melody Extraction9 2.1 Audio Analysis Methods............................9 2.2 The Melody Extraction Algorithm...................... 10 2.2.1 Problem Statement and Definitions.................. 11 2.2.2 Note Segmentation........................... 11 2.2.3 Note Extraction............................ 14 2.2.4 Parameterisation and Evaluation................... 16 2.3 Bayesian Melody Search............................ 19 2.3.1 Method Outline............................ 20 2.3.2 Distribution of Melodies........................ 20 2.3.3 Note and Key Calculation....................... 22 2.3.4 Search Space.............................. 24 3 Feature Engineering 25 3.1 Audio Features................................. 25 3.2 Melodic Features................................ 27 iv Contents v 3.3 Rhythmic Features............................... 29 3.4 Dissonance Features.............................. 32 3.4.1 Quantifying Dissonance........................ 33 3.4.2 The Self-Similarity Matrix....................... 34 3.4.3 Feature Measurements......................... 36 4 Data Analysis 37 4.1 Audio Stimuli.................................. 37 4.1.1 Materials................................ 38 4.1.2 Human Ratings............................. 38 4.1.3 Manipulated Stimuli.......................... 40 4.2 Data Preparation................................ 41 4.2.1 Aggregation Schemes.......................... 41 4.2.2 Data filtering.............................. 42 4.3 Feature Distributions and Correlations.................... 44 5 Classification Models 47 5.1 Statistical Modelling.............................. 47 5.1.1 The Logistic Model........................... 48 5.1.2 Support Vector Machines....................... 49 5.1.3 Ensemble Methods........................... 50 5.2 Feature Selection................................ 50 5.2.1 Evaluation............................... 51 5.2.2 Search Procedure............................ 52 5.3 Results...................................... 54 6 Validation Experiment 56 6.1 Setup...................................... 56 6.1.1 Material................................. 57 6.1.2 Procedure................................ 59 6.2 Results...................................... 61 6.2.1 Rating Distributions.......................... 61 6.2.2 Feature Analysis............................ 63 6.2.3 Model Predictions........................... 64 7 Discussions and Conclusion 66 7.1 Feature Methods................................ 66 7.2 Data Analysis.................................. 70 7.3 The Models................................... 72 7.4 Experiment Results............................... 76 7.5 Final Thoughts................................. 78 A Feature Summary 80 B Feature Correlations 82 C Model Features 84 References 85 List of Figures 2.1 Stages of the note segmentation procedure.................. 12 2.2 Comparison between the transcription and the stable mean pitch note extraction method................................ 15 3.1 Krumhansl-Kessler key profiles for each pitch class.............. 27 3.2 Example of a normalised onset envelope, along with the note onset times. 31 3.3 The dissonance measure curve for two notes for a range of ratios of their fundamental frequency.............................. 34 3.4 Self-similarity plots using the dissonance measure of a six note piano melody. 35 4.1 Distribution of final scores in mcg....................... 39 4.2 Features measureed in manipulated stimuli.................. 40 4.3 Distribution of data points of at from the ratings obtained from the mcg experiment.................................... 42 4.4 Distribution of mcg data subsets, and the effects of filtering the data... 43 4.5 Distributions of stability scores for transforming and non-transforming stimuli...................................... 44 4.6 Distribution of stimuli for three features against their ratings........ 46 6.1 Screenshots of the experiment box at the four main stages of a trial.... 60 6.2 Distributions of scores for all stimuli in the experiment, along with cate- gorical breakdown...............................
Load more

Recommended publications
  • Major Heading
    THE APPLICATION OF ILLUSIONS AND PSYCHOACOUSTICS TO SMALL LOUDSPEAKER CONFIGURATIONS RONALD M. AARTS Philips Research Europe, HTC 36 (WO 02) Eindhoven, The Netherlands An overview of some auditory illusions is given, two of which will be considered in more detail for the application of small loudspeaker configurations. The requirements for a good sound reproduction system generally conflict with those of consumer products regarding both size and price. A possible solution lies in enhancing listener perception and reproduction of sound by exploiting a combination of psychoacoustics, loudspeaker configurations and digital signal processing. The first example is based on the missing fundamental concept, the second on the combination of frequency mapping and a special driver. INTRODUCTION applications of even smaller size this lower limit can A brief overview of some auditory illusions is given easily be as high as several hundred hertz. The bass which serves merely as a ‘catalogue’, rather than a portion of an audio signal contributes significantly to lengthy discussion. A related topic to auditory illusions the sound ‘impact’, and depending on the bass quality, is the interaction between different sensory modalities, the overall sound quality will shift up or down. e.g. sound and vision, a famous example is the Therefore a good low-frequency reproduction is McGurk effect (‘Hearing lips and seeing voices’) [1]. essential. An auditory-visual overview is given in [2], a more general multisensory product perception in [3], and on ILLUSIONS spatial orientation in [4]. The influence of video quality An illusion is a distortion of a sensory perception, on perceived audio quality is discussed in [5].
    [Show full text]
  • City Research Online
    City Research Online City, University of London Institutional Repository Citation: Cross, I. (1989). The cognitive organisation of musical pitch. (Unpublished Doctoral thesis, City University London) This is the accepted version of the paper. This version of the publication may differ from the final published version. Permanent repository link: https://openaccess.city.ac.uk/id/eprint/7663/ Link to published version: Copyright: City Research Online aims to make research outputs of City, University of London available to a wider audience. Copyright and Moral Rights remain with the author(s) and/or copyright holders. URLs from City Research Online may be freely distributed and linked to. Reuse: Copies of full items can be used for personal research or study, educational, or not-for-profit purposes without prior permission or charge. Provided that the authors, title and full bibliographic details are credited, a hyperlink and/or URL is given for the original metadata page and the content is not changed in any way. City Research Online: http://openaccess.city.ac.uk/ [email protected] The Cognitive Organisation of Musical Pitch A dissertation submitted in fulfilment of the requirements of the degree of Doctor of Philosophy in The City University, London by Ian Cross September1989 ABSTRACT This thesis takes as its initial Premise the idea that the rationales for the forms of pitch organisation employed within tonal music which have been adopted by music theorists have strongly affected those theorists` conceptions of music, and that it is of critical importance to music theory to investigate the potential origination of such rationales within the human sciences.
    [Show full text]
  • The Octave Illusion and Auditory Perceptual Integration
    The Octave Illusion and Auditory Perceptual Integration DIANA DEUTSCH University of California, San Diego, La Jolla, California I. Introduction . 1 II. The Octave Illusion . 2 A. The Basic Effect . 2 B. Handedness Correlates . 4 C. Further Complexities: Ears or Auditory Space? . 6 D. Dependence of the Illusion on Sequential Interactions . 6 III. Parametric Studies of Ear Dominance . 7 A. Apparatus . 7 B. Experiment 1 . 7 C. Experiment 2 . 9 D. Experiment 3 . 10 E. Experiment 4 . .11 F. Hypothesized Basis for Ear Dominance . 13 G. Discussion . 13 IV. Parametric Studies of Lateralization by Frequency . .15 A. Experiment 1 . 15 B. Experiment 2. 16 C. Experiment 3 . 16 D. Experiment 4 . 16 E. Discussion . 18 V. The What–Where Connection . 18 Discussion . 19 VI. Conclusion . 19 References . 20 I. INTRODUCTION hemisphere damage to Brodmann’s areas 39 and 40 has been found to produce deficits in visual perceptual clas- A philosophical doctrine stemming from the empiri- sification (Warrington and Taylor, 1973). Further, vari- cists of the seventeenth and eighteenth centuries is that ous studies on human and subhuman species point to objects manifest themselves simply as bundles of attrib- an anatomical separation between the pathways medi- ute values. This doctrine has had a profound influence ating pattern discrimination on the one hand and local- on the thinking of sensory psychologists and neurophys- ization on the other (Ingle et al., 1967-1968). For exam- iologists. For example, it is assumed that when we see an ple, Schneider found that ablation of visual cortex in object, we separately appreciate its color, its shape, its hamsters led to an inability to discriminate visual pat- location in the visual field, and so on.
    [Show full text]
  • Music Perception 13
    Music Perception 13 LEARNING OBJECTIVES ISLE EXERCISES 13.1 Kurdish Music Example 13.1 Explain how frequency is related to pitch, chroma, and the octave. 13.2 Javanese Gamelan Music Example 13.2 Summarize the basic neuroscience of music, including how training and experience can affect the representation of music in the brain. 13.3 Ancient Greek Music 13.4 35,000-Year-Old Flute 13.3 Discuss how learning and culture affect music perception. distribute 13.5 Is This Music? 13.6 The Octave and or Tone Similarity INTRODUCTION 13.7 Pentatonic Music 13.8 Meter and Beat Wherever you travel, you will find music. It may sound very different from the music you are accustomed to hearing, but you will recognize it instantly as music. In Kurdistan, we 13.9 Bolero Clip find a unique culture of music featuring such instrumentspost, as the tanbur (a fretted string 13.10 Attack and Decay instrument), the qernête (a double-reed wind instrument), and the şimşal (a flutelike 13.11 Examples of Melody instrument) (Figure 13.1). Although most of you may never have heard of these instru- ments and may never have heard Kurdish music before, you would instantly recognize 13.12 Types of Scales them as musical instruments, and you might even like Kurdish music (see ISLE 13.1 for 13.13 Gestalt Principles an example of Kurdish music). Review copy, © Aurora Photos/Alamy 13.14 Gestalt Principle: Proximity: Bach’s Partita No. 3 in E major not 13.15 A Shave and a Haircut 13.16 Cross-Modal Matchings as a Simulation of Synesthesia Do 13.17.
    [Show full text]
  • A Criterion-Related Validity Test of Selected Indicators of Musical Sophistication Using Expert Ratings
    A CRITERION-RELATED VALIDITY TEST OF SELECTED INDICATORS OF MUSICAL SOPHISTICATION USING EXPERT RATINGS DISSERTATION Presented in Partial Fulfillment of the Requirements for The Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Joy E. Ollen, M.A. * * * * * The Ohio State University 2006 Dissertation Committee: Approved by Professor David Huron, Adviser Professor David Butler ____________________________________ Professor R. J. David Frego Adviser School of Music Professor Joseph Gliem Copyright by Joy Elaine Ollen 2006 ABSTRACT Music researchers regularly test the hypothesis that participants will respond differently based upon their levels of musical sophistication. Circumstances may not permit any extensive pre-testing and researchers often select or group participants according to their answers to simple survey-type questions related to their musical background. A number of these indicators have been criticized as being inadequate; yet, to date, their validity has remained untested. The goal of this dissertation was to test the validity of selected indicators, using expert ratings as the criterion variable, and to compile a short questionnaire that researchers may use to classify research participants as either more or less musically sophisticated. To that end, three survey projects were conducted. The first project sampled 743 music studies published between 1928 and 2004 in order to obtain information about how researchers were measuring musical sophistication. Two of the most commonly used indicators were “Formal musical training” (e.g., years of private lessons) and “Year or level in a formal music program” (e.g., music or non-music major). However, 38 different categories of indicators were identified in the sample and were used in 173 unique ways—either singly or in combination.
    [Show full text]
  • Chapter 8 the Perception of Auditory Patterns Diana Deutsch University of California, San Diego
    Chapter 8 The Perception of Auditory Patterns Diana Deutsch University of California, San Diego 1 INTRODUCTION Over the last half-century a considerable body of knowledge has accumulated concerning visual shape perception, based on findings in experimental psychology, neurophysiology, artificial intelligence and related fields. These developments have not, however, been paralleled by analogous developments in audition. It is inter- esting, therefore, that the foundations of pattern perception were laid by scientists who were as much concerned with auditory and musical phenomena as they were with visual ones. Wertheimer (1924/1938) described the beginnings of Gestalt theory thus: 'Historically, the most important impulse came from von Ehrenfels who raised the following problem. Psychology has said that experience is a compound of elements; we hear a melody and then, upon hearing it again, memory enables us to recognize it. But what is it that enables us to recognize the melody when it is played in a new key? The sum of its elements is different, yet the melody is the same; indeed, one is often not even aware that a transposition has been made.' (p. 4) In another paper in which he proposed the Gestalt principles of perceptual organization, Wertheimer (1923/1955) frequently presented musical illustrations along with visual ones. For example, in order to illustrate that 'more or less dissimilarity operates to determine experienced arrangement' he wrote: 'With tones, for example, C, C~, E, F, G~, A, C, C~... will be heard in the grouping ab/cd... ; and C, C~, D, E, F, F~, G~, A, A~, C, C~, D ... in the grouping abc/def' (1923/1955, p.
    [Show full text]
  • Cognitive Neuroscience of Music 1 Cognitive Neuroscience of Music
    Cognitive neuroscience of music 1 Cognitive neuroscience of music The cognitive neuroscience of music is the scientific study of brain-based mechanisms involved in the cognitive processes underlying music. These behaviours include music listening, performing, composing, reading, writing, and ancillary activities. It also is increasingly concerned with the brain basis for musical aesthetics and musical emotion. Scientists working in this field may have training in cognitive neuroscience, neurology, neuroanatomy, psychology, music theory, computer science, and other allied fields. Cognitive neuroscience of music is distinguished from related fields such as music psychology, music cognition and cognitive musicology in its reliance on direct observations of the brain, using such techniques as functional magnetic resonance imaging (fMRI), transcranial magnetic stimulation (TMS), magnetoencephalography (MEG), electroencephalography (EEG), and positron emission tomography (PET). Neurological Bases Pitch For each pitch there is a corresponding part of the tonotopically organized basilar membrane in the inner ear which responds to the sound and sends a signal to the auditory cortex. Sections of cells in the cortex are responsive to certain frequencies, which range from very low to very high in pitches.[1] This organization may not be stable and the specific cells that are responsive to different pitches may change over days or months.[2] Melody Processing in the Secondary Auditory Cortex Studies suggest that individuals are capable of automatically
    [Show full text]
  • A. Acoustic Theory and Modeling of the Vocal Tract
    A. Acoustic Theory and Modeling of the Vocal Tract by H.W. Strube, Drittes Physikalisches Institut, Universität Göttingen A.l Introduction This appendix is intended for those readers who want to inform themselves about the mathematical treatment of the vocal-tract acoustics and about its modeling in the time and frequency domains. Apart from providing a funda­ mental understanding, this is required for all applications and investigations concerned with the relationship between geometric and acoustic properties of the vocal tract, such as articulatory synthesis, determination of the tract shape from acoustic quantities, inverse filtering, etc. Historically, the formants of speech were conjectured to be resonances of cavities in the vocal tract. In the case of a narrow constriction at or near the lips, such as for the vowel [uJ, the volume of the tract can be considered a Helmholtz resonator (the glottis is assumed almost closed). However, this can only explain the first formant. Also, the constriction - if any - is usu­ ally situated farther back. Then the tract may be roughly approximated as a cascade of two resonators, accounting for two formants. But all these approx­ imations by discrete cavities proved unrealistic. Thus researchers have have now adopted a more reasonable description of the vocal tract as a nonuni­ form acoustical transmission line. This can explain an infinite number of res­ onances, of which, however, only the first 2 to 4 are of phonetic importance. Depending on the kind of sound, the tube system has different topology: • for vowel-like sounds, pharynx and mouth form one tube; • for nasalized vowels, the tube is branched, with transmission from pharynx through mouth and nose; • for nasal consonants, transmission is through pharynx and nose, with the closed mouth tract as a "shunt" line.
    [Show full text]
  • Essay Deutsch D
    1 Essay Deutsch D National Hearing Conservation Association Annual Conference , 2003, February, Dallas. Invited lecture. Invited Lay language paper presented at the 156th meeting of the Acoustical Society of America. Absolute pitch is associated with a large auditory digit span A clue to its genesis. Mozart Festival, Kennedy Center for the Performing Arts , 1998, Washington, D. 1983, November. Schiffman, Sensation and Perception. Heller and W. , and Dooley, K. Auditory Perception, Cognition and Action Meeting , 2002, November, Kansas City. Stereo TV, Invited presentation. Marks, The Unity of the Senses. Journal of the Acoustical Society of America , 1996, 99, 2482,. The pitch levels of female speech in two Chinese villages. , Sensation and Perception. Physics Today , 2010, February, 40-45,. Ear Club , 2004, May, Department of Psychology, University of California, Berkeley. Invited Lay language paper presented at the 148th meeting of the Acoustical Society of America, San Diego, November,. New York Oxford University Press. Mozart Festival, Kennedy Center for the Performing Arts , 1998, Washington, D. Basic Structure and Function in the Central Nervous System , 1974, New York MacMillan Co. Representation of pitch and pitch combinations. Invited Lay Language paper presented at the 149th Meeting of the Acoustical Society of America. and Deutsch, D. Musical space. Two issues at the interface between speech and music. Paper presented at the. Winter Conference on Brain Research , 1986, Keystone. International conference on Psychology and the Arts , 1983, Cardiff, Wales. Psychonomic Society Meeting , 1984, El Paso. Encyclopedia of Perception , 2009, 1, 160-164, Sage. Invited Presentation, Annual meeting of the. , Psychological Foundations of Musical Behavior. , and Dooley, K.
    [Show full text]
  • Why You Hear What You Hear
    Web: http://www.pearl-hifi.com 86008, 2106 33 Ave. SW, Calgary, AB; CAN T2T 1Z6 E-mail: [email protected] Ph: +.1.403.244.4434 Fx: +.1.403.245.4456 Inc. Perkins Electro-Acoustic Research Lab, Inc. ❦ Engineering and Intuition Serving the Soul of Music Please note that the links in the PEARL logotypegyp above are “live” and can be used to direct youry web browser to our site or to oopenpen an e-maie-maill message winwindowdow aaddressedddressed to ourseourselves.lves. To view our item listings on eBay, click here. To see the feedback we have left for our customers, click here. This document has been prepared as a public service . Any and all trademarks and logotypes used herein are the property of their owners. It is our intent to provide this document in accordance with the stipulations with respect to “fair use” as delineated in Copyrights - Chapter 1: Subject Matter and Scope of Copyright; Sec. 107. Limitations on exclusive rights: Fair Use. Public access to copy of this document is provided on the website of Cornell Law School at http://www4.law.cornell.edu/uscode/17/107.html and is here reproduced below: Sec. 107. - Limitations on exclusive rights: Fair Use Notwithstanding the provisions of sections 106 and 106A, the fair use of a copyrighted work, includ- ing such use by reproduction in copies or phono records or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for class- room use), scholarship, or research, is not an infringement of copyright.
    [Show full text]
  • Musical Illusions
    Musical Illusions Diana Deutsch Department of Psychology University of California, San Diego La Jolla, CA 92093 [email protected] In Squire, L. (Ed.) New Encyclopedia of Neuroscience, (Oxford, Elsevier, in press) Author information: Name and affiliation to appear in published work: Diana Deutsch, Professor of Psychology, University of California, San Diego Full address: Professor Diana Deutsch Department of Psychology University of California, San Diego 9500 Gilman Dr. #0109 La Jolla, CA 92093-0109, USA 2 keywords and phrases for indexing: glissando illusion handendness illusion individual differences language modularity music musical illusions octave illusion perception plasticity scale illusion speech tritone paradox 3 Synopsis Musical illusions are explored, and their implications for brain function are discussed. There are striking variations across listeners in perception of these illusions. For certain illusions, perceptual differences correlate with handedness, indicating that they reflect variations in innate brain organization. For another illusion, perceptual differences correlate with the language or dialect to which the listener was exposed, particularly early in life, so pointing to an influence of early exposure to speech sounds on how music is perceived. Implications of the illusions for modularity of function are discussed. 4 Musical Illusions D Deutsch, University of California, San Diego, La Jolla, CA, USA Introduction Over the last decade, there have been significant advances in understanding the brain mechanisms underlying music perception. Functional imaging studies of normal individuals have provided considerable information concerning the anatomical structures involved. Studies of patients with musical deficits resulting from brain lesions have supplied much complementary information, as have studies of individuals with congenital amusia. Investigations of musical illusions – misperceptions of music by normal individuals – provide further insights into the brain circuitry underlying music perception.
    [Show full text]
  • Psychoacoustic Manipulation of the Sound Induced Illusory Flash
    PSYCHOACOUSTIC MANIPULATION OF THE SOUND INDUCED ILLUSORY FLASH Sonia Wilkie Catherine Stevens Roger Dean [email protected] [email protected] [email protected] MARCS Auditory Laboratories University of Western Sydney Locked Bag 1797 Penrith South 1797 Sydney, Australia ABSTRACT auditory beeps. This illusory percept appears to occur Psychological research on cross-modal perception has because of the superior resolution of the auditory system focused on the manipulation of sensory information for rhythmic perception which, in this case, overrides predominantly by visual information. There is a lacuna visual information. in using auditory stimuli to manipulate other sensory The Sound Induced Illusory Flash is a recent discov- information. The Sound Induced Illusory Flash is one ery. Whilst recent research4 5 has focused on neural illusory paradigm that uses the auditory system to bias mechanisms that underpin the illusion, only the initial 6 7 8 other sensory information. However, more research is studies … … explore basic structural variables that give needed into the different conditions under which the rise to the illusion. It is these structural variables that Sound Induced Illusory Flash manifests and is en- are of interest for further exploration of the illusion hanced or reduced. experience. Such research outcomes provide new oppor- The experiment reported here investigates the effect tunities for creative application and multimedia trans- of new auditory variables on the Sound Induced Illusory mission techniques where auditory stimuli might influ- Flash. The variables to be discussed include the use of ence visual perception in novel and interesting ways. pitch intervals and harmonic relationships.
    [Show full text]