DOCSLIB.ORG

Audio Data Analysis

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Audio Data Analysis Audio data analysis CES Data Scientist Slim ESSID Audio Data Analysis and Signal Processing team [email protected] http://www.telecom-paristech.fr/~essid Credits O. GILLET, C. JODER, N. MOREAU, G. RICHARD, F. VALLET, … Slim Essid About “audio”… ►Audio frequency: the range of audible frequencies (20 to 20,000 Hz) Threshold of pain Audible sound Minimal audition threshold CC Attribution 2.5 Generic Frequencies (kHz) 2 CES Data Science – Audio data analysis Slim Essid About “audio”… ►Audio content categories To ad -minis -ter medi Speech Music Environmental 3 CES Data Science – Audio data analysis Slim Essid About “audio”… ►An important distinction: speech vs non-speech Speech signals Music & non-speech (environmental) “Simple” production model: No generic production model: the source-filter model “timbre”, “pitch”, “loudness”, … Image: Edward Flemming, course materials for 24.910 Topics in Linguistic Theory: Laboratory Phonology, Spring 2007. MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology. Downloaded on 05 May 2012 4 CES Data Science – Audio data analysis Slim Essid About “audio”… ►Different research communities Music Information Speech Research Signal representations Music Speech classification recognition (genre, mood, …) Audio coding Speaker Transcription Source recognition separation Speech Rhythm Sound enhancement analysis synthesis … … … Machine Listening / Computer audition 5 CES Data Science – Audio data analysis Slim Essid About “audio”… ►Research fields Acoustics Linguistics Psychology Psychoacoustics Audio content Musicology analysis Signal Knowledge processing engineering Machine learning Databases Statistics 6 CES Data Science – Audio data analysis Slim Essid About “audio”… ►Research fields Acoustics Linguistics Psychology Psychoacoustics Audio content Musicology analysis Signal Knowledge processing engineering Machine learning Databases Statistics 7 CES Data Science – Audio data analysis Slim Essid Why analyse audio data? . For archive management, indexing » Broadcast content segmentation and classification: speech/music/jingles…, speakers » Music autotagging: genre (classical, jazz, rock,…), mood, usage… » Search engines . For broadcasters » Music/effects/speech excerpt search » Playlist generation, Djing 9 CES Data Science – Audio data analysis Slim Essid Why analyse audio data? . For designers and producers » Audio sample search » Music transcription (beat, rhythm, chords, notes) » Broadcast content monitoring, plagiarism detection, hit prediction . For end-users » Content-based search (shazam++) » Non-linear and interactive content consuming (“skip intro”, “replay the chorus”, Karaoke: “remove the vocals”…) » Recommendation » Personalised playlist generation 10 CES Data Science – Audio data analysis Slim Essid Audio-driven multimedia analysis . Motivation for audio-driven content analysis » critical information is conveyed by the audio content » audio and visual information play complementary roles for the detection of key concepts/events . Video examples 11 CES Data Science – Audio data analysis Slim Essid Audio-driven multimedia analysis ►Video examples Use audio-based laughter detection • Applause detection • Keyword spotting: “Goal!” • Cheering detection • Sound loudness • Applause/cheering detection 12 CES Data Science – Audio data analysis Slim Essid Audio-driven multimedia analysis ►Key audio-based components Speech activity Jingle Emotion detection detection recognition Speech/Music/ Applause/Laughter/ Speaker Social signal … identification processing: detection roles, interactions,… Speaker diarization: “Who spoke when?” Speech-to-text transcription Natural Music language classification processing genre, mood, … At the heart of all components: a classification task (supervised or unsupervised) 13 CES Data Science – Audio data analysis Slim Essid General classification architecture ►Overview Development Audio segment Feature vectors database Feature Classifier extraction training TRAINING/DEVELOPMENT PHASE (AT THE LAB.) Decision functions Testing database Feature extraction Classification Segment identified TESTING/EVALUATION PHASE (EXPLOITATION) 15 CES Data Science – Audio data analysis Slim Essid Classification architecture ►Feature extraction process Feature Temporal Preprocessing computation integration Motivation: Exercise In Python: . signal denoising/enhancement . information rate reduction, eg. subsampling - load an audio file; - normalise it; . normalisation, eg.: - visualise it. Use librosa 16 CES Data Science – Audio data analysis Slim Essid Classification architecture ►Feature extraction process Feature Temporal Preprocessing computation integration Relies on audio signal processing techniques 17 CES Data Science – Audio data analysis Slim Essid Audio signal analysis ►Short-Term analysis windows Drawing by J. Laroche, modified 18 CES Data Science – Audio data analysis Slim Essid Signal framing Attack Sustain Release N H m m+1 m+2 » Static temporal segmentation » Dynamic temporal segmentation 19 CES Data Science – Audio data analysis Slim Essid Feature types . Temporal features: extracted directly from the waveform samples . Spectral features: extracted from a frequential representation of the signal . Perceptual features: extracted using a perceptual representation based on psychoacoustic considerations 20 CES Data Science – Audio data analysis Slim Essid Temporal features - ZCR Zero Crossing Rates Characterises noisy and transient sections 21 Spectral analysis ►Discrete Fourier Transform In practice: computed using the Fast Fourier Transform (FFT) 23 CES Data Science – Audio data analysis Slim Essid Discrete Fourier Transform (DFT) ►Important properties . Being a discrete time Fourier Transform, the DFT is periodic, with 푓 period 1 (in reduced frequency 푓 = ; 푓푠 : sampling frequency) 푓푠 . For signals 푥(푛) and 푦 푛 ; 푛 ∈ {0, … , 푁 − 1} Property Numerical series DFT Linearity {푎푥 푛 + 푏푦 푛 } {푎푋 푘 + 푏푌 푘 } ∗ Hermitian symmetry 푥 푛 real 푋 푘 = 푋 (−푘) 2푗휋푘 Time translation 푥(푛 − 푛0) − 푛0 푋 푘 푒 푁 Convolution 푥 푛 ⋆ 푦 푛 푋 푘 푌(푘) ≜ 푥 푘 푦(푛 − 푘) 푘 ∗ ∗ Conjugation {푥 푛 } {푋 −푘 } 24 CES Data Science – Audio data analysis Slim Essid Spectral analysis ►Spectral analysis by Short-Term Fourier Transform (STFT) Drawing by J. Laroche 25 CES Data Science – Audio data analysis Slim Essid Spectral analysis ►Violin excerpt: 20-ms overlapping windows ( 푠푟 = 44.1kHz ; 푁 = 882 samples ) 26 CES Data Science – Audio data analysis Slim Essid Spectral analysis ►Spectrogram C note (262 Hz) produced by a piano and a violin Temporal signal Spectrogram From M. Mueller & al. « Signal Processing for Music Analysis, IEEE Trans. On Selected topics in Signal Processing », October 2011. 27 CES Data Science – Audio data analysis Slim Essid Spectral analysis ►Exercise In Python: . Compute short-term spectra of an audio signal using FTT . Compute and display spectrogram . Use » scipy.fftpack » librosa 28 CES Data Science – Audio data analysis Slim Essid Spectral analysis ►Limitations of the spectrogram representation . Large representation » Typically 512 coefs every 10 ms » High dimensionality . Much detail » Redundant representation » High-level features (pitch, vibrato, timbre) are not highlighted Still a low-level representation, not yet a model 29 CES Data Science – Audio data analysis Slim Essid The source-filter model . Distinction between: » source: excitation fine spectral structure » filter: resonator coarse structure Source signal Speech (Vocal folds) Resonator (Vocal tract) Filter X(f) H(f) Y(f) 30 CES Data Science – Audio data analysis Slim Essid Cepstrum ►Principle . Source-filter model: . In the frequency domain: . By inverse DFT: where : real cepstrum definition deconvolution is thus achieved: filter is separated from excitation . First few cepstral coefficients » low quefrency: “slow iDFT waves” » represent the filter spectral envelope . Next coefficients represent the source fine spectral structure 31 CES Data Science – Audio data analysis Slim Essid Cepstral representations ►MFCC: Mel Frequency Cepstral Coefficients Audio frame Mel scale DFT Magnitude spectrum Triangular filter banc in Mel scale 32 CES Data Science – Audio data analysis Slim Essid Cepstral representations ►MFCC: Mel Frequency Cepstral Coefficients Audio frame DFT Magnitude spectrum Triangular filter banc in Mel scale 33 CES Data Science – Audio data analysis Slim Essid Cepstral representations ►MFCC: Mel Frequency Cepstral Coefficients Audio frame DFT Discrete Cosine Transform: • nice decorrelation properties Magnitude spectrum (like PCA) • yields diagonal covariance Triangular filter banc matrices in Mel scale Log DCT 13 first coefs (in general) 34 CES Data Science – Audio data analysis Slim Essid Cepstral representations ►MFCC: Mel Frequency Cepstral Coefficients Audio frame DFT First and second derivatives: speed and acceleration Magnitude spectrum Coarse temporal modelling Triangular filter banc in Mel scale Log DCT 39-coefficient 13 first coefs dt feature vector (in general) dt² 35 CES Data Science – Audio data analysis Slim Essid About MFCCs ►… very popular! . In speech applications: » Well justified: source-filter model makes sense » Nice properties from a statistical modelling viewpoint: decorrelation » Effective: state-of-the-art features for speaker and speech tasks . In general audio classification: » “Source-filter” model does not always hold » Still, MFCCs work well in practice! they are the default choice 36 CES Data Science – Audio data analysis Slim Essid MFCC ►Exercise
Load more

Recommended publications
  • Automatic Music Transcription: an Overview Emmanouil Benetos Member, IEEE, Simon Dixon, Zhiyao Duan Member, IEEE, and Sebastian Ewert Member, IEEE
    1 Automatic Music Transcription: An Overview Emmanouil Benetos Member, IEEE, Simon Dixon, Zhiyao Duan Member, IEEE, and Sebastian Ewert Member, IEEE I. INTRODUCTION IV-F, as well as methods for transcribing specific sources The capability of transcribing music audio into music within a polyphonic mixture such as melody and bass line. notation is a fascinating example of human intelligence. It involves perception (analyzing complex auditory scenes), cog- A. Applications & Impact nition (recognizing musical objects), knowledge representation A successful AMT system would enable a broad range (forming musical structures) and inference (testing alternative of interactions between people and music, including music hypotheses). Automatic Music Transcription (AMT), i.e., the education (e.g., through systems for automatic instrument design of computational algorithms to convert acoustic music tutoring), music creation (e.g., dictating improvised musical signals into some form of music notation, is a challenging task ideas and automatic music accompaniment), music production in signal processing and artificial intelligence. It comprises (e.g., music content visualization and intelligent content-based several subtasks, including (multi-)pitch estimation, onset and editing), music search (e.g., indexing and recommendation of offset detection, instrument recognition, beat and rhythm track- music by melody, bass, rhythm or chord progression), and ing, interpretation of expressive timing and dynamics, and musicology (e.g., analyzing jazz improvisations and other non- score typesetting. Given the number of subtasks it comprises notated music). As such, AMT is an enabling technology with and its wide application range, it is considered a fundamental clear potential for both economic and societal impact. problem in the fields of music signal processing and music AMT is closely related to other music signal processing information retrieval (MIR) [1], [2].
    [Show full text]
  • Real-Time Programming and Processing of Music Signals Arshia Cont
    Real-time Programming and Processing of Music Signals Arshia Cont To cite this version: Arshia Cont. Real-time Programming and Processing of Music Signals. Sound [cs.SD]. Université Pierre et Marie Curie - Paris VI, 2013. tel-00829771 HAL Id: tel-00829771 https://tel.archives-ouvertes.fr/tel-00829771 Submitted on 3 Jun 2013 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Realtime Programming & Processing of Music Signals by ARSHIA CONT Ircam-CNRS-UPMC Mixed Research Unit MuTant Team-Project (INRIA) Musical Representations Team, Ircam-Centre Pompidou 1 Place Igor Stravinsky, 75004 Paris, France. Habilitation à diriger la recherche Defended on May 30th in front of the jury composed of: Gérard Berry Collège de France Professor Roger Dannanberg Carnegie Mellon University Professor Carlos Agon UPMC - Ircam Professor François Pachet Sony CSL Senior Researcher Miller Puckette UCSD Professor Marco Stroppa Composer ii à Marie le sel de ma vie iv CONTENTS 1. Introduction1 1.1. Synthetic Summary .................. 1 1.2. Publication List 2007-2012 ................ 3 1.3. Research Advising Summary ............... 5 2. Realtime Machine Listening7 2.1. Automatic Transcription................. 7 2.2. Automatic Alignment .................. 10 2.2.1.
    [Show full text]
  • 2007–09 Program Requirements (.Pdf)
    1 FOREWORD F ROM THE DEAN http://www.yorku.ca/grads/calendar/ Graduate study involves a level of engagement with subject matter, in Business, Law, Education, Translation and Social Work and in fellow students, and faculty members that marks a high point in health-related disciplines focused through York’s new Faculty of one’s intellectual and creative development. At the master’s and Health. Innovative and unique interdisciplinary programs have Doctoral levels, graduate study in one way or another is at the centre been created in such areas as Environmental Studies, Earth & Space of research and scholarly intensity within the University and provides Science, Social & Political Thought, Interdisciplinary Studies, exciting challenges and opportunities. Women’s Studies, and our most recent programs: Humanities, Human Resources Management, and Critical Studies in Disability. Since its inception in 1963, the Faculty of Graduate Studies has A further innovative dimension has involved the creation of a grown from 11 students in a single graduate program to more number of specialized graduate diplomas—such as Early Childhood than 5000 students in 46 programs. York’s graduate studies are Education, and Environmental/Sustainability Education—which expanding, with five new graduate programs in development, three may be earned concurrently with the master’s or Doctoral degree of which begin this year; 11 more programs are expanding, either in several programs, and which may also be taken as stand-alone adding a doctoral program where there is an existing master’s, graduate diplomas. York offers 32 graduate diplomas. The Faculty or adding new fields or different master’s programs.
    [Show full text]
  • Music Similarity: Learning Algorithms and Applications
    UNIVERSITY OF CALIFORNIA, SAN DIEGO More like this: machine learning approaches to music similarity A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Computer Science by Brian McFee Committee in charge: Professor Sanjoy Dasgupta, Co-Chair Professor Gert Lanckriet, Co-Chair Professor Serge Belongie Professor Lawrence Saul Professor Nuno Vasconcelos 2012 Copyright Brian McFee, 2012 All rights reserved. The dissertation of Brian McFee is approved, and it is ac- ceptable in quality and form for publication on microfilm and electronically: Co-Chair Co-Chair University of California, San Diego 2012 iii DEDICATION To my parents. Thanks for the genes, and everything since. iv EPIGRAPH I’m gonna hear my favorite song, if it takes all night.1 Frank Black, “If It Takes All Night.” 1Clearly, the author is lamenting the inefficiencies of broadcast radio programming. v TABLE OF CONTENTS Signature Page................................... iii Dedication...................................... iv Epigraph.......................................v Table of Contents.................................. vi List of Figures....................................x List of Tables.................................... xi Acknowledgements................................. xii Vita......................................... xiv Abstract of the Dissertation............................. xvi Chapter 1 Introduction.............................1 1.1 Music information retrieval..................1 1.2 Summary of contributions..................1
    [Show full text]
  • 2021 Finalist Directory
    2021 Finalist Directory April 29, 2021 ANIMAL SCIENCES ANIM001 Shrimply Clean: Effects of Mussels and Prawn on Water Quality https://projectboard.world/isef/project/51706 Trinity Skaggs, 11th; Wildwood High School, Wildwood, FL ANIM003 Investigation on High Twinning Rates in Cattle Using Sanger Sequencing https://projectboard.world/isef/project/51833 Lilly Figueroa, 10th; Mancos High School, Mancos, CO ANIM004 Utilization of Mechanically Simulated Kangaroo Care as a Novel Homeostatic Method to Treat Mice Carrying a Remutation of the Ppp1r13l Gene as a Model for Humans with Cardiomyopathy https://projectboard.world/isef/project/51789 Nathan Foo, 12th; West Shore Junior/Senior High School, Melbourne, FL ANIM005T Behavior Study and Development of Artificial Nest for Nurturing Assassin Bugs (Sycanus indagator Stal.) Beneficial in Biological Pest Control https://projectboard.world/isef/project/51803 Nonthaporn Srikha, 10th; Natthida Benjapiyaporn, 11th; Pattarapoom Tubtim, 12th; The Demonstration School of Khon Kaen University (Modindaeng), Muang Khonkaen, Khonkaen, Thailand ANIM006 The Survival of the Fairy: An In-Depth Survey into the Behavior and Life Cycle of the Sand Fairy Cicada, Year 3 https://projectboard.world/isef/project/51630 Antonio Rajaratnam, 12th; Redeemer Baptist School, North Parramatta, NSW, Australia ANIM007 Novel Geotaxic Data Show Botanical Therapeutics Slow Parkinson’s Disease in A53T and ParkinKO Models https://projectboard.world/isef/project/51887 Kristi Biswas, 10th; Paxon School for Advanced Studies, Jacksonville,
    [Show full text]
  • Data-Driven Audio Recognition: a Supervised Dictionary Approach
    DATA-DRIVEN AUDIO RECOGNITION: A SUPERVISED DICTIONARY APPROACH APREPRINT Imad Rida Laboratoire BMBI Compiègne Université de Technologie de Compiègne Compiègne, France 2021-01-01 ABSTRACT Machine hearing or listening represents an emerging area. Conventional approaches rely on the design of handcrafted features specialized to a specific audio task and that can hardly generalized to other audio fields. Unfortunately, these predefined features may be of variable discrimination power while extended to other tasks or even within the same task due to different nature of clips. Motivated by this need of a principled framework across domain applications for machine listening, we propose a generic and data-driven representation learning approach. For this sake, a novel and efficient supervised dictionary learning method is presented. Experiments are performed on both computational auditory scene (East Anglia and Rouen) and synthetic music chord recognition datasets. Obtained results show that our method is capable to reach state-of-the-art hand-crafted features for both applications arXiv:2012.14761v1 [cs.SD] 29 Dec 2020 Keywords Audio · Dictionary learning · Music · Scene Humans have a very high perception capability through physical sensation, which can include sensory input from the eyes, ears, nose, tongue, or skin. A lot of efforts have been devoted to develop intelligent computer systems capable to interpret data in a similar manner to the way humans use their senses to relate to the world around them. While most efforts have focused on vision perception which represents the dominant sense in humans, machine hearing also known as machine listening or computer audition represents an emerging area [1].
    [Show full text]
  • Interpretable Machine Learning
    SYM POS IUM28. + 29. NOV 2019 KLOSTER SANKT JOSEF 2. Netzwerkkongress der ZD.B-Initiativen für die Wissenschaft ABSTRACTBAND PROGRAMM SEITE 4 SNAPSHOTS SEITE 14 ABSTRACTS SEITE 52 2 | PROGRAMM PROGRAMM | 3 PROGRAMM 09:00 UHR Ankunft im Kloster St. Josef in Neumarkt 08:45 UHR Begrüßung TAG 10:15 UHR Eröffnung des Symposiums TAG 09:00 UHR Blitz-Intro für Postersession C Begrüßung durch das Zentrum Digitalisierung.Bayern Postersession C 10:45 UHR Keynote – Multi-Inter-Trans!? Zusammenarbeiten jenseits der Disziplin 09:15 UHR Prof. Dr. Ruth Müller C1 C2 C3 Munich Center for Technology in Society 01DO. 28. NOV. 2019 Technische Universität München 02FR. 29. NOV. 2019 11:30 UHR LET’S TALK ABOUT: Interdisziplinarität in der wissenschaftlichen Praxis 10:15 UHR Kaffeepause Prof. Dr. Oliver Amft – Universität Erlangen-Nürnberg LET’S TALK ABOUT: Innovation und Impact – Welche Rolle spielt die Dr. Jörg Haßler – LMU München 10:45 UHR Wissenschaft beim digitalen Fortschritt? Prof. Dr. Nicholas Müller – HAW Würzburg-Schweinfurth Prof. Dr. Andreas Festag – TH Ingolstadt Prof. Dr. Ruth Müller – TU München Prof. Dr. Albrecht Schmidt – LMU München Prof. Andreas Muxel – HAW Augsburg Prof. Dr. Björn Schuller – Universität Augsburg Prof. Dr. Eva Rothgang – OTH Amberg-Weiden Prof. Dr. Ramin Tavakoli Kolagari – TH Nürnberg Prof. Dr. Verena Tiefenbeck – Universität Erlangen-Nürnberg 12:30 UHR Mittagspause 11:45 UHR TREFFEN DER PHD MEET-UP 13:30 UHR Gruppenfoto ZD.B-ARBEITSKREISE Austausch der Austausch der Blitz-Intro für Postersession A 13:45 UHR ZD.B-Professor*innen & ZD.B-Doktorand*innen Nachwuchsgruppen- Postersession A 14:00 UHR leiter*innen A1 A2 A3 A4 12:30 UHR Mittagspause 15:00 UHR FELLOWS COACHING D OPEN SPACE I 13:30 UHR FELLOWS COACHING E OPEN SPACE II 15:45 UHR Kaffeepause 14:15 UHR Keynote – Open Science – warum & wie machen wir das? Dr.
    [Show full text]
  • Statement of Research Interests Sumit Basu
    Statement of Research Interests Sumit Basu sbasu@ sumitbasu.net http://www.media.mit.edu/~sbasu Post-Doctoral Researcher, Microsoft Research PhD (September 2002), MIT Department of Electrical Engineering and Computer Science Thesis Advisor: Professor Alex (Sandy) Pentland, MIT Media Laboratory My research objective is simple: I want to play with audio. I want to take auditory streams from the world, explore them, search through them, cut them apart, extract information from them, filter them, morph them, change them, and play them back into the ether. I want to find new and better ways to do these things and more, then teach them to others, both students and colleagues. I want to make audio useful and fun for a broad variety of communities. I want to build a myriad of interfaces, personal monitoring mechanisms, professional audio tools, toys, and instruments using these methods. In short, I want to do for audio what my colleagues in the computer vision and computer graphics communities have done with images and video. Over the years, I have worked in human-computer interfaces, computer vision/graphics, signal processing, statistical modeling/machine learning, and of course computer audition. W hen I began my graduate studies, I was initially drawn to computer vision and Sandy Pentland‘s group at the MIT Media Lab because of their strong sense of play œ they were extracting interesting meta-information from visual streams, such as the location of a user‘s head and hands, and using it for interactive applications like playing with a virtual dog. I joined their efforts and spent several years working on computer vision and interactive vision systems.
    [Show full text]
  • Tzanetakis, George Curriculum Vitae (Updated July 2016) 1
    Tzanetakis, George Curriculum Vitae (updated July 2016) 1 Research Interests Music Information Retrieval, Audio Signal Processing, Machine Learning, Human Computer Interaction, Digital Libraries, Software Frameworks for Audio Processing, Auditory Scene Analysis Education • Ph.D in Computer Science, Princeton University 2002 Manipulation, Analysis and Retrieval Systems for Audio Signals Advisor: Perry Cook • MA in Computer Science, Princeton University 1999 • BSE Computer Science, Magna Cum Laude, University of Crete, Greece 1997 • Music Education – Music Theory and Composition - Music Department, Princeton University (10 courses while doing PhD in Computer Science) (1997-2001) – Musicology, saxophone performance, and theory Athenaum Conservatory, Athens, Greece (1993-1997) – Piano and theory - Heraklion Conservatory, Greece (1985-1995) Professional Employment History • 2010-present: Associate Professor of Computer Science, University of Victoria, BC Canada (also cross-listed in Music, and in Electrical and Computer Engineering) • 2011 (6 months): Visiting Scientist, Google Research, Mountain View, California Collaboratos: Dick Lyon, Douglas Eck, Jay Yagnik, David Ross, Tom Walters • 2003-2010: Assistant Professor of Computer Science, University of Victoria, BC Canada (also cross-listed in Music, and in Electrical and Computer Engineering) • 2002-2003: Postdoctoral Fellow, Computer Science, Carnegie Mellon University (Computer Music Group, Informedia Group) Collaborators: R.Dannenberg, C.Atkenson, A.Hauptmann, H.Wactlar, and C. Faloutsos
    [Show full text]
  • Semantic Annotation of Music Collections: a Computational Approach
    Semantic Annotation of Music Collections: A Computational Approach Mohamed Sordo TESI DOCTORAL UPF / 2011 Directors de la tesi: Dr. Xavier Serra i Casals Dept. of Information and Communication Technologies Universitat Pompeu Fabra, Barcelona, Spain Dr. Òscar Celma i Herrada Gracenote, Emeryville, CA, USA Copyright c Mohamed Sordo, 2011. Dissertation submitted to the Department of Information and Communication Technologies of Universitat Pompeu Fabra in partial fulfillment of the require- ments for the degree of DOCTOR PER LA UNIVERSITAT POMPEU FABRA, with the mention of European Doctor. Music Technology Group (http://mtg.upf.edu), Dept. of Information and Communica- tion Technologies (http://www.upf.edu/dtic), Universitat Pompeu Fabra (http://www. upf.edu), Barcelona, Spain. A Radia, Idris y Randa. Me siento muy orgulloso de ser vuestro hijo y tu hermano. A toda mi familia. Acknowledgements During these last few years, I had the luck to work with an amazing group of people at the Music Technology Group. First and foremost, I would specially like to thank 3 people regarding this dissertation. Xavier Serra, for giving me the opportunity to join the Music Technology Group, and for his wise advices in key moments of the thesis work. Òscar Celma, for being the perfect co- supervisor a post–graduate student can have. Whether it was for guidance or for publishing, he was always there. Fabien Gouyon, who would have been without any doubt the third supervisor of this thesis. I especially thank him for giving me the opportunity to join his research group in the wonderful city of Porto, as a research stay.
    [Show full text]
  • ARVR Presentations
    Mark F. Bocko | Professor Department: Electrical and Computer Engineering Focus: Spatial audio Pilot Project: “Development of a quantitative framework for spatial audio characterization” Project Goals • Develop quantitative methods to assess spatial audio rendering systems • Incorporate quantitative binaural hearing models into audio system design tools • Predict what listeners will report hearing (locations, spatial extent of sources, diffusiveness) October 1, 2018 Geunyoung Yoon | Professor Department: Ophthalmology, The Institute of Optics, Center for Visual Science, Biomedical Engineering Focus: Physiological Optics, Vision Correction, Visual Psychophysics, Optical Imaging, Biomechanics, Eye Diseases Lab website: http://www.cvs.rochester.edu/yoonlab/ RESEARCH TOPICS: OCULAR OPTICS & CUSTOMIZED ACOOMMODATION & PRESBYOPIA VISION CORRECTION • Vergence-Accommodation conflict • Eye’s aberration and visual quality under VR/AR environments • Ocular wavefront sensing • Extended depth of focus technology • Advanced ophthalmic lenses • Accommodating intraocular lens • Sport vision • Peripheral vision and optics • Optical metrology • Binocular accommodation • Emmetropization / Refractive error OCULAR OPTICS and VISION ANTERIOR SEGMENT IMAGING • Adaptive optics vision simulator • Mechanisms of pathologic cornea • Adaptation to habitual optics diseases • Neural processing and perception • Ocular surface diseases and dry eye • Binocular integration • Corneal biomechanics • Neural plasticity • Multimodal high-resolution ocular • Stereopsis imaging
    [Show full text]
  • Computational Auditory Scene Induction
    NORTHWESTERN UNIVERSITY Computational Auditory Scene Induction A DISSERTATION SUBMITTED TO THE GRADUATE SCHOOL AND THE DEPARTMENT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE OF NORTHWESTERN UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS for the degree DOCTOR OF PHILOSOPHY Field of Computer Science By Jinyu Han EVANSTON, ILLINOIS August 2012 2 c Copyright by Jinyu Han 2012 All Rights Reserved 3 ABSTRACT Computational Auditory Scene Induction Jinyu Han Real world sound is a mixture of different sources. The sound scene of a busy cof- feehouse, for example, usually consists of several conversations, music playing, laughter and maybe a baby crying, the door being slammed, different machines operating in the background and more. When humans are confronted with these sounds, they rapidly and automatically adjust themselves in this complex sound environment, paying attention to the sound source of interest. This ability has been labeled in psychoacoustics under the name of Auditory Scene Analysis (ASA). The counterpart to ASA in machine listening is called Computational Auditory Scene Analysis (CASA) - the efforts to build computer models to perform auditory scene anal- ysis. Research on CASA has led to great advancement in machine systems capable of analyzing complex sound scene, such as audio source separation and multiple pitch esti- mation. Such systems often fail to perform in presence of corrupted or incomplete sound scenes. In a real world sound scene, different sounds overlap in time and frequency, in- terfering with and canceling each other. Sometimes, the sound of interest may have some 4 critical information totally missing, examples including an old recording from a scratched CD or a band-limited telephone speech signal.
    [Show full text]