DOCSLIB.ORG

Robust In-Car Speech Recognition Based on Nonlinear Multiple Regressions

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Robust In-Car Speech Recognition Based on Nonlinear Multiple Regressions Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2007, Article ID 16921, 10 pages doi:10.1155/2007/16921 Research Article Robust In-Car Speech Recognition Based on Nonlinear Multiple Regressions Weifeng Li,1 Kazuya Takeda,1 and Fumitada Itakura2 1 Graduate School of Information Science, Nagoya University, Nagoya 464-8603, Japan 2 Department of Information Engineering, Faculty of Science and Technology, Meijo University, Nagoya 468-8502, Japan Received 31 January 2006; Revised 10 August 2006; Accepted 29 October 2006 Recommended by S. Parthasarathy We address issues for improving handsfree speech recognition performance in different car environments using a single distant microphone. In this paper, we propose a nonlinear multiple-regression-based enhancement method for in-car speech recogni- tion. In order to develop a data-driven in-car recognition system, we develop an effective algorithm for adapting the regression parameters to different driving conditions. We also devise the model compensation scheme by synthesizing the training data using the optimal regression parameters and by selecting the optimal HMM for the test speech. Based on isolated word recognition experiments conducted in 15 real car environments, the proposed adaptive regression approach shows an advantage in average relative word error rate (WER) reductions of 52.5% and 14.8%, compared to original noisy speech and ETSI advanced front end, respectively. Copyright © 2007 Weifeng Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 1. INTRODUCTION ization (CMN) [8], codeword-dependent cesptral normal- ization (CDCN) [9], and so on. Spectral subtraction was The mismatch between training and testing conditions is originally proposed in the context of the enhancement of one of the most challenging and important problems in au- speech quality, but it can be used as a preprocessing step tomatic speech recognition (ASR). This mismatch may be for recognition. However, its performance suffers from the caused by a number of factors, such as background noise, annoying “musical tone” artifacts. CMN performs the sim- speaker variation, a change in speaking styles, channel effects, ple linear transformation and aims to remove the cep- and so on. State-of-the-art ASR techniques for removing the stral bias. Although effective for the convolutional distor- mismatch usually fall into the following three categories [1]: tions, this technique is not successful for the additive noise. robust features, speech enhancement, and model compensa- CDCN may be somewhat intensive to compute since it de- tion. The first approach seeks parameterizations that are fun- pends on the online estimation of the channel and additive damentally immune to noise. The most widely used speech noise through an iterative EM approach. Model compen- recognition features are the Mel-frequency cepstral coeffi- sation approach aims to adapt or transform acoustic mod- cients (MFCCs) [2]. MFCC’s lack of robustness in noisy or els to match the noisy speech feature in a new testing en- mismatched conditions has led many researchers to inves- vironment. The representative methods include multistyle tigate robust variants or novel feature extraction algorithm. training [8], maximum-likelihood linear regression (MLLR) Some of these researches could be perceptually based on, for [10], and Jacobian adaptation [11, 12]. Their main disad- example, the PLP [3]andRASTA[4], while other approaches vantage is that they require the retraining of a recognizer are related to the auditory processing, for example, gamma- or adaptation data, which leads to much higher complex- tone filter [5]andEIHmodel[6]. ity than speech enhancement approach. Most speech en- Speech enhancement approach aims to perform noise hancement and model compensation methods are accom- reduction by transforming noisy speech (or feature) into plished by linear functions such as simple bias removal, an estimate that more closely resembles clean speech (or affine transformation, linear regression, and so on. How- feature). Examples falling in this approach include spec- ever, it is well known that distortion caused even by ad- tral subtraction [7], Wiener filter, cepstral mean normal- ditive noise only is highly nonlinear in the log-spectral or 2 EURASIP Journal on Advances in Signal Processing cepstral domain. Therefore, a nonlinear transformation or 7 9101112 compensation is more appropriate. The use of a neural network allows us to automatically 5 6 learn the nonlinear mapping functions between the refer- ence and testing environments. Such a network can han- 1 dle additive noise, reverberation, channel mismatches, and 34 combinations of these. Neural-network-based feature en- hancement has been used in conjunction with a speech recognizer. For example, Sorensen used a multilayer net- work for noise reduction in the isolated word recogni- tion under F-16 jet noise [13]. Yuk and Flanagan em- ployed neural networks to perform telephone speech recog- nition [14]. However, the feature enhancement they im- 5 3 plemented was performed in the ceptral domain and the 9 clean features were estimated using the noisy features 10 only. 7 In previous work, we proposed a new and effective 11 multimicrophone speech enhancement approach based on 12 4 multiple regressions of log spectra [15] that used multi- 1 6 ple spatially distributed microphones. Their idea is to ap- proximate the log spectra of a close-talking microphone ff by e ectively the combining of the log spectra of dis- Figure 1: Side view (top) and top view (bottom) of the arrangement tant microphones. In this paper, we extend the idea to of multiple spatially distributed microphones and the linear array in single-microphone case and propose that the log spec- the data collection vehicle. tra of clean speech are approximated through the nonlin- ear regressions of the log spectra of the observed noisy speech and the estimated noise using a multilayer percep- tron (MLP) neural network. Our neural-network-based fea- ture enhancement method incorporates the noise informa- Five spatially distributed microphones (#3 to #7) are placed tion and can be viewed as a generalized log spectral subtrac- around the driver. Among them, microphone #6, located at tion. the visor location to the speaker (driver), is the closest to In order to develop a data-driven in-car recognition sys- the speaker. The speech recorded at this microphone (also tem, we develop an effective algorithm for adapting the re- named “visor mic.”) is used for speech recognition in this gression parameters to different driving conditions. In order paper. A four-element linear microphone array (#9 to #12) to further reduce the mismatch between training and testing with an interelement spacing of 5 cm is located at the visor conditions, we synthesize the training data using the optimal position. regression parameters, and train multiple hidden Markov The test data includes Japanese 50 word sets under 15 driving conditions (3 driving environments ×5 in-car states models (HMMs) over the synthesized data. We also develop = several HMM selection strategies. The devised system results 15 driving conditions as listed in Table 1). Table 2 shows in a universal in-car speech recognition framework including the average signal-to-noise ratio (SNR) for each driving con- both the speech enhancement and the model compensation. dition. For each driving condition, 50 words are uttered by The organization of this paper is as follows: in Section 2, each of 18 speakers. A total of 7000 phonetically balanced we describe the in-car speech corpus used in this paper. In sentences (uttered by 202 male speakers and 91 female speak- Section 3, we present the regression-based feature enhance- ers) were recorded for acoustical modeling. (3600 of them ment algorithm, and the experimental evaluations are out- were collected in the idling-normal condition and 3400 of lined in Section 4.InSection 5, we present the environmen- them were collected while driving the DCV on the streets tal adaptation and model compensation algorithms. Then near Nagoya University (city-normal condition).) the performance evaluation on the adaptive regression-based Speech signals are digitized into 16 bits at a sampling speech recognition framework is reported in Section 6.Fi- frequency of 16 kHz. For spectral analysis, a 24-channel nally Section 7 concludes this paper. MFB analysis is performed on 25-millisecond-long win- dowed speech, with a frame shift of 10 milliseconds. Spec- tral components lower than 250 Hz are filtered out to com- 2. IN-CAR SPEECH DATA AND SPEECH ANALYSIS pensate for the spectrum of the engine noise, which is con- centrated in the lower-frequency region. Log MFB parame- A data collection vehicle (DCV) has been specially designed ters are then estimated. The estimated log MFB vectors are for developing the in-car speech corpus at the Center for transformed into 12 mean normalized Mel-frequency cep- Integrated Acoustic Information Research (CIAIR), Nagoya stral coefficients (CMN-MFCC) using discrete cosine trans- University, Nagoya, Japan [16]. The driver wears a headset formation (DCT) and mean normalization, after which the with a close-talking microphone (#1 in Figure 1) placed in it. time derivatives (Δ CMN-MFCC) are calculated. Weifeng Li et al. 3 Feature vector (log MFB) Estimated Noisy speech
Load more

Recommended publications
  • Linear Statistical Modeling of Speech and Its Applications -- Over 36 Year History of LPC
    Linear Statistical Modeling of Speech and its Applications -- Over 36 year history of LPC -- Fumitada ITAKURA Graduate School of Engineering, Nagoya University Furo-cho 1, Chikusa-ku Nagoya 464-8603, Japan [email protected] Abstract which are locally time-invariant for describing important Linear prediction of speech signal is now a common speech events. A time series obtained by sampling a speech knowledge for students, engineers and researchers who are signal shows a significant autocorrelation between adjacent interested in speech science and technology. The first samples. The short time autocorrelation function is related to introduction of the concept to the international speech the running spectrum, which plays the most important role in community was two presentations [6],[7] at the 6th ICA held speech perception. Let {xn [ ], n= integer} be a discrete time in Tokyo in 1968. While Atal and Schroeder used the concept speech waveform sampled at every ∆T seconds. Assume that of linear prediction to encode speech waveform efficiently, {[x nii− ],= 0," , p } are (p+1) dimensional random variables Itakura and Saito used the concept of maximum likelihood taken from a stochastic process which is stationary over a estimation of speech spectrum to implement a new type of short interval such as 30 milliseconds. When x[n ] is a real vocoder. Since this epoch, these concepts are investigated widely in order to achieve accurate speech analysis, efficient sampled value, the next autoregressive (AR) relation is transmission of speech and speech recognition. In this paper, assumed: the history of LPC will be surveyed mainly based on my p α[][ixn−= i ]σε [ n ] (1) personal experience.
    [Show full text]
  • The History of Linear Prediction the History of Linear Predictionl
    [dsp HISTORY] Bishnu S. Atal The History of Linear Predictionl n 1965, while attending a seminar on coding (LPC) method, then the multi- an airplane. Since the plane moves, one information theory as part of my pulse LPC and the code-excited LPC. must predict its position at the time the Ph.D. course work at the Polytechnic shell will reach the plane. Wiener’s work Institute of Brooklyn, New York, I PREDICTION AND appeared in his famous monograph [2] came across a paper [1] that intro- PREDICTIVE CODING published in 1949. Iduced me to the concept of predictive The concept of prediction was at least a At about the same time, Claude coding. At the time, there would have quarter of a century old by the time I Shannon made a major contribution [3] been no way to foresee how this concept learned about it. In the 1940s, Norbert to the theory of communication of sig- would influence my work over the years. Wiener developed a mathematical theory nals. His work established a mathe- Looking back, that paper and the ideas for calculating the best filters and pre- matical framework for coding and that it generated must have been the dictors for detecting signals hidden in transmission of signals. Shannon also force that started the ball rolling. My noise. Wiener worked during the described a system for efficient encoding story, told next, recollects the events that Second World War on the problem of of English text based on the predictability led to proposing the linear prediction aiming antiaircraft guns to shoot down of the English language.
    [Show full text]
  • Multichannel Speech Enhancement Based on Generalized Gamma Prior Distribution with Its Online Adaptive Estimation
    IEICE TRANS. INF. & SYST., VOL.E91-D, NO.3 MARCH 2008 439 PAPER Special Section on Robust Speech Processing in Realistic Environments Multichannel Speech Enhancement Based on Generalized Gamma Prior Distribution with Its Online Adaptive Estimation Tran HUY DAT†a),Nonmember, Kazuya TAKEDA† †, Member, and Fumitada ITAKURA† ††, Fellow SUMMARY We present a multichannelspeech enhancementmethod oped an approach based on speech estimation using adaptive based on MAP speechspectral magnitude estimation using a generalized generalized gamma distribution modeling on spectrum do- gammamodel of speechprior distribution,where the model parameters main [9], [10] and were able to achieve better performance are adapted from actualnoisy speechin a frame-by-framemanner. The utilizationof a moregeneral prior distribution with its onlineadaptive esti- than conventional methods. The point of this method is that mationis shownto be effectivefor speechspectral estimation in noisyen- using more general prior distribution with online adaptation vironments.Furthermore, the multi-channelinformation in terms of cross- could more accurately estimate the signal and noise statis- channelstatistics are shownto be useful to betteradapt the prior distribu- tics and therefore improve the speech estimation, resulting tion parametersto the actualobservation, resulting in better performance in better performance of speech enhancement. Furthermore, of speechenhancement algorithm. We tested the proposed algorithmin an in-car speechdatabase and obtained significantimprovements of the the generalized gamma model is suitable in the implemen- speech recognitionperformance, particularly under non-stationarynoise tation, yielding tractable-form solutions for the estimation conditionssuch as music, air-conditionerand open window. and adaptation. In this work, we shall extend this method key words: multi-channelspeech enhancement, speech recognition, gen- to the multi-channel case to improve the speech recognition eralizedgamma distribution, moment matching performance.
    [Show full text]
  • Curriculum Vitae Sadaoki Furui, Ph.D. 1. Education 2. Honors and Awards
    Curriculum Vitae Sadaoki Furui, Ph.D. Chair of the Board of Trustees, Toyota Technological Institute at Chicago 6045 S. Kenwood Ave., Chicago, IL 60637 USA [email protected] Chief Research Director, National Institute of Informatics 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430 Japan [email protected] Professor Emeritus, Tokyo Institute of Technology 2-36-6 Shinmachi, Setagaya-ku, Tokyo, 154-0014 Japan [email protected] Tel/Fax: +81-3-3426-0087, Cell: +81-80-4088-1199 1. Education Ph.D., Mathematical Engineering and Instrumentation Physics, University of Tokyo, Tokyo, Japan, 1978 M. S., Mathematical Engineering and Instrumentation Physics, University of Tokyo, Tokyo, Japan, 1970 B. S., Mathematical Engineering and Instrumentation Physics, University of Tokyo, Tokyo, Japan, 1968 2. Honors and Awards 1975 Yonezawa Prize (IEICE) 1985 Sato Paper Award (ASJ) 1987 Sato Paper Award (ASJ) 1988 Best Paper Award (IEICE) 1989 Senior Award (Best Paper Award) (IEEE) 1989 Achievement Award from the Japanese Minister of Science and Technology 1990 Book Award (IEICE) 1993 Best Paper Award (IEICE) 1993 Distinguished Lecturer (IEEE) 1993 Fellow (IEEE) 1996 Fellow (Acoustical Society of America) 2001 Fellow (IEICE) 2001 Mira Paul Memorial Award (Acoustical Foundation, India) 2003 Best Paper Award (IEICE) 2003 Achievement Award (IEICE) 2005 Signal Processing Society Award (IEEE) 2006 Achievement Award from the Japanese Minister of Education, Culture, Sports, Science and Technology 2006 Purple Ribbon Medal from Japanese Emperor 2008 Distinguished Achievement
    [Show full text]
  • Blind Signal Separation for Recognizing Overlapped Speech
    J. Acoust. Soc. Jpn. (E) 19, 6 (1998) Blind signal separation for recognizing overlapped speech Tomohiko Taniguchi, Shouji Kajita, Kazuya Takeda, and Fumitada Itakura Department of Information Electronics, Graduate School of Engineering, Nagoya Univer- sity, Furo-cho 1, Chikusa-ku Nagoya, 464-8603 JAPAN (Received 26 February 1998) Utilization of blind signal separation method for the enhancement of speech recognition accuracy under multi-speaker is discussed. The separation method is based upon the ICA (Independent Component Analysis) and has very few assumptions on the mixing process of speech signals. The recognition experiments are performed under various conditions con- cerned with (1) acoustic environment, (2) interfering speakers and (3) recognition systems. The obtained results under those conditions are summarized as follows. (1) The separation method can improve recognition accuracy more than 20% when the SNR of interfering signal is 0 to 6dB, in a soundproof room. In a reverberant room, however, the improvement of the performance is degraded to about 10%. (2) In general, the recognition accuracy deterio- rated more as the number of interfering speakers increased and the more improvement by the separation is obtained. (3) There is no significant difference between DTW isolated word discrimination and HMM continuous speech recognition results except for the fact that saturation in improvement is observed in high SNR condition in HMM CSR. Keywords: ICA, Blind signal separation, Overlapped speech PACS number: 43. 72. Ew, 43. 72. Ne always apart from each other, has not been clarified 1. INTRODUCTION yet. Tracking speech structure hidden in the acous- Since the multi-speaker situation is quite usual in tic signal, for example harmonic structure or for- speech communication, recognizing overlapped mant frequency, has been also studied in Ref.5-7) speech is one of the most typical and important The method, however, has inherent difficulty in problems of speech recognition in real world.
    [Show full text]
  • Linear Predictive Coding and the Internet Protocol a Survey of LPC and a History of of Realtime Digital Speech on Packet Networks
    Foundations and Trends R in sample Vol. xx, No xx (2010) 1{147 c 2010 DOI: xxxxxx Linear Predictive Coding and the Internet Protocol A survey of LPC and a History of of Realtime Digital Speech on Packet Networks Robert M. Gray1 1 Stanford University, Stanford, CA 94305, USA, [email protected] Abstract In December 1974 the first realtime conversation on the ARPAnet took place between Culler-Harrison Incorporated in Goleta, California, and MIT Lincoln Laboratory in Lexington, Massachusetts. This was the first successful application of realtime digital speech communication over a packet network and an early milestone in the explosion of real- time signal processing of speech, audio, images, and video that we all take for granted today. It could be considered as the first voice over In- ternet Protocol (VoIP), except that the Internet Protocol (IP) had not yet been established. In fact, the interest in realtime signal processing had an indirect, but major, impact on the development of IP. This is the story of the development of linear predictive coded (LPC) speech and how it came to be used in the first successful packet speech ex- periments. Several related stories are recounted as well. The history is preceded by a tutorial on linear prediction methods which incorporates a variety of views to provide context for the stories. Contents Preface1 Part I: Linear Prediction and Speech5 1 Prediction6 2 Optimal Prediction9 2.1 The Problem and its Solution9 2.2 Gaussian vectors 10 3 Linear Prediction 15 3.1 Optimal Linear Prediction 15 3.2 Unknown
    [Show full text]
  • DSP for IN-VEHICLE and MOBILE SYSTEMS This Page Intentionally Left Blank DSP for IN-VEHICLE and MOBILE SYSTEMS
    DSP FOR IN-VEHICLE AND MOBILE SYSTEMS This page intentionally left blank DSP FOR IN-VEHICLE AND MOBILE SYSTEMS Edited by Hüseyin Abut Department of Electrical and Computer Engineering San Diego State University, San Diego, California, USA <[email protected]> John H.L. Hansen Robust Speech Processing Group, Center for Spoken Language Research Dept. Speech, Language & Hearing Sciences, Dept. Electrical Engineering University of Colorado, Boulder, Colorado, USA <[email protected]> Kazuya Takeda Department of Media Science Nagoya University, Nagoya, Japan <[email protected]> Springer eBook ISBN: 0-387-22979-5 Print ISBN: 0-387-22978-7 ©2005 Springer Science + Business Media, Inc. Print ©2005 Springer Science + Business Media, Inc. Boston All rights reserved No part of this eBook may be reproduced or transmitted in any form or by any means, electronic, mechanical, recording, or otherwise, without written consent from the Publisher Created in the United States of America Visit Springer's eBookstore at: http://ebooks.kluweronline.com and the Springer Global Website Online at: http://www.springeronline.com DSP for In-Vehicle and Mobile Systems Dedication To Professor Fumitada Itakura This book, “DSP for In-Vehicle and Mobile Systems”, contains a collection of research papers authored by prominent specialists in the field. It is dedicated to Professor Fumitada Itakura of Nagoya University. It is offered as a tribute to his sustained leadership in Digital Signal Processing during a professional career that spans both industry and academe. In many cases, the work reported in this volume has directly built upon or been influenced by the innovative genius of Professor Itakura.
    [Show full text]