DOCSLIB.ORG

Blind Signal Separation for Recognizing Overlapped Speech

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Blind Signal Separation for Recognizing Overlapped Speech J. Acoust. Soc. Jpn. (E) 19, 6 (1998) Blind signal separation for recognizing overlapped speech Tomohiko Taniguchi, Shouji Kajita, Kazuya Takeda, and Fumitada Itakura Department of Information Electronics, Graduate School of Engineering, Nagoya Univer- sity, Furo-cho 1, Chikusa-ku Nagoya, 464-8603 JAPAN (Received 26 February 1998) Utilization of blind signal separation method for the enhancement of speech recognition accuracy under multi-speaker is discussed. The separation method is based upon the ICA (Independent Component Analysis) and has very few assumptions on the mixing process of speech signals. The recognition experiments are performed under various conditions con- cerned with (1) acoustic environment, (2) interfering speakers and (3) recognition systems. The obtained results under those conditions are summarized as follows. (1) The separation method can improve recognition accuracy more than 20% when the SNR of interfering signal is 0 to 6dB, in a soundproof room. In a reverberant room, however, the improvement of the performance is degraded to about 10%. (2) In general, the recognition accuracy deterio- rated more as the number of interfering speakers increased and the more improvement by the separation is obtained. (3) There is no significant difference between DTW isolated word discrimination and HMM continuous speech recognition results except for the fact that saturation in improvement is observed in high SNR condition in HMM CSR. Keywords: ICA, Blind signal separation, Overlapped speech PACS number: 43. 72. Ew, 43. 72. Ne always apart from each other, has not been clarified 1. INTRODUCTION yet. Tracking speech structure hidden in the acous- Since the multi-speaker situation is quite usual in tic signal, for example harmonic structure or for- speech communication, recognizing overlapped mant frequency, has been also studied in Ref.5-7) speech is one of the most typical and important The method, however, has inherent difficulty in problems of speech recognition in real world. applying to the separation of non-speech signal such Known as "cocktail party effect," humans can focus as ambient noise. on the particular speaker's speech under the exis- While these methods assume the spatial or spec- tence of interfering speech. Based on different inter- tral characteristics on the overlapping signals, pretations of cocktail party effect various recently proposed blind signal separation approaches for recognizing overlapped speech have methods8,9) do not require any special conditions on been studied. the signal mixture process except for; 1) the source One approach is to form a directional beam signals are super-Gaussians, 2) no time difference in which does not receive the interfering signal, based the mixing process, 3) the source signals are statisti- on array signal processing.1-3) The effectiveness of cally independent. Thus blind signal separation the method in applying to speech recognition is seems to be applicable to wider range of problems confirmed when the interfering signal is Gaussian and potentially suites for recognizing overlapped random signal.4) However, the performance under speech. multi-speaker situation, where the interfering signal The purpose of this paper is to clarify the is also speech signal and the sound sources are not effectiveness of the blind signal separation method 385 J. Acoust. Soc. Jpn. (E) 19, 6 (1998) in terms of acoustic preprocessing for recognizing Joint entropy can be calculated as the expected overlapped speech. For the purpose the speech value of the log likelihood of joint distribution by recognition experiments are performed under vari- ous conditions of interfering speech, SNR and recog- (5) nition system. and the joint distribution of y can be determined in This paper consists of the following sections . In terms of joint distribution of x, and Jacobian of Section 2, the blind separation method will be the mapping from x to y briefly described. Then, in Section 3, the experi- mental conditions for evaluating the method will be described. After discussions on the results of exper- (6) iments in Section 4, we will conclude this paper in Section 5. Since the joint distribution of x is independent from 2. METHOD W, it is enough to find W which maximizesthe first In this section, the basic method of blind signal term, E [1n|J|], in order to maximizejoint entropy of separation is described.9) y. The gradient descent method can, then, be If the statistically independent source signals applied to search for the W by differentiating loga- [n],•c, sN[n] are mixed by an N•~N mixing matrix rithm of Jacobian with respectto each element of W A, the mixed signals x1[n],•c, xN[n] are obtained as matrix: follows: (7) (1) where x=(x1[n], x2[n], xN[n])T and s=(s1[n], (8) s2[n],•csN[n])T. Note that there is no time Where ƒÉ is a learning factor and i is the number of difference among terms of a source signal, si[n], in iteration. The iteration can be regarded being contributing the mixed signal, xi[n]. converged when the ƒÉƒ¢W becomes less than Based upon this modeling, the blind separation of predetermined threshold value. Finally, with the signals can be formalized as a problem of finding estimated matrix W the source signal can be recover- mixing matrix A, or more directly, inverse matrix, ed by inverting the mixing A-1. From the assumption that the source signals are independent, the inverse matrix A-1 is expected to (9) be estimated as a matrix of minimizing mutual 3. EXPERIMENTAL SETUP information between the signals obtained by multi- plying an unknown matrix W to the mixed signal, 3.1 Recording Environment i.e. The overlapped speech for recognition experi- (2) ments are recorded both in an unreverberant sound- proof room and a quiet but reverberant (T60=0.70s) Bell et al. found that, as far as s follows a super- laboratory room. The back ground noise levels are Gaussian distribution, the minimization of the 28 (dBA), and 36 (dBA) respectively in those rooms. mutual information, I(Wx), can be achieved by The arrangement of loudspeakers and microphones maximizing joint entropy after nonlinearly squash- for recording multi-speaker speech, are illustrated in ing Wx so as to bound its variance. Defining y as Fig. 1, for the soundproof room condition. The the squashed version of Wx, therefore, the above same loudspeakers and microphones are arranged in minimizing procedure can be accomplished by the same way in a larger (3m•~5m) laboratory (3) room for the recording under reverberant condition. The test and the interfering speech are played by loudspeaker 1 and 2 respectively. Two directional (4) microphones are located at the same point so as to where g(x) is a non-linear squashing, e.g. sigmoid, form a 90•‹ angle. As shown in Fig. 2, at 200-4,000 function. Hz frequency band, the characteristics of the transfer 386 T. TANIGUCHI et al: BLIND SIGNAL SEPARATION FOR RECOGNIZING OVERLAPPED SPEECH Sources of speech and interfering signals are stor- ed in digital 16bit form, 16kHz sampling. The mixed signals are recorded in 8kHz and 16kHz for DTW and CSR experiments, respectively. As for interfering speech, human speech-like noise (HSLN), which is a kind of bubble noise generated by superimposing independent speech signals,12) is utilized. The utterance of each interfering speech is selected to be different from interfered speech. By changing the number of superpositions, we can simulate various multispeaker conditions. When HSLN of one superposition is used for interference, the overlap of two speakers speech is simulated, Fig. 1 Arrangement of speakers and micro- whereas when the number of superpositions is large, phones for recording overlapped speech. then cock-tail party sound is simulated. However, The outer frame corresponds to the size of the soundproof room. it should be noted that the experiment does not simulate a real cock-tail party environment, because interfering speech are not spatially distributed. When the number of superpositions is more than some hundreds, HSLN sounds like stationary noise. 3.2 Recognition Systems Both DTW isolated word discrimination and HMM continuous speech recognition experiments are performed in a speaker dependent manner. The isolated word discrimination task is to discriminate the utterances of a phonetically similar Japanese city Fig. 2 Transfer characteristics from loud- name pair. For the DTW experiment, SGDS13) is speaker 1, to microphone 1 and 2. Since the used as a spectral measure as it outperformed spectral responses of the two channels are MFCC in a preliminary experiment. 68 utterances almost identical, the mixed signal can be rep- of five male speakers are used and the results below resented by linear addition, i.e. x1=a11s1++ are average values of the five male speakers. The a12s2. rest of conditions are summarized in Table 1 Table 1 Acoustic analysis conditions for recog- function from a loudspeaker to microphones are nition experiments. almost the same except for their gains. Therefore, the resultant mixed signals satisfy the mixing condi- tion of linear addition, i.e. x1=a11s1+a12s2. Furthermore, under that condition, the difference of arriving time from the same speaker is negligible between two microphones. (In our preliminary experiment, at least 10dB improvement of Signal-to- Deviation Ratio can be obtained when the time difference between two microphones is less than 0.375ms, i.e. 12cm in length.10,11)) In the other words, ƒÑ11=ƒÑ2i and ƒÑ12=ƒÑ22 hold for the mixing process of the below form. (10) 387 J. Acoust. Soc. Jpn. (E) 19, 6 (1998) together with CSR case. tion, the accuracy of interfering signal (SEP 2) is For continuous speech recognition (CSR) experi- greatly reduced. This is also the evidence of the ment, HTK speech recognizer is used with the condi- effectiveness of signal separation.
Load more

Recommended publications
  • Linear Statistical Modeling of Speech and Its Applications -- Over 36 Year History of LPC
    Linear Statistical Modeling of Speech and its Applications -- Over 36 year history of LPC -- Fumitada ITAKURA Graduate School of Engineering, Nagoya University Furo-cho 1, Chikusa-ku Nagoya 464-8603, Japan [email protected] Abstract which are locally time-invariant for describing important Linear prediction of speech signal is now a common speech events. A time series obtained by sampling a speech knowledge for students, engineers and researchers who are signal shows a significant autocorrelation between adjacent interested in speech science and technology. The first samples. The short time autocorrelation function is related to introduction of the concept to the international speech the running spectrum, which plays the most important role in community was two presentations [6],[7] at the 6th ICA held speech perception. Let {xn [ ], n= integer} be a discrete time in Tokyo in 1968. While Atal and Schroeder used the concept speech waveform sampled at every ∆T seconds. Assume that of linear prediction to encode speech waveform efficiently, {[x nii− ],= 0," , p } are (p+1) dimensional random variables Itakura and Saito used the concept of maximum likelihood taken from a stochastic process which is stationary over a estimation of speech spectrum to implement a new type of short interval such as 30 milliseconds. When x[n ] is a real vocoder. Since this epoch, these concepts are investigated widely in order to achieve accurate speech analysis, efficient sampled value, the next autoregressive (AR) relation is transmission of speech and speech recognition. In this paper, assumed: the history of LPC will be surveyed mainly based on my p α[][ixn−= i ]σε [ n ] (1) personal experience.
    [Show full text]
  • The History of Linear Prediction the History of Linear Predictionl
    [dsp HISTORY] Bishnu S. Atal The History of Linear Predictionl n 1965, while attending a seminar on coding (LPC) method, then the multi- an airplane. Since the plane moves, one information theory as part of my pulse LPC and the code-excited LPC. must predict its position at the time the Ph.D. course work at the Polytechnic shell will reach the plane. Wiener’s work Institute of Brooklyn, New York, I PREDICTION AND appeared in his famous monograph [2] came across a paper [1] that intro- PREDICTIVE CODING published in 1949. Iduced me to the concept of predictive The concept of prediction was at least a At about the same time, Claude coding. At the time, there would have quarter of a century old by the time I Shannon made a major contribution [3] been no way to foresee how this concept learned about it. In the 1940s, Norbert to the theory of communication of sig- would influence my work over the years. Wiener developed a mathematical theory nals. His work established a mathe- Looking back, that paper and the ideas for calculating the best filters and pre- matical framework for coding and that it generated must have been the dictors for detecting signals hidden in transmission of signals. Shannon also force that started the ball rolling. My noise. Wiener worked during the described a system for efficient encoding story, told next, recollects the events that Second World War on the problem of of English text based on the predictability led to proposing the linear prediction aiming antiaircraft guns to shoot down of the English language.
    [Show full text]
  • Multichannel Speech Enhancement Based on Generalized Gamma Prior Distribution with Its Online Adaptive Estimation
    IEICE TRANS. INF. & SYST., VOL.E91-D, NO.3 MARCH 2008 439 PAPER Special Section on Robust Speech Processing in Realistic Environments Multichannel Speech Enhancement Based on Generalized Gamma Prior Distribution with Its Online Adaptive Estimation Tran HUY DAT†a),Nonmember, Kazuya TAKEDA† †, Member, and Fumitada ITAKURA† ††, Fellow SUMMARY We present a multichannelspeech enhancementmethod oped an approach based on speech estimation using adaptive based on MAP speechspectral magnitude estimation using a generalized generalized gamma distribution modeling on spectrum do- gammamodel of speechprior distribution,where the model parameters main [9], [10] and were able to achieve better performance are adapted from actualnoisy speechin a frame-by-framemanner. The utilizationof a moregeneral prior distribution with its onlineadaptive esti- than conventional methods. The point of this method is that mationis shownto be effectivefor speechspectral estimation in noisyen- using more general prior distribution with online adaptation vironments.Furthermore, the multi-channelinformation in terms of cross- could more accurately estimate the signal and noise statis- channelstatistics are shownto be useful to betteradapt the prior distribu- tics and therefore improve the speech estimation, resulting tion parametersto the actualobservation, resulting in better performance in better performance of speech enhancement. Furthermore, of speechenhancement algorithm. We tested the proposed algorithmin an in-car speechdatabase and obtained significantimprovements of the the generalized gamma model is suitable in the implemen- speech recognitionperformance, particularly under non-stationarynoise tation, yielding tractable-form solutions for the estimation conditionssuch as music, air-conditionerand open window. and adaptation. In this work, we shall extend this method key words: multi-channelspeech enhancement, speech recognition, gen- to the multi-channel case to improve the speech recognition eralizedgamma distribution, moment matching performance.
    [Show full text]
  • Curriculum Vitae Sadaoki Furui, Ph.D. 1. Education 2. Honors and Awards
    Curriculum Vitae Sadaoki Furui, Ph.D. Chair of the Board of Trustees, Toyota Technological Institute at Chicago 6045 S. Kenwood Ave., Chicago, IL 60637 USA [email protected] Chief Research Director, National Institute of Informatics 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430 Japan [email protected] Professor Emeritus, Tokyo Institute of Technology 2-36-6 Shinmachi, Setagaya-ku, Tokyo, 154-0014 Japan [email protected] Tel/Fax: +81-3-3426-0087, Cell: +81-80-4088-1199 1. Education Ph.D., Mathematical Engineering and Instrumentation Physics, University of Tokyo, Tokyo, Japan, 1978 M. S., Mathematical Engineering and Instrumentation Physics, University of Tokyo, Tokyo, Japan, 1970 B. S., Mathematical Engineering and Instrumentation Physics, University of Tokyo, Tokyo, Japan, 1968 2. Honors and Awards 1975 Yonezawa Prize (IEICE) 1985 Sato Paper Award (ASJ) 1987 Sato Paper Award (ASJ) 1988 Best Paper Award (IEICE) 1989 Senior Award (Best Paper Award) (IEEE) 1989 Achievement Award from the Japanese Minister of Science and Technology 1990 Book Award (IEICE) 1993 Best Paper Award (IEICE) 1993 Distinguished Lecturer (IEEE) 1993 Fellow (IEEE) 1996 Fellow (Acoustical Society of America) 2001 Fellow (IEICE) 2001 Mira Paul Memorial Award (Acoustical Foundation, India) 2003 Best Paper Award (IEICE) 2003 Achievement Award (IEICE) 2005 Signal Processing Society Award (IEEE) 2006 Achievement Award from the Japanese Minister of Education, Culture, Sports, Science and Technology 2006 Purple Ribbon Medal from Japanese Emperor 2008 Distinguished Achievement
    [Show full text]
  • Linear Predictive Coding and the Internet Protocol a Survey of LPC and a History of of Realtime Digital Speech on Packet Networks
    Foundations and Trends R in sample Vol. xx, No xx (2010) 1{147 c 2010 DOI: xxxxxx Linear Predictive Coding and the Internet Protocol A survey of LPC and a History of of Realtime Digital Speech on Packet Networks Robert M. Gray1 1 Stanford University, Stanford, CA 94305, USA, [email protected] Abstract In December 1974 the first realtime conversation on the ARPAnet took place between Culler-Harrison Incorporated in Goleta, California, and MIT Lincoln Laboratory in Lexington, Massachusetts. This was the first successful application of realtime digital speech communication over a packet network and an early milestone in the explosion of real- time signal processing of speech, audio, images, and video that we all take for granted today. It could be considered as the first voice over In- ternet Protocol (VoIP), except that the Internet Protocol (IP) had not yet been established. In fact, the interest in realtime signal processing had an indirect, but major, impact on the development of IP. This is the story of the development of linear predictive coded (LPC) speech and how it came to be used in the first successful packet speech ex- periments. Several related stories are recounted as well. The history is preceded by a tutorial on linear prediction methods which incorporates a variety of views to provide context for the stories. Contents Preface1 Part I: Linear Prediction and Speech5 1 Prediction6 2 Optimal Prediction9 2.1 The Problem and its Solution9 2.2 Gaussian vectors 10 3 Linear Prediction 15 3.1 Optimal Linear Prediction 15 3.2 Unknown
    [Show full text]
  • DSP for IN-VEHICLE and MOBILE SYSTEMS This Page Intentionally Left Blank DSP for IN-VEHICLE and MOBILE SYSTEMS
    DSP FOR IN-VEHICLE AND MOBILE SYSTEMS This page intentionally left blank DSP FOR IN-VEHICLE AND MOBILE SYSTEMS Edited by Hüseyin Abut Department of Electrical and Computer Engineering San Diego State University, San Diego, California, USA <[email protected]> John H.L. Hansen Robust Speech Processing Group, Center for Spoken Language Research Dept. Speech, Language & Hearing Sciences, Dept. Electrical Engineering University of Colorado, Boulder, Colorado, USA <[email protected]> Kazuya Takeda Department of Media Science Nagoya University, Nagoya, Japan <[email protected]> Springer eBook ISBN: 0-387-22979-5 Print ISBN: 0-387-22978-7 ©2005 Springer Science + Business Media, Inc. Print ©2005 Springer Science + Business Media, Inc. Boston All rights reserved No part of this eBook may be reproduced or transmitted in any form or by any means, electronic, mechanical, recording, or otherwise, without written consent from the Publisher Created in the United States of America Visit Springer's eBookstore at: http://ebooks.kluweronline.com and the Springer Global Website Online at: http://www.springeronline.com DSP for In-Vehicle and Mobile Systems Dedication To Professor Fumitada Itakura This book, “DSP for In-Vehicle and Mobile Systems”, contains a collection of research papers authored by prominent specialists in the field. It is dedicated to Professor Fumitada Itakura of Nagoya University. It is offered as a tribute to his sustained leadership in Digital Signal Processing during a professional career that spans both industry and academe. In many cases, the work reported in this volume has directly built upon or been influenced by the innovative genius of Professor Itakura.
    [Show full text]
  • Robust In-Car Speech Recognition Based on Nonlinear Multiple Regressions
    Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2007, Article ID 16921, 10 pages doi:10.1155/2007/16921 Research Article Robust In-Car Speech Recognition Based on Nonlinear Multiple Regressions Weifeng Li,1 Kazuya Takeda,1 and Fumitada Itakura2 1 Graduate School of Information Science, Nagoya University, Nagoya 464-8603, Japan 2 Department of Information Engineering, Faculty of Science and Technology, Meijo University, Nagoya 468-8502, Japan Received 31 January 2006; Revised 10 August 2006; Accepted 29 October 2006 Recommended by S. Parthasarathy We address issues for improving handsfree speech recognition performance in different car environments using a single distant microphone. In this paper, we propose a nonlinear multiple-regression-based enhancement method for in-car speech recogni- tion. In order to develop a data-driven in-car recognition system, we develop an effective algorithm for adapting the regression parameters to different driving conditions. We also devise the model compensation scheme by synthesizing the training data using the optimal regression parameters and by selecting the optimal HMM for the test speech. Based on isolated word recognition experiments conducted in 15 real car environments, the proposed adaptive regression approach shows an advantage in average relative word error rate (WER) reductions of 52.5% and 14.8%, compared to original noisy speech and ETSI advanced front end, respectively. Copyright © 2007 Weifeng Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 1. INTRODUCTION ization (CMN) [8], codeword-dependent cesptral normal- ization (CDCN) [9], and so on.
    [Show full text]