Bayesian Interpolation in a Dynamic Sinusoidal Model with Application to Packet-Loss Concealment
Total Page:16
File Type:pdf, Size:1020Kb
18th European Signal Processing Conference (EUSIPCO-2010) Aalborg, Denmark, August 23-27, 2010 BAYESIAN INTERPOLATION IN A DYNAMIC SINUSOIDAL MODEL WITH APPLICATION TO PACKET-LOSS CONCEALMENT Jesper Kjær Nielseny, Mads Græsbøll Christenseny, Ali Taylan Cemgilyy, Simon J. Godsillyyy, and Søren Holdt Jenseny yAalborg University yyBoğaziçi University yyyUniversity of Cambridge Department of Electronic Systems Department of Computer Engineering Department of Engineering Niels Jernes Vej 12, DK-9220 Aalborg 34342 Bebek Istanbul, TR Trumpington St., Cambridge, CB2 1PZ, UK {jkn,mgc,shj,tl}@es.aau.dk [email protected] [email protected] ABSTRACT white Gaussian state and observation noise sequences with 2 In this paper, we consider Bayesian interpolation and pa- covariance matrix Q and variance σw, respectively. We also rameter estimation in a dynamic sinusoidal model. This assume a Gaussian prior for the initial state vector s1 with model is more flexible than the static sinusoidal model since mean vector µ and covariance matrix P . For a non-zero it enables the amplitudes and phases of the sinusoids to be state covariance matrix, the dynamic sinusoidal model in (1) time-varying. For the dynamic sinusoidal model, we derive is able to model non-stationary tonal signals such as a wide a Bayesian inference scheme for the missing observations, range of speech and audio signal segments. We are here con- hidden states and model parameters of the dynamic model. cerned with the problem of performing interpolation and pa- The inference scheme is based on a Markov chain Monte rameter estimation in the model in (1) from a Bayesian per- Carlo method known as Gibbs sampler. We illustrate the spective which offer some conceptual advantages to classical performance of the inference scheme to the application of statistics (see, e.g., [8]). For example, the Bayesian approach packet-loss concealment of lost audio and speech packets. offers a standardised way of dealing with nuisance parame- ters and signal interpolation [4]. The downside of using the 1. INTRODUCTION Bayesian methods is that they struggle with practical prob- lems such as evaluation of high-dimensional and intractable Interpolation of missing, corrupted and future samples in sig- integrals. Although various developments in Markov chain nal waveforms is an important task in several applications. Monte Carlo (MCMC) methods (see, e.g., [9]) in recent years For example, speech and audio signals are often transmit- have overcome these problems to a great extend, the meth- ted over packet-based networks in which packets may be ods still remain very computational intensive. lost, delayed or corrupted. If the contents of neighbouring Within the field of econometrics, the dynamic sinusoidal packets are correlated, the erroneous packets can be approx- model in (1) is well-known and referred to as the stochastic imately reconstructed by using suitable interpolation tech- cyclical model [10]. Two slightly different stochastic cyclical niques. The simplest interpolation techniques employ sig- models were given a fully Bayesian treatment using MCMC nal repetition [1] and signal stretching [2], whereas more inference techniques in [11] and [12]. Neither of these, how- advanced interpolation techniques are based on filter bank ever, considered the case where some observations are miss- methods such as GAPES and MAPES [3], and signal mod- ing. In the audio and speech processing field, the dynamic elling such as autoregressive models [4,5], hidden Markov sinusoidal model has also been considered by Cemgil et al. in models [6], and sinusoidal models [7]. An integral part of [13, 14, 15]. However, they considered the frequency param- the techniques based on signal modelling is the estimation of eter as a discrete random variable and based their inference the signal parameters. Given estimates of these parameters, on approximate variational Bayesian methods. the interpolation task is simply a question of simulating data In this paper, we extend the above work by developing from the model. In this paper, we develop an interpolation an inference scheme for the dynamic sinusoidal model based and parameter estimation scheme by assuming a dynamic si- on MCMC inference techniques. We consider the frequency nusoidal model for an observed signal segment. This model parameter as a continuous random variable and allow some can be written as a linear Gaussian time-invariant state space of the observations to be missing. To achieve this, we develop model given by a Gibbs sampling scheme. The output of this sampler can be T used for forming histograms of the unknown parameters of yn = b sn + wn (observation equation) (1) interest. These histograms have the desirable property that sn+1 = Asn + vn (state equation) they converge to the probability distribution of these un- known parameters when the number of generated samples is where n = 1;:::;N label the uniform sampled data in time, increased, and they therefore enable us to extract statistical and features for the model parameters as well as for performing T b = [1 0 1 0] (2) the interpolation of the missing observations. It should be ··· noted that although this inference scheme can be used for A = diag(A1; ; Al; AL) (3) estimating parameters of signals with no missing observa- ··· ··· cos !l sin !l tions, the primary focus of this paper is on the application Al = exp( γl) ; (4) − sin !l cos !l of reconstructing missing observations from signal segments − which are assumed to have been generated by a dynamic with !l [0; π] and γl > 0 denoting the (angular) frequency, sinusoidal model. and the2 log-damping coefficient of the l’th sinusoid, respec- The paper is organised as follows. In Sec.2, we formalise tively. Further, sn is the state vector, and vn and wn are the problem by setting up the Bayesian framework. This en- ables us in Sec.3 to develop the interpolation and inference The work of J.K. Nielsen was supported by the Oticon Foun- scheme. In Sec.4, we illustrate the performance of the inter- dation’s Scholarship. © EURASIP, 2010 ISSN 2076-1465 239 polating scheme by use of simulations, and Sec.5 concludes tions. For the prior distribution, we assume the factorisation this paper. 2 p(s1; θ) = p(s1)p(!)p(γ)p(q)p(σw) 2. PROBLEM FORMULATION " L # Y 2 In the Bayesian approach, all variables of the model in (1) = p(s1) p(!l)p(γl)p(ql) p(σw) (8) are random variables, and we partition them as l=1 T y = [y1; y2; ; yN ] where p(s1) has a normal distribution (s1; µ; P ), p(!l) has Observations: N ··· a uniform distribution (!l; 0; π), p(γl) has an exponential Latent variables: S = [s1; s2; ; sN ] U 2 ··· distribution Exp(γl; λl), and p(σw) and p(ql) have inverse 2 2 Model parameters: θ = !; γ; q; σw gamma distributions (σ ; αw; βw) and (ql; αv;l; βv;l). f g IG w IG The model evidence p(yo) is independent of S and θ and where !, γ and q are L-dimensional vectors consisting of is therefore a mere scale factor which can be ignored in the the L frequencies, the L log-damping parameters and the inference stage. L state noise variances, respectively. The nth state vector T T T sn = s ; ; s consists of L two-dimensional state 3. INFERENCE SCHEME n;1 ··· n;L vectors pertaining to the L sinusoids. Conditioned on the In the Bayesian framework, all statistical inference is based previous state vector, each of these L two-dimensional state on the posterior distribution over the unknown variables or I I vectors has isotropic covariance matrix ql 2, where 2 is the a marginal posterior distribution over some of these. As de- 2 2 identity matrix, so that Q = diag(q) I2 where is the × ⊗ ⊗ rived in the previous section, we have to generate samples Kronecker product. We also assume that R of the elements in from p(S; θ y ) in order to be able to do this. Unfortunately, y o are missing or corrupted, and that we know their indices this distributionj has a very complicated form, and we are 1;:::;N . Using this set of indices, we define the I ⊂ f g therefore not able to sample directly from it. We therefore vectors ym , yI and yo , ynI containing the R missing have to resort to other sampling techniques in order to enable or corrupted observations and the N R valid observations, statistical inference based on this distribution. One of the − respectively. The notation ( )\∗ denotes ’without element ’. simplest and most popular numerical sampling techniques is · ∗ The primary objective of this paper is to recover ym the Gibbs sampler [17] which is an MCMC-based algorithm from yo. This can be achieved in various ways, e.g., by us- and suitable for this task. The Gibbs sampler draws samples ing MAP/MMSE estimate w.r.t. the posterior distribution from a multivariate distribution, say p(x) = p(x1;:::; xK ), p(y y ) or by drawing a sample from p(y y ). The MAP- by breaking it into a number of conditional distributions mj o mj o based interpolation produces the most probable interpolants. p(xk xnk) of smaller dimensionality from which samples are For audio and speech signals, however, MAP/MMSE-based obtainedj in an alternating pattern. Specifically, for the τ’s interpolation tends to produce over-smoothed interpolants in iteration, we sample for k = 1;:::;K from the sense that they do not agree with the stochastic part of the valid observations [16]. A more typical interpolant can [τ+1] [τ+1] [τ+1] [τ] [τ] x p(xk x1 ;:::; x − ; x ;:::; x ) : (9) be obtained by drawing a single sample from p(y y ) [4].