mathematics for (signal processing for communication)

2007 spatial processing / beam forming

reader: sections 3.1-3.3

Marc Geilen PT 9.35 [email protected] http://www.es.ele.tue.nl/education/5ME00/

Eindhoven University of Technology 2 overview

• optimal beam formers – deterministic approach – stochastic approach • colored noise • the 3 data model

assume we receive d (narrow band) signals on an antenna array: d =+=+ xanAsnkiikkkk∑ s , i =1 objective: construct a receiver weight vector w such that = H ykkwx is an estimate of one of the sources, or all sources: = H yWxkk 4 deterministic approach

=⇔= noiseless case xAskk XAS dxN MxNMxd

objective: find W such that WXH = S

minimize WXH − S

we consider two scenarios: • A is known • S is known 5 deterministic approach

scenario 1 use direction finding methods to determine how many signals there are and what their response vectors are

A is known: = … Aa[]1 ad

(α2) (α1)

(t)

(α) 6 deterministic approach

scenario 2 the sending signal makes use of a known training sequence, agreed in the protocol

S is known, goal: select W

alternatively, this could be the case via decision feedback

Im(sk)

1/ 2

Re(s ) −1/ 2 1/ 2 k

−1/ 2 7 deterministic approach

=⇔= noiseless case xAskk XAS

objective: find W such that WXH = S

with A known − XAS=⇔ AXS††1 =,() A = AAAHH

hence, we set WAH = †

all interference is cancelled (if M ≥ d):

H = WA Id 8 deterministic approach

=⇔= noiseless case xAskk XAS

objective: find W such that WXH = S

with S known

− WSXSXXXHHH==†1††(), AXSW == () H

(after training, W is used to estimate the unknown S)

again, all interference is cancelled (M ≥ d, N ≥ d):

WAH = I 9 deterministic approach

noisy case XASN=+

two possible optimization criteria

model matching: adapting the model of A or S to minimize residual: XAS− 2 F

output error minimization

2 WXH − S F 10 deterministic approach

noisy case XASN=+

model matching: adapting the model of A (or S) to minimize residual: 2 XAS− F

A

Aˆ 11 deterministic approach

noisy case XASN=+

output error minimization 2 WXH − S F n k s x y k A k W k

e k - 12 model matching

with A known: zero-forcing solution

2 ˆˆSXASsuch that − is minimal F ˆSAX= ††⇒ WH = A

WHA=I: all interference is cancelled (hence zero-forcing) (under what conditions?)

the ZF beamformer maximizes the output Signal-to-Interference Ratio (SIR) 13 example model matching

the ZF beamformer satisfies WHA=I

let w1 be the first column of W, the beamformer of the first signal. HH= ⇒ ……= WA I w12[][] a ad 0 0 ⊥ { … } waa12,,d

thus, w1 projects out all other sources, except source 1. but what about noise? d ==HHH + yt11()wx () t∑ wa 1ii st () wn 1 () t i=1 =+H st11()wn () t the effect on the noise is not considered! 14 model matching

zero-forcing solution

WXHH=+ S() A† N

the output noise depends on A+ and can be large, since

= ⇒ †1H = − AUAAAΣΣ V() A U AA V A

-1 this happens if ΣA is large, i.e. if A is ill-conditioned σ 1  1 σd e.g. if directions are very close 15 model matching σ 2 11u σ 2 22u

a2

a1

to discriminate a1 and a2, the ZF beamformer amplifies noise in the direction of u2 16 model matching

with S known:

2 AXASˆˆ such that − is minimal F − AXSXSSSˆ ==†1HH()

this does not specify the beamformer, but it is natural to set

WAH = ˆ† 17 output error minimization

objective: minimize the output error with S known 2 WWXSHHsuch that − is minimal F WSXH = † − note thatXXXX†1= HH() , so that

11−− WSXXXRRWRRHHH=(H11 )= ˆˆ , = ˆˆ -1 NN XS X X XS 1 RXXˆ = H : sample data covariance matrix X N

1 RXSˆ = H : sample correlation between the sources XS N and the received data 18 output error minimization

H 2 objective: minimize the output error WX− S F with A known and assuming independent sources:

observe that 11 SSHH→→ I,0 SN NN 11 1 RXSASSNSAˆ ==HHH +→ XS NN N =→ˆˆ-1− 1 WRRXXS RA X

= H RxxX E[]is the true data covariance matrix. with finite samples, we take the estimate from XXH: = ˆ−1 WRAX 19 deterministic approach

= ˆ−1 WRAX

this is the Linear Minimum Mean Square Error (LMMSE) or Wiener receiver.

• it maximizes the Signal-to-Interference-plus-Noise Ratio (SINR) at the output. • it does not cancel all interference:

WAH ≠ I (because the loss in interference is compensated by the gain in removing noise) 20 Wiener filtering

remember that for a convolution filter h(t) the optimal receiver in terms of output signal-to-noise ratio is given by: Hf()() Sf Gf*()= Hf()2 Sf ()+ Nf ()

spectral decomposition of array receiver

H XASN=+ AUV= Σ •columns of V are vectors in ‘signal-space’ • that obtain the same attenuation (singular values of Σ) • their impact in ‘antenna-space’ are the columns of U 21 spectral decomposition

H AUV= Σ

u3 x2 σ u A σ 11 22u

x1

x3

signal-space antenna-space 22 Wiener filtering

spectral decomposition of array receiver H XASN=+ AUV= Σ

define: XUXSVSNUN**===HHH *

then X* =+UASNH () HH H =+UUΣ VS UN ** = ΣSN+ 23 Wiener filtering

signal and noise are ‘spatially’ white in (transformed) signal-space and antenna-space respectively

RnnUnUnUnnUUIUI=E**H==== E ( HHHHHH )( )  E  σσ 2 2 n*    nn

RssVsVsVssVVIVI=E**HHHHHHH==== E ( )( )  E  s*    24 Wiener filtering

covariance of the transformed antenna samples:

Rxxsnsn=E**HHHH=+ E ( ** )( ** += )  E ssnn **** +  x* ΣΣ  ΣΣ  =+=+σσ222 ΣΣIInn Σ I

a diagonal matrix! in this space, ‘virtual’ antenna elements are independent!

Rxssnsssns=E**HHHH=+ E ( * * ) *  = E ** + **  xs** ΣΣ   **HH ** =+=+=ΣΣΣEEss ns 0 25 Wiener filtering

thus, the Wiener receiver in the transformed space is

WX***HH== S⇒ W * SX **†

−1 ***H**HHH11 − 1 *-1 WSXXXRRWRR=,== ˆˆ** * ˆˆ * ** NNXS X X XS −1 *22→+σ WI()ΣΣn a diagonal matrix (!) with σ Hf()() Sf w * = k compare: Gf*( ) = kk, σσ22+ 2 kn Hf() Sf ()+ Nf () (in this space, beam forming is trivial)

YVYVWX==****HHH = VWUX WUWV= * H 26 comparing receivers

output error interference zero-forcing receiver (solid) Wiener receiver Wiener receiver (dashed) (what about ZF?) 27 comparing receivers

==H σσ… AUVΣΣ, diag(1 ,d ) • zero forcing = H WUWVZF 1 W diagonal with w ZF = ZF k, k σ k • Wiener receiver = H WUWWiener V σ W diagonal with wWiener = k Wiener k, k σσ22+ kn 28 stochastic approach

assume a model with 1 source =+ =HH = + H xakkks, nys k wxwawn k ()() k k we make the following assumptions: 2 ==H E[sskkk ] 1,E[]n0 and define (spatial noise ‘color’): = H Rnnn :E[kk ] so that 2 =+HHHHH + E[|ys | ] E[(wakk wn )( wa s kk wn ) ] =+++HHHHHHHH2 E[wa |ssskkkkkkk | aw wa nw wn aw wnnw ] =HH +++ H H E[waaw 0 0 wnnwkk ] =+HH H ()()waaw wRwn 29 stochastic approach

2 =+HH H E[|y | ] waaw wRwn

signal noise

so the Signal to Noise Ratio (SNR) at the output of the Wiener receiver is:

E[| (waH )s |2 ] waawHH SNR ()w ==k out H 2 H E[| (wnk ) | ] wRwn 30 stochastic approach

two stochastic optimization criteria • maximum likelihood the likelihood of a set of data X is the probability (density) of obtaining that particular set of data, given the transmitted signal S. The Maximum Likelihood Estimate of S is the value of S that maximizes the probability of receiving that particular X. (compare deterministic model matching)

• stochastic output error minimization the error at the output y of the receiver is a , which depends on the weight vector w. Choose the weight vector w, which minimizes the variance of this process. (compare deterministic output error minimization) 31 stochastic model matching

assume a model with d sources

=+ =… ⇔ =+ xAsnkkk(1kN,,) XASN

assume sk to be deterministic

noise i.i.d. in time (temporally white), and spatially white 2 (Rn=σ I) and jointly complex Gaussian distributed

2 nk − 0.4 2 1 σ 2 σ ⇔= 0.3 n0Inkk~(,)CN pe () 2 0.2 0.1

0 25 20 15 10 25 20 15 πσ 5 10 5 0 0 32 stochastic model matching

nk=xk-Ask so the probability (density) to receive a certain vector xk, when sk has been transmitted is: − 2 − xAskk ==−=1 σ 2 pp(|)xskk ( n k x k As k ) e 2πσ

− 2 ∑ xAs− 2 N −−xAskk N kk 1122 pe(|)XS==∏ σσ e σπσ22πσ  k=1 XAS− 2 N − F 1 2 = e σ 2 πσ XAS− 2 − F 2 =⋅const e σ

(p(X|S) is the likelihood of receiving a certain data matrix X, for a given transmitted data matrix S) 33 stochastic model matching

XAS− 2 − F pconste(|)XS=⋅σ 2

deterministic maximum likelihood technique: estimate S as the one which maximizes the likelihood of the actually received X ˆSXSsuch that p ( | ) is maximal

XAS- 2 - F ⇔ ˆS such that eσ 2 is maximal

⇔ ˆSXASsuch that -2 is minimal F

for white Gaussian noise, Maximum Likelihood is equivalent to deterministic model matching (hence, the solutions are also the same) 34 stochastic derivation

stochastic output error minimization minimize the Linear Minimum Mean Square Error cost:

 2 Jw()=− E wxH s  kk it can be worked out as follows: 2 Jw()=−=− EwxHHHH s E ()() wx s wx − s kk kk kk =−−+HH H[]  H  2 wEEsEsEs xxwwkk x kk  kk xw  k  =−HH[] − H +2 wRwx wEssEss akk kk a k w Es k 

2 if sk is regarded stochastic with E[|sk| ]=1, then =HHH −−+ Jw()wRwx wa aw 1 35 stochastic output error minimization

=HHH −−+ J()wwRwwaawx 1 differentiate with respect to w: let w=u-jv wit u and v real valued, then the gradient is ∂∂ JJ ∂∂ uv11 11 1 1 ∇=∇−∇=JJjJ»» − j wu22 v 2 2 ∂∂ JJ ∂∂ uvdd ∇=∇=∇HH H = with properties wwwwa a,0 aw wRwxx Rw ⇒ ∇= − wJ Rwx a the minimum J(w) is attained for ∇= ⇒ = −1 wJ 0wRax we thus obtain the Wiener receiver 36 stochastic output error minimization

the minimum J(w) is attained for the Wiener receiver = −1 wRax

the expected output error becomes:

=HHH−−1111 −−+ − − Jmin aRxxx RR a aR x a aR x a 1 =−H −1 1 aRx a − =−HHσ 2 + 1 1 aaa()s Rn a ()−1 =−1 aaaHH + 2 Ia nσ 37 stochastic output error minimization

the expected output error becomes:

−1 =−HH +σ 2 Jmin 1 aaa()n Ia (−−11 )()() =−HH +222 H + H + 1 aaa()()()nnnσσσ Iaaa aa ()()()−11− =−1 aaaHH +2 I aaa H +2 aaa H + 2 ()nnnσσ −−11 =−1 aaaHH+ 222 I aa H + Iaa Ha + nnnσσσ −1 σ =+1− aaaHHa 2 n σ

→ σ 2  2 →→σ 2 Jmin 1 i f n a Jmin 0 if n 0 38 colored noise

what if noise vector is not spatially white?

H assume noise has a known covariance E[nn ]=Rn≠σ2I

-1/2 ‘prewhiten’ the data with a square root factor Rn

=+ ⇒ −−1/ 2=+ 1/ 2 − 1/ 2 xAsnkkk RxRAsRn nknknk =+ xAsnkkk

==H −−1/ 2 1/ 2 = note that RnnRRRInkknnnE 

2 so that the noise nk is white, with variance σ =1 39 colored noise

covariance matrices have a square root:

==HH RXXXUVX Σ

= HH RUVVUX ΣΣ 2 H = UUΣ

1/ 2 = H −−1/ 2= 1 H RUUX Σ RUUX Σ

check that 1/ 2 1/ 2 = 1/ 2− 1/ 2 = RRXX R X RRXX I 40 colored noise

we use the known results on the new variables: = −1/ 2 = −1/ 2 = −1/ 2 ARn A xRxknk nRnknk

the ZF equalizer becomes ==†1HH− yAxAAAxkk() k = HH−−−−−1/2 1/2 1 HH 1/2 1/2 ((ARn ) Rnnnk A ) AR ( ) R x ==HH−−11 − 1 H ()ARnnkk A AR x Wx ⇒ = −−−111H WRAARAnn()

the Wiener receiver is the same as before, since Rn is not used in the derivation check: ==−−−−−11/21/211/21/21 = − WRARxx()nnn RR R AR n RA x ⇒ ==HH−1/ 2 = H YWXWRn XWX ⇒ = −1/ 2 = −1 WARn W Rx 41 the matched filter

consider a single signal in white noise, =+H =σ 2 xakkks, nE  nn kk I the ZF beamformer is given by − wa==()†1HH aaa ( ) =γ a

note: a scalar multiplication does not change the output SNR

w=a is known as a matched filter, classical beamformer and Maximum Ratio Combining (MRC) 42 the matched filter

with non-white noise, =+H = xakkks, nE  nnR kk n

we have seen that the zero forcing beam former in non- white noise equals: = −−−111H WRAARAnn()

hence, in case of a single signal we get: ==−−−−1111H γ wRaaRann() Ra n

thus, the matched filter in non-white noise is = −1 wRan 43 the matched filter

similarly the Wiener filter, in white noise,

==−−121H +σ wRax () aa Ia σσσ−− =+(aaHHH21 I )()() a a a + 2 a a + 21 σσσ−− =+()()()aaHHH21 I aa a + 2 a a a + 21 σσσ−− =+()()()aaHHH21 I aaa + 2 Iaaa + 21 −− =+()()()aaHHH21 Iσσσ aa + 2 I a a a + 21 =+H 21− σ aa() a γ = a

it is also equal to a multiple of the matched filter! 44 the matched filter

Wiener Filter in colored noise, (using whitening)

= −1/ 2 aRn a ==−−11H + wRaxn() aaR a =+−−1/ 2H 1 RaaIan () =+−−1/ 2H 1 Raaan (1) = γ −1 Ran

this is again equal to a multiple of the matched filter for colored noise 45 the matched filter

the colored noise case is relevant also for the following reason: with more than one signal, we can write the model as =+=+′′ + xkkkk Asna1s () Asn kk this is of the form

=+ =H +σ 2 xakkks, n RAAn '' I

where the “noise” is colored due to the contribution of the interfering sources. 46 matched filter

in summary, • the matched filter is optimal with only one signal • equal to both zero-forcing and Wiener filter (with one signal there is no interference) • with colored noise, they also coincide • noise color can be used to capture interference of other signals 47 summary

•criteria: model matching vs. output error minimization • solutions: zero-forcing, Wiener filter • zero forcing optimizes: model matching, interference • Wiener filter optimizes: output error, SNR • matched filter: for a single signal both coincide and the beam former equal the response vector •both deterministic and stochastic analysis • colored noise