Approximations to Joint-ML and ML Symbol-Channel Estimators in MUD CDMA

Thomas Fabricius Ole Nørklit Informatics and Mathematical Modelling Algorithm Design Technical University of Denmark Nokia Mobile Phones Copenhagen, Denmark Copenhagen, Denmark

Abstract—In this contribution will we conceptually derive two symbol- ML. The reason for proposing these new estimators, is to re- channel estimators, the joint-ML and the ML, both having exponential veal the better performance achieved from using both the apri- complexity. Pragmatically we derive three approximations with polyno- mial complexity, one to the joint-ML: the pseudo-joint-ML; two to the ori statistics of the symbols and the properties of the signa- ML: the naive-ML and the linear-response-ML. We asses the resulting ture waveform in addition to a correct ML estimator of the average bit error rates empirically. Performance gains of several dBs are channel. The approximations applied are inspired from [6] observed from using the ML based approximations compared to the joint- on Independent Component Analysis (ICA), being similar to ML. Blind MUD CDMA [7]. The derivation is based on a rela-

Keywords—CDMA, Multiuser Detection, Blind channel estimation tively simple channel model: Synchronous Users in Flat Fading CDMA; but the methods also applies to more general I. INTRODUCTION models. The paper is organised as follows: First we introduce Traditional code division multiple access (CDMA) systems the channel model; then we review optimal symbol detection are interference limited due to receivers designed using a code according to both minimum probability of error and minimum matched ﬁlter or correlator followed by detection. However, BER. Next we derive the joint-ML detector and its approxima- communication networks with high data rates, force the use of tion: the pseudo-joint-ML. Lastly we derive the ML detector short spreading codes. This will generate additional multiple and its approximations: the naive-ML and the linear-response- access interference (MAI) and in such situations interference ML. The performance of the detectors are assessed empirically will limit the receiver performance. As shown by Verdu´ [1] and discussed, followed by a conclusion.

CDMA systems need not to be interference limited; when using an optimal multiuser detector (MUD) instead of the tra- II. USERS CDMA IN FLAT FADING ditional receiver. These optimal receivers have exponential Assuming a ﬂat slow fading channel1, and BPSK modula- complexity, so sub-optimal solutions to the multiuser detection tion the received base band CDMA signal can be modelled as

problem have been proposed [1], [2]. In traditional settings the

channel is estimated from pilot sequences, but blind estimation

¢¥¤ ¦£!#"%$&¢'¤§¦ ¡£¢¥¤§¦©¨

of the channel has received focus, aiming to increase the spec- (1)

tral efﬁciency. A blind linear detector that minimises the out-

put energy (MOE) is reported in [1], asymptotically the detec-

*),+

¤ ¢¥¤§¦

tor approaches the linear MMSE detector. In [3] the received where is time and is the ( ’th users signature waveform

.-/0-01

signal is projected into a subspace, where a linear decorrelating with unit energy and is the bit duration, is

2 (

or linear MMSE detector is formed, by only knowing the sig- user ( ’s ’th bit in the block of duration , is the ’th $&¢¥¤§¦ nature waveform from the user of interest. In [4] the likelihood users fading coefﬁcient, and is a stationary white complex

for the unknown channel is established by using the statistical Gaussian process with unit spectral density. )4+

properties of the symbols and the signature waveform of the We process the signal ¡£¢¥¤§¦ with the conventional matched

1 user of interest. Besides taking the a priori statistics of the -5 ﬁlter bank, each ﬁlter indexed by (3 , followed by in-

symbols into account, the next natural step is to include the a tegration and sampling at each bit instant to get the sufﬁcient "£6 priori statistics of the fading. The last approach is employed in statistics2 (for , , and )

[5], resulting in a Space Alternating Generalized EM (SAGE)

algorithm.

¨;:#< ¡£¢'¤§¦ ¢¥¤> ¦@?A¤&¨ !D$ 7

98 98 98¥

Our contribution is twofold, ﬁrst we derive the joint-ML 98¥ (2)

= symbol-channel estimator and an approximation to this, de- noted pseudo-joint-ML detector. Secondly we derive the ML E

We assume nothing about the distribution of the fading. F detector for the channel i.e. marginalising out the symbols, and We can understand this change to sufﬁcient statistics as a projection onto a two approximations to this: naive-ML and Linear-Response- subspace spanned by the signature waveforms.

¢'¤§¦ ¢¥¤§¦

where 8 is the correlation between signal and , IV. JOINT-ML SYMBOL-CHANNEL ESTIMATION $

and 98' is a Gaussian random variable with both real and Constructing the joint-ML estimator, we divide the maximi-

" 6

imaginary part having zero mean and variance . Using the sation into two parts; one for the symbols given the channel

" 6

badcCG

above we have the joint likelihood for , , and and one for the channel given the symbols. Using of the

- joint likelihood (3) the symbol maximisation becomes

¡£¢¥¤§¦ / / © / " ¨

¦ ¨

" ©

! $

3 ¢

¨eNTG Zhg;H ¤ i© ¦ © ¢¥¤ i© ¦

¨ ¨ ¨ (9)

(3) <0PR

VYX

S fU

¢¥¤ © ¦"!#©%$ ¢¥¤ © ¦"&('

¨ ¨

which is identical to (8) for known . The update for the chan-

¢¥¤ ¦ ¨

where we have introduced matrix notation using 8 nel given the symbols is a convex optimisation problem, hence

7 ¢ ¦ ¨ ¢ ¦ ¨*) ¢ ¦ ¢+©.¦ ¨

8 8 8 8

8 we can take the partial derivative with respect to the channel

( ( 3

, ¨ , , and

"£6

¢ ¦ #

8 and , equate to zero and solve

, means transpose and complex conjugate, and is

j j

! !ml5$

¢ ¦ ¨ gkNG ¢7¤ ¦ the usual matrix trace. gkNG

¨ (10)

3 3

PTIMAL ETECTORS 3 .no© ¤p¤ ! © ¤

$ ! ! ! ! III. O D !#q

¨ ¨ ¨

" ¨

Given the noisy data and the channel one goal is to minimise (11)

the expected BER, another to minimise the probability of error. 2

lsrIt5u

3 3

¢ ¦ ¨ ¢ ¦ ¢M©.¦ ! gkNG%¢ ¦

8 8 8 8

2wv ¨ The expected BER is deﬁned as where ¨ and

of a matrix means the diagonal of that matrix arranged as a col-

- -

,.-0/

!%3

¨ - #

<>= ? @9ACB

21 umn vector. Since the symbols are discrete the optimisation in

¨547698;: ¨ (4)

2 equation (9) has to be carried out by enumeration3, i.e. ex-

<>= ? @ B

1 haustive search, to surely obtain the global joint-ML solution.

where the average 4¥698;: , as indicated, is taken with

" 6 rIt5u ¤ We see that the evaluation of can be omitted if only the

respect to all bit realizations ¨ and all noise realizations .

¢ ¦

¦ 6

¡¢¥¤ / / " ¦ yx{z

symbol estimates are of interest. Asymptotically 2 ¦ The joint distribution ¨ is found by the likelihood

(3) multiplied by the a priori distribution of the transmitted the joint-ML detector will minimise the probability of error,

¢ ¦ ¡ since the channel estimate will approach the actual channel and

symbols ¨

equation (9) becomes equivalent to equation (8). This results

6 6

¦ ¦

/ " ¦&¨D¡ ¤ ¡ ¢7¤ / / / " ¡¢ ¦

¦ ¦

¨ ¨

¨ (5) in the complexity being equivalent to the complexity of (8).

where the dependence on © is omitted due to the fact that it is A. Pseudo-joint-ML Estimation known.

Since the complexity of the joint-ML estimator is exponen-

It can be shown, under some regularity conditions, that the tial in we derive a local search algorithm for the symbols.

¦ ¦

¢ ¦ ¨ -

ba;cIG

¦ ¦

expected BER subject to the constraint that ¨ is Taking the partial derivative of of the joint likelihood

¦ ¦

¢ ¦ ¨ -

8 ¨

minimised by (3) with respect to ¨ , subject to the constraint ,

¨FE GIH

<>= ? @9AJB

1 equating to zero, and solving yields

¨ :

¨%47698 (6)

¤ E GIH

¨FE GIH|¢ ! ¢7¤ ¢+© ¦ ¦

for all given received data , working element wise. v ¨

¨ (12)

The probability of error is deﬁned as the probability that at E GIH where acts on each element. Taking partial derivatives with

least one of the 2 bits in the block are detected wrongly. We

"£6

¡ ¢+K ¨,- ¦ ¨ - L) ¢ L ¦

! respect to and , equating to zero and solving yields the

¨ ¨

can deﬁne ¨ and so the

" 6 probability of error becomes equations (10) and (11) for and .

The equations holds in minima since the gradient of the joint

¦ 3

¡¢MK ¨ - / " ¦©¨ - ) ¢ # ! ¦

<>= ? @ B 1

2 likelihood equals zero. Solving the equations can be carried

¦ ¨ 47698;: ¨ (7)

out in many ways leading to e.g. hard decision successive in- which is shown to be minimised for any received data ¤ by terference cancellation [2] (HSIC) or hard decision parallel in-

terference cancellation (HPIC). Updating equation (12) suffers

¨ONI G Z[N ¡¢ ¦ ¤ / / " ¦\

¨ ¦ ¨ (8)

TS ¥UWVYX $ Thus, it does not in general achieve the joint-ML solution, this

Equation (8) is the well known Maximum A Posterior (MAP) is the reason for the name pseudo-joint-ML. In this context 6

solution, which reduces to maximum likelihood for uniform we can see the update of and " as channel estimation us-

¡¢ ¦ ing hard decision feedback from equation (12). We can see the

prior distribution ¨ .

¢"^`_ ¦

The complexity per bit for both methods are ] due to pseudo-joint-ML as trying to minimise the probability of error. }

the summation and exhaustive search, which is intractable due Equation (10) can eventually be substituted back into (9) before the enu- to the exponential law in . meration.

¢ ¦ ¤

V. ML CHANNEL ESTIMATOR we now propose an arbitrary distribution ¨ which has sup-

¦ 6

¡ ¢ ¤ / / © / " ¦ ¦ port where the posterior distribution ¨ has Since the joint likelihood is the likelihood for the symbols support. Using Jensens inequality we obtain

jointly with the channel, we now derive the correct likelihood

¡ ¢7¤ / / © / "£6 ¦

for the channel alone.

¦ ¨

¢ ¦ adcCG / © / " ¦¦¥ a;cIGY¡ ¢7¤F¦

In ML settings it is common to distinguish between hidden ¢

<0PR

VYX

¥U variables (incomplete data), visible variables (complete data $

besides the incomplete data) and parameters4. The likelihood (18)

¢ ¦ ¨ ¡ ¢ ¦ ¤ / /T© / "£6 ¦

" 6

¨ ¦

of the channel parameters and is found as the joint like- Equality is obtained if ¨ which can

be proved by Bayes rule. Making this choice will obviously

lihood times the prior of the hidden variables ¨ marginalized have the same complexity as calculating the likelihood exactly.

with respect to the hidden variables ¨ . The likelihood for

¢ ¦

¤ ¨ and then becomes Now, the idea is to come up with a distribution that makes

the calculation of the right side of (18) tractable and makes the

¡¢7¤F¦ / /T© / "£6%¦

bound tight. The resulting distribution being an approximation

/T© / " ¦&¨ ¡ ¢7¤

¢ ¦

(13)

¤ ¨

to the posterior distribution of ¨ . If is parameterised we

<0PR

VYX

¥U

$ can use (18) as a cost function to minimise with respect to the

¢ ¦

parameters under the constraint that ¨ sums to one.

¡ ¢ ¦©¨ $

where the prior is ¨ . Taking partial derivatives of Here we take

badcCG " 6

of the likelihood (13) with respect to , respectively,

§ §

¦©¨ ¢ ¦

equating to zero and solving we get ¢

¤ ¤ ¨

¨ (19)

j j

! lh$

g;NIG ¢ ¦©¨ gkNG ¢¥¤ ¦ 1

¨%4 (14)

i.e. to factorise completely, where

¤p¤ ! © #>¢M© ¤

! ! ! $ ! !

1 1

¨s¨ 4 ¨54

" ¨

A A

- ¨ - !©¨

(15)

' '

/ ¢ ¦©¨

¨ (20)

with

is chosen as a Bernoulli distribution with parameter .

¢ ¦ ¨ ¢ ¦ ¢M© ¦ ! \

8 8 8 8

¨"¨

1 < B

v 2wv

¨%¨ 4

(16) 1

4! 8

This parameterisation is chosen to make ¨ .

This approximation is in the physics literature [8] referred to

<>= ? @ B

1 1

4 :

where we deﬁne 4¥698 i.e. the posterior ex- as the naive approximation.

pectation. We see that the ML estimator, contrary to pseudo- ¢ ¤

With the above ¨ we write out the right side of equation ¨

joint-ML estimator, needs knowledge of the noise variance. (18) and take the partial derivative with respect to the ’s

" 6

¦ ¨ -

The ML estimator of and is a more efﬁcient estimator ¢ #$¤

under the constraint ¨ and equating to zero. This

¢&% ¦ ¨'¨

than the joint-ML estimator. This can be explained by the yields the equation for the matrix 1

fact that the ML estimate of the channel depends on ¨%4 and

¢ ! !

¤ ¢+© ¦ % % ¨"(TNIH*)

4 ¨ ¨%¨ which ﬂuctuates less than the that maximises the

v (21)

¨s¨

joint likelihood and used in the joint-ML channel esti- 1

mate (10). Since we have to calculate ¨54 , we get the optimal We see that (21) only differs from pseudo-joint-ML updates in

¨ E GIH

(TNIH*) ¢,+ ¦ E GIH ¢ ¦

¨ ¨54

BER symbol estimator, equation (6): , as a side @9A

(12) by instead of . 6 product. Asymptotically the channel estimate approaches the To update and " we use the ML estimate (14) and

actual channel, implying the symbol estimate asymptotically (15) described in the previous section with the approximation

¨% ¨

! !

minimises the expected BER. Unfortunately this also implies

1 1 1 1

4/- ¨s¨ 4! 8 .- ¨%4! 8 ¨%¨

that ¨%4 and

¢¢¡ ¦

¨0%1% ¢ ¦

! ! <B

the same complexity as . <B

1 1

¨54 8 ¨ 4 8 , where we have used that ¨ factorises. A. Approximate ML Channel Estimator

We see that the algorithm obtained is very similar to that of

1 1

¨%¨ 4 Because of the high complexity to obtain ¨54 and , the pseudo-joint-ML, except we get soft decisions instead of the aim now is to approximate these in a less costly way than hard decisions. The way we solve equation (21) gives either

doing the exact likelihood estimates. soft SIC (SSIC) or soft PIC (SPIC). If we use successive up-

badcCG %

With likelihood deﬁned by dates i.e. update one at a time we can prove that the bound %

(18) is made tighter in each update of . Since the updates

¡¢¥¤ ¦ / / © / "£6 ¦

"£6 %

ba;cIGY¡ ¢7¤ T© / " ¦&¨ badcCG

/ of and are convex given this also tightens the bound.

(17)

<0PR The result being a Generalized EM algorithm .

¥UWVYX

$ 2

For this to be true both steps have to do hill-climbing towards the true £ This is contrary to the Bayesian approach where we only have hidden vari- likelihood, but not necessarily going to a minimum in each step. The proof is ables and visible variables, the ﬁrst including the parameters. left out due to limited space.

0 8

B. Linear Response Correction to Channel Estimate where the diagonal matrix 8 is deﬁned as

% 8 The ML estimate in (14) and (15) requires knowledge of .

! Now we can estimate

1 1

¨s¨ 4

both ¨%4 and . In the previous section these were ap-

¢ ¦

proximated by taking averages with respect to . This gave

! !

¢ ¦ ! ¢%1% ¦

8 8 8

< = ? @#B

¨%¨ 4¥698 : - 4

a naive approximation to ¨%¨ due to the fact that the cor- (27)

¢ ¦

relations are neglected when ¨ is chosen to factorise. A

1 4 correction to the estimate of ¨%¨ can be found by using the

which can be used directly in (14) and (15) to improve the

" 6

following identity for the second cumulant

estimate of and .

! !

8 8

98 8 98 8

1 1 1

¨ ¨ 4 ¨ ¨ 4

4 C. Complexity of the Approximate Algorithms

1 (22)

¨ 4

¨ " ¡

/ The updates in equation (12) and (21) have the same com-

¢7¤ ¦

! 8 8

¢ ¦

plexity of order ] per bit. The updates for in equation

¦ ¢ ¦ ¢

] 2 ]

Proof: (14) has a complexity of for and per bit

¡ for . The linear response corrected estimate requires

¢ ¦

# 1

¨ ¨

¨ 4

6 6

¨ " "

] due to the fact that at each bit instant we have to invert

¡ ¡

/ /

¢7¤ ¦ ¢¥¤ ¦

8 8 8 8

! a matrix.

= ? @ B

¨ ¨ ¦

6 8k:

6 "

¨ VI. SIMULATIONS AND DISCUSSION

¢7¤ ¦

! 8 98

98 8

# the correlations of the signature waveforms, here calculated on

¨ ¦ ¨ ¨

¡¢¥¤ / © / " ¦

6 the basis of random PN-sequences, with chips. The chan-

nel coefﬁecients are all constructed to have unit length and

¦ 6 ¦ 6

8 8

# ¨ - #

¦ ¦

¨ ¨ ¨ ¨

random phase, the block length is ﬁxed to 2 . For each

¡¢7¤ / © / " ¦

6 6

block simulated new random sequences are generated. We de-

! !

8 8 8 8

1 1

1 ﬁne the system load as the ratio of the number of users to the

¨ ¨ 4 ¨ 4 ¨ 4 ¢

spreading gain .

1 ¨ But since we don’t want to calculate 4 , we employ the

We use the three proposed detectors as follows: We ﬁrst following approximation

make an initial guess on by using preamble bits, then

< = ? @ B

1 we use this to get an initial guess on

¨ 4¥6 8 :

¡ ¡

/ /

(23)

¢¥¤ ¦ ¢7¤ ¦

! 8 8 ! 8 8

or ¨ corresponding to either joint-ML

or ML algorithms. We calculate the initial noise variance by which is called the linear response correction [8]. equation (11). Then we process two stages of each algorithm,

We now do the partial derivative on with successive updates in decending power order.

- In ﬁgure (1) we present Monte Carlo studies of the algo-

-!"

¢ !

% ¢ ¤ ¦ ¨ ( NH*)¢

rithms. Every point is simulated until bit errors were

6 ¨#

- 6

reached, giving an accuracy on BER of .

(24)

¢§¢ In the left plot of ﬁgure (1) we have simulated the Bit Er-

¦£ £§£ £

£ ror Rate of user where SNR is the corresponding signal

to noise ratio for ¨ users and the number of chips

$ from (21) to yield the approximation (23) for the second cu- 2 . The conventional detector fails to estimate the

mulant transmitted bits due to the high SIR produced by the non-

orthogonal signature waveforms. A rutine calculation shows

8 8

)

%2v

¥¤ ¦

¢ that , where we can use

8 8

6 on average when using random PN-sequences. This implies

3 3

( (

%2v

which for the given case yields a bit error rate

%Qv

/ of in the absence of noise. At a BER of

¦£ £§£ 8 98 8

/*0

- ? ¨ -!

$21

" 6

, linear-response-ML detector gains ¨ compared

to naive-ML detector and ¨ compared to pseudo-joint-ML

(25)

detector. The loss down to the single user bound is around 3

8 8

Isolating we get dB.4

- Strictly this is an optimistic error bar, since the bit errors inside a block

! $ &

8 8

5 6

8 probably are correlated due to the estimate of and but at the desired

} 3

v (26)

BER= 798!: we mainly observed single errors.

K=20 α=K/N =1.2−1 c 20

−1 10

−2 1 b 10 P SNR α=K/N =1.2−1 14 c P =10−3 b

−3 10

4 6 8 10 12 14 16 18 20 8 10 15 20 25 30 35 40 45 50 SNR

1 K

¦8 ¥§¦¨¡¤£ ©

Fig. 1. Left plot: BER for user 1 for the three proposed methods, the single user bound, conventional, and the initial detector, at ¢¡¤£ and . Right

} E

798 : ¡ ¡ 7 £!: plot: SNR of user 1 to obtain a BER ¡ for different number of users at a constant load . Conventional detector: solid-dot, Initial detector: solid with round markers, pseudo-joint-ML: dotted, naive-ML: solid, linear-response-ML: dashed, and Single user bound: dash-dot.

The plot to the right shows the performance for a ﬁxed sys- important for the ML channel estimate.

¨ -C\ $

tem load i.e. fixed spectral efficiency and fixed

/*0

¨ -! 1 BER $ . At ﬁrst sight it seems a bit contra intu- ACKNOWLEDGEMENTS itive that the performance is increased for increasing , the

The authors would like to thank Professor Dr. Bernard

SIR - so we could expect the performance to be equal for

Fleury, Aalborg University Denmark and Dr. Niels Mørch, all . But in the derivation of the algorithms we have implic- Nokia Denmark for their carefull reading of the manuscript. itly assumed that the MAI can be approximated by a gaussian variable, i.e. using the Central Limit theorem. The conclusion REFERENCES is that MAI looks more and more gaussian for increasing num-

[1] Sergio Verdu,´ Multiuser Detection, Cambridge University Press, The Pitt ber of users and hence performance is increased for fixed Building, Trumpington Street, Cambridge, CB2 1RP United Kingdom, 1998. load . As expected, due to the soft tentative decission, naive- [2] M. Varanasi, B Aazhang, “Near-optimum detection in synchronous code- ML perform better than pseudo-joint-ML for all , but it is division multiple-access systems,” IEEE Trans. Communication, vol. 39, hard to quantize whether this performance gain is due to the no. 5, pp. 725–735, May 1991. better channel estimate or the soft vs. hard tentative decission. [3] Wang Xiaodong, H.V. Poor, “Blind multiuser detection: a subspace approach,” IEEE Trans. on Information Theory, vol. 44, pp. 677–690, Mar. For the linear-response-ML vs. the naive-ML we see that we 1998. gain up to 2 dB, this because of the improved channel estimate, [4] M. F. Bugallo, J. Mí guez, L. Castedo, “A Maximum Likelihood Ap- since the tentative decisions in both cases are soft. In future proach to Blind Multiuser Interference Cancellation,” IEEE Trans. On Signal Processing, vol. 49, no. 6, pp. 1228–1239, June 2001. work will we examine the dependence on 2 and the near-far [5] A. Kocian, B.H. Fleury, “Iterative joint symbol detection and channel resistance. estimation for DS/CDMA via the SAGE algorithm,” in Proc. 11th IEEE International Symposium on Personal, Indoor and Mobile Radio Commu- VII. CONCLUSION nications, 2000 (PIMRC'2000)., 2000, vol. 2, pp. 1410–1414. [6] P. A.d.F.R. Højen-Sørensen, O. Winther, L. K. Hansen, “Ensemble Learn- In this contribution three approximative detectors for joint ing and Linear Response Theory for ICA,” in In Advances in Neural Infor- symbol-channel estimation are derived. One approximating mation Processing Systems 13 (NIPS'2000)., T. K. Leen, T. G. Dietterich, V. Tresp, Ed. 2001, pp. 542–548, MIT Press. the joint-ML detector, and two approximating the ML detec- [7] J. Joutsensalo, T. Ristaniemi, “Learning algorithms for blind multiuser tor for the channel. The empirical results shows a bennefit detection in CDMA downlink ,” in Proc. 9th IEEE International Sym- of using approximations to ML for the channel instead of the posium on Personal, Indoor and Mobile Radio Communications, 1998 (PIMRC'1998)., 1998, vol. 3, pp. 1040–1044. approximating joint-ML detector. Furthermore can it be con- [8] Giorgi Parisi, Statistical Field Theory, Addison-Wesley Publishing Com- cluded that the cross-correlations between individual bits are pany, New York, 1994.