Low-Density Parity-Check Codes : Unequal Error Protection And

ISSN: 1402-1544 ISBN 978-91-86233-XX-X Se i listan och fyll i siffror där kryssen är

DOCTORAL T H E SIS Sara Sandberg Sandberg Sara

Department of Computer Science and Electrical Engineering Division of Systems and Interaction

Low-Density Parity-Check Codes - Codes Low-DensityParity-Check ISSN: 1402-1544 ISBN 978-91-86233-14-3 Low-Density Parity-Check Codes -

Luleå University of Technology 2009 Unequal Error Protection and Reduction of Clipping Effects Unequal Error Protection and Reduction of Clipping Effects Clipping of Reduction and Protection Error Unequal

Sara Sandberg

Luleå University of Technology

Low-Density Parity-Check Codes - Unequal Error Protection and Reduction of Clipping Eﬀects

Sara Sandberg

Division of Systems and Interaction Department of Computer Science and Electrical Engineering Lule˚a University of Technology Lule˚a, Sweden

Supervisor:

Professor James P. LeBlanc Tryck: Universitetstryckeriet, Luleå

ISSN: 1402-1544 ISBN 978-91-86233-14-3 Luleå www.ltu.se To Marcus, Miriam, and Jakob iv Abstract

The invention of low-density parity-check (LDPC) codes made reliable communication possible at transmission rates very close to the theoretical limit predicted by Shannon. However, communication close to the Shannon limit requires very long codes and results in long delay and high encoder and decoder complexity. In many communication scenarios, constraints on delay, complexity and power prohibit communication with arbitrarily low error probability. To achieve good performance it is then important that the code is appropriately matched to the other parts of the communication system. In this thesis, LDPC codes for two different communication scenarios are studied. A common scenario is communication of information bits with unequal importance for the perceptual quality after source decoding. This is the case for example in many networks and for transport of multimedia data, where one frame may consist of a header, some payload and additional payload for increased quality. Errors in the header data may cause the whole frame to be useless, while errors in the additional payload generally cause only a small quality reduction. A code with unequal error protection (UEP) is designed to protect some bits more than others, thus providing a reduced bit-error rate (BER) for the important bits. This work studies design of LDPC codes with UEP capability for bandwidth-efficient higher order constellations. A flexible design algorithm for irregular UEP-LDPC codes is proposed, which is applicable to arbitrary signal constellations, an arbitrary number of classes of bits with different importance and arbitrary sizes of the classes. Simulations using 8-PSK modulation show that the overall BER is reduced if codes are properly designed for the modulation scheme, compared to the BER achieved by standard UEP codes designed for BPSK modulation. Codes designed by the proposed algorithm also provide more UEP capability, especially at high SNR. Moreover, further work shows that the UEP capability of an irregular LDPC code is not only dependent on the variable node degrees as is widely believed. The LDPC construction algorithms, that place the edges in the graph according to the degree distributions, also play a critical role for the UEP behavior of an LDPC code. The differences in UEP capability are explained by introduction of detailed check node degree distributions that describe differences in the code structure.

v vi

LDPC codes for the orthogonal frequency division multiplexing (OFDM) system are also studied. OFDM enables simple equalization and has been adopted in several standards. However, OFDM is sensitive to frequency-selective fading and introduces a large peak-to-average power ratio (PAPR) of the transmitted signal. These problems can be alleviated by pre-multiplying the OFDM block with a spreading matrix that both reduces the PAPR of the transmitted signal and increases the frequency diversity. Simulation of an OFDM system with clipping shows that the performance gain by spreading is substantial also when an LDPC code, which on its own improves the performance signiﬁcantly, is applied to the OFDM system. PAPR reduction may also be achieved by deliberate clipping of the signal, prior to the transmitter high-power ampliﬁer. Clipping will however introduce losses and receiver methods to mitigate such clipping losses are investigated. We consider Bayesian estimation of the unclipped signal as well as statistical characterization of the clipping distortion, that is fed to the LDPC decoder. The results show that for an LDPC coded OFDM system, the improvement by these clipping mitigation methodsisminimal. Contents

Part I - General Introduction xiii

Chapter 1–Thesis Introduction 1 1.1Motivation...... 1 1.2ADigitalCommunicationSystem...... 3

Chapter 2–Error Control Coding 9 2.1HistoricalDevelopment...... 9 2.2LinearBlockCodes...... 11 2.3ConvolutionalCodes...... 20 2.4TurboCodes...... 24 2.5CodedModulation...... 27 2.6PerformanceComparison...... 29

Chapter 3–Low-Density Parity-Check Codes 33 3.1 Fundamentals of LDPC Codes ...... 33 3.2DecodingofLDPCCodes...... 36 3.3DensityEvolution...... 40

Part II - Introduction to the Speciﬁc Research Topics 47

Chapter 4–Overview of Clipping Mitigation Strategies in OFDM 49 4.1OFDMBasics...... 49 4.2ReductionofthePeak-To-AveragePowerRatio...... 52 4.3 Reduction of Clipping Eﬀects in the Receiver ...... 54 4.4SpreadOFDM...... 55 4.5CodedOFDM...... 56

Chapter 5–Overview of Unequal Error Protection Codes 59 5.1UEPCodes...... 60 5.2UEP-LDPCCodes...... 62 5.3IrregularUEP-LDPCCodes...... 63

vii viii

Chapter 6–Research Contributions 67 6.1ClippingMitigationinLDPCCodedOFDMSystems...... 67 6.2DesignofUnequalErrorProtectionLDPCCodes...... 69

Part III - Research Papers 87

Paper A-Performance of LDPC Coded Spread OFDM with Clipping 89 1 Introduction...... 91 2 TheOFDMSystem...... 92 3 LDPCcodesforOFDMandSOFDM...... 95 4 Results...... 97 5 Conclusions...... 100

Paper B-Receiver-oriented Clipping-effect Mitigation in OFDM - A Worthy Approach? 105 1 Introduction...... 107 2 SystemDescriptionandChannelModel...... 108 3 CharacterizationofClippingNoise...... 110 4 BayesianEstimation...... 112 5 ResultsandDiscussion...... 113 6 Conclusions...... 115

Paper C-Design of Unequal Error Protection LDPC Codes for Higher Order Constellations 119 1 Introduction...... 121 2 SystemModel...... 122 3 Modulation...... 123 4 UEP-LDPCCodes...... 124 5 SimulationResults...... 131 6 Conclusions...... 134

Paper D-Design of Bandwidth-Efficient Unequal Error Protection LDPC Codes 137 1 Introduction...... 139 2 SystemModel...... 143 3 UEP-LDPCCodesforHigherOrderConstellations...... 147 4 SimulationResults...... 154 5 Conclusions...... 160 ix

Paper E-On the UEP Capabilities of Several LDPC Construction Algorithms 165 1 Introduction...... 167 2 ConstructionAlgorithms...... 169 3 SimulationResults...... 171 4 RelevantGraphProperties...... 175 5 ModiﬁedPEG-ACEConstructionwithIncreasedUEPCapability.... 184 6 Conclusions...... 185 x Acknowledgements

The first person I would like to express my gratitude to is my supervisor Professor James LeBlanc. Thank you for convincing me that I would find the Ph.D. studies fun and for the guidance and support that you have given me. Because of your enthusiasm and dedication to my education, becoming a Ph.D. student is a choice I have never regretted. Also, many thanks go to my assistant advisor, Professor Bane Vasic from the Univer- sity of Arizona, that has supported me with his coding expertise and interesting ideas. Thanks also for giving me the opportunity to visit you and your group for three months and the hospitality you all showed me and my family. Magnus Lundberg Nordenvaad has been my assistant advisor since 2007. Thank you for your valuable support. During the M-Pipe project, I got to know Neele von Deetzen from Jacobs University Bremen, Germany. Since then we have written three papers together. I have really enjoyed our discussions and collaboration, from which I think both of us have learned a lot. Thanks also for being a good friend. It has been a pleasure to get to know you Neele! I would also like to thank all my colleagues at the department of computer science and electrical engineering. All together you make up a friendly and inspiring atmosphere that makes it enjoyable to go to work. I would especially like to thank my friends in the A3200-corridor for joining me in the coffee room and Johan Carlson for providing the LATEX template for this thesis. Acknowledgments also to the European Commission for co-funding this work, that is part of the FP6/IST project M-Pipe, and to the PCC++ graduate school. Finally, I would like to express my sincere gratitude to my husband Marcus. Thank you for everything!

Sara Sandberg Lule˚a, February 2009

xi xii Part I

General Introduction xiv Chapter 1 Thesis Introduction

1.1 Motivation

Error control coding is a critical part of many of the tools everyone uses today, for example mobile phones, CDs, DVDs and hard drives. The main idea of error control coding is to add redundancy to the information bits, which makes error detection and error correction possible. Coding theory started with the pioneering work of Shannon in 1948, [1]. He developed fundamental limits on the efficiency of communication over noisy channels and his channel coding theorem states that if the signaling rate is less than the channel capacity, arbitrarily low error probabilities can be achieved by applying error control codes. Since then, the focus of much coding research has been to find good encoding and decoding techniques to make reliable communication possible at rates close to the capacity. With the invention of iterative decoding algorithms for code structures well suited for this type of decoding, the performance achieved by practically usable codes took a huge step closer to the performance limit. In this work, the focus is on low-density parity-check (LDPC) codes, which are described by a very sparse parity-check matrix. LDPC codes are decoded by a simple iterative decoding algorithm and the performance of these codes approaches the Shannon limit. The question then arises, why bother with coding research when it is already possible to achieve arbitrarily low error probabilities for rates very close to the theoretical limit? One answer is that there are still many error control applications where an arbitrarily low error probability is a luxury or not even possible. Coding research is of importance to improve performance in such applications. To achieve capacity, Shannon proposed infinitely long random codes. However, in practice the length of the code is limited by delay constraints and complexity issues. Power constraints and bad channel conditions can also degrade the error probability and make reliable communication difficult.

1 2 Thesis Introduction

Many communication systems involve transmission of bits of unequal importance, for example transmission of source coded video or speech, or transmission over a network where protocol data is more important than the payload. If reliable transmission of all data can not be achieved, it is desirable that the important bits have more redundancy than less important bits and thus are reconstructed with less errors. A code with unequal error protection (UEP) provides such differentiation in the error protection. In this work we study how to design LDPC codes with UEP for bandwidth-efficient higher order modulations. We also investigate how different LDPC construction algorithms affect a code’s capability of providing UEP. The differences in UEP capability are explained by the introduction of detailed check node degree distributions, that describe differences in the code structure. A specific communication technique considered in this work is the orthogonal frequency division multiplexing (OFDM) system. The main advantage of OFDM is that it converts a frequency selective channel into a number of flat subchannels that are easily equalized. However, the drawback is that the signal to be transmitted usually has a very high peak-to-average power ratio (PAPR), that causes problems in the transmitter high- power amplifier. An LDPC coded OFDM system is considered and we study spreading (also called linear precoding) as a way to reduce the PAPR and increase the frequency diversity. Simulations show that there is a significant gain due to spreading also when LDPC codes are applied, especially in systems with clipping in the high-power amplifier. A simple way to reduce the high PAPR is deliberate amplitude clipping prior to the high-power amplifier, which causes clipping distortion. Two different methods for reducing the deleterious effects of clipping in the receiver have been investigated. However, the study suggests that in an LDPC coded OFDM system there is little to be gained by these receiver-oriented methods. The thesis is divided into three parts. Part I (including Chapters 1, 2 and 3) contains a general introduction to digital communication and error control coding. In the next section, we take a step back and describe the context of the two communication scenarios by presenting a typical digital communication system. Chapter 2 gives a brief history of coding theory and presents both linear block codes, convolutional codes, turbo codes, and coded modulation. Since the focus of this work is on LDPC codes, they are covered in more detail in Chapter 3. Part II (including Chapters 4, 5 and 6) focuses on the two research topics covered in this work. In chapter 4, the OFDM system is described and the benefits and drawbacks of this modulation scheme are discussed. An overview of state of the art methods to reduce the PAPR and mitigate clipping effects is presented. UEP codes are considered in Chapter 5, where both a brief history and state of the art 1.2. A Digital Communication System 3

information codeword Source bits Channel bits Information Modulator encoder encoder

Channel Linear filter

Noise

Source Channel Destination Demodulator decoder decoder

Figure 1.1: Block diagram of a typical digital communication system. of UEP code design are presented. Chapter 6 presents the main results of this work and summarizes the appended papers. Finally, Part III contains the appended research papers.

1.2 A Digital Communication System

A simplified model of a digital communication system is shown in Figure 1.1 and the different blocks are briefly described below. There are many textbooks on the subject of digital communication that the reader can refer to for a thorough introduction, e.g. [2–4]. The information source is assumed to be in digital form and the data is usually encoded by a source encoder to achieve data compression. We assume that the source coded information bits are equally likely 0 or 1. They may also be of unequal importance for the perceptive performance of the overall communication system. The information bits are fed to the channel encoder, which introduces redundancy in the information sequence. This redundancy can be used by the channel decoder to reduce the impact of channel effects such as noise and interference. The result is increased reliability of the received data. In this work, the main focus is on channel coding, that is, on the design and implementation of channel encoders and decoders. The codeword bits from the channel encoder are passed to the digital modulator, which maps the bits to appropriate signal waveforms. The simplest form of modulation is to map each binary zero to one waveform and each binary one to some other waveform that is easy to distinguish from the waveform representing the zero. This is called binary modulation and is one form of single-carrier modulation. In the last decade, multi-carrier 4 Thesis Introduction modulation has become more popular and especially OFDM has received much attention. In multi-carrier systems, the superposition of several waveforms representing several bits are transmitted in each time interval. The communication channel model represents the physical medium that connects the transmitter with the receiver. This medium can be the atmosphere as well as wire lines, optical fibers, etc, but it could also be a storage medium. All received waveforms will be more or less corrupted due to thermal noise from electronic devices, non-linear distortion in the high-power amplifier, interference from other transmissions, atmospheric noise, fading, etc. Two different channel models are mainly used in this work. The simplest model is the additive white Gaussian noise (AWGN) model, where the noise, denoted by w, is a Gaussian random process with one-sided power spectral density N0 and the linear filter in the block diagram of Figure 1.1 is assumed to be only a unity gain. The 2 2 noise samples are independent and belong to the distribution N (0,σw), where N (μ, σ ) denotes the Gaussian distribution with mean μ and variance σ2.Theprobability density function (pdf) of the AWGN is 1 w2 fw(w)= exp − . (1.1) 2 2 2πσw 2σw This is a commonly used channel model, since AWGN is present in any communication system. The second model considered is the linear filter channel model, which is characterized as a linear filter with AWGN. The linear filter is often time-varying. The demodulator transforms the channel-corrupted transmitted waveforms to a sequence of numbers that represents estimates of the transmitted data symbols. These numbers can be either a binary sequence of hard decisions or it can be numbers quantized to more than two levels (or even continuous values), so called soft decisions.Hard decisions are often used because of the resulting simplicity of the demodulator and the channel decoder. Soft-decision decoding is more difficult to implement, but it offers a significant performance improvement compared to hard-decision decoding. The demodulated signal (also called the received sequence) is passed to the channel decoder, which uses the redundancy added by the channel encoder to reconstruct the information sequence. With more redundancy added, the decoded output signal is more likely to equal the transmitted information sequence. Finally, the source decoder attempts to reconstruct the original information from the output of the channel decoder and knowledge of the source encoding method. The following examples will concretize the concepts of modulation, transmission over an AWGN channel, and error correcting coding.

Example: Phase Modulation In phase modulation, usually called phase-shift keying (PSK), the phase of the carrier signal is modulated and the choice of the carrier 1.2. A Digital Communication System 5

0 00 1 11

10 BPSK (M =2) QPSK (M =4) 011

010 001

000 110

100 111 101 8-PSK (M =8)

Figure 1.2: Constellation diagrams for BPSK, QPSK and 8-PSK. phase conveys the information. In M-PSK, M distinct signal waveforms with different phases are assigned unique patterns of bits. They are represented by 2π sm(t)=g(t)cos 2πfct + (m − 1) m =1,... ,M , (1.2) M where g(t) is the signal pulse shape. The signal waveforms may be represented as linear combinations of two orthonormal basis functions (see for example [2] for more details). Each signal waveform can be determined by the coefficients of these basis functions, which correspond to a point in the plane. Figure 1.2 shows constellation diagrams for M-PSK (M =2, 4 and 8). If two different signal waveforms are considered (M=2), the modulation is called binary PSK (BPSK) and the constellation is one-dimensional. With four different signal waveforms (M=4), the scheme is called quadrature PSK (QPSK) and corresponds to a two-dimensional constellation. Each point in an M-PSK constellation corresponds to a signal waveform with a distinct phase. In this example, the mapping of bits to signal waveforms is chosen according to Gray encoding, so that the most likely errors will only result in a single bit error. M-PSK carries log2(M) bits of information in each signaling interval. 6 Thesis Introduction

fr|0 sent(r)

0.4

0.3

0.2

0.1

r 0 Eb

−0.1 −2 −1 0 1 2 3 4 5 6 7

Figure 1.3: Probability density function of the received value, given that a 0 was transmitted using BPSK. The shaded area corresponds to the probability of bit error.

Example: Uncoded BPSK Transmission in AWGN We consider uncoded BPSK transmission in AWGN, since this is a benchmark that any coded system is typically compared to. Two signal waveforms, both with energy Eb, correspond to the transmission of√ 0 and 1, respectively. The Euclidean distance between the two constellation points is 2 Eb, [2].√ Assuming that a 0 is transmitted and that the 0 is mapped to the constellation point + Eb, the received signal is r = Eb + w. (1.3)

2 The AWGN w belongs to N (0,σw), that is, it has a Gaussian distribution with zero 2 mean and√ variance σw = N0/2. The received signal, given that a 0 is sent, thus belongs 2 to N ( Eb,σw). Figure 1.3 shows the pdf of r given that a 0 was sent. The decision threshold is chosen to be zero and we assume that a 0 was sent if r>0andthata1was sent if r<0. The area of the shaded region in the ﬁgure corresponds to the probability that r<0 and a 1 is erroneously assumed to be sent. The total probability of bit error is an average of the probability of bit error given that a 0 was sent and the probability of bit error given that a 1 was sent. Assuming that both bit-values are equally likely and are transmitted using the same energy, the total probability of error equals the probability of error given that a 0 was sent. Thus, the total probability of bit error is given by 0 Pe = fr|0 sent(r)dr . (1.4) −∞ 1.2. A Digital Communication System 7

This probability can be written as 2Eb Pe = Q , (1.5) N0 where the Q function 1 ∞ y2 Q(x)=√ exp − dy (1.6) 2π x 2 is the integral of the unit-variance zero-mean Gaussian pdf. The probability of error is reduced if the signal-to-noise ratio (SNR) given by Eb/N0 is increased, which can be achieved by increasing the transmitted energy Eb.

Example: Error Correction by a Repetition Code Let us now consider a very simple error-correcting code, the repetition code. The idea is straightforward. Instead of transmitting only a single 0 or 1, the information bit is repeated a prearranged number of times, for example three, in order to reduce the probability of bit error after decoding. Imaginethatwetransmittheinformation1001011usingthelengththreerepetition code. Each information bit is mapped to a codeword by the encoding rule 0 → 000 and 1 → 111. The codewords 000 and 111 can be seen as a concatenation of the information bit and two redundant bits, called parity bits, which equal the information bit. The codeword bits are transmitted and hard decisions on the received bits, which are affected by noise, are fed to the decoder. The decoder considers one received word at a time and attempts to correct any bit errors by finding the closest codeword, which differs from the received word in the minimum number of bits, through majority voting. If two or three bits in a received word of length three are 0, it is assumed that the codeword 000, corresponding to the information 0, was sent. On the other hand, if two or three bits in a received word are 1, it is assumed that a 1 was sent. An example transmission is given in Table 1.1. The table shows 7 information bits and their corresponding codewords. The received words, which are affected by noise, are also given. The estimated information bits, which are the output from the decoder, are given in the last column. Because of the two added parity bits, a length three repetition code has the ability to correct one error or detect two errors in each received word. As seen in the example transmission, the third received word contains one bit error, which is corrected. Received word 6 contains two bit errors, which can be detected but not corrected. The decoder detects errors in the received word, but assumes by majority voting that a 0 was sent. 8

Table 1.1: An example transmission using the repetition code of length three. Codeword Information Codeword Received Decoder number bit word output 1 1 111 111 1 2 0 000 000 0 3 0 000 010 0 4 1 111 111 1 5 0 000 000 0 6 1 111 001 0 7 1 111 111 1

The repetition code can reduce the probability of bit error, but it is not for free. The cost is a loss in the information transfer rate. Instead of transmitting only one bit, three bits need to be transmitted, which consumes three times more power. Therefore, for a code to be useful, it needs to perform better than uncoded transmission using the same power per information bit. In this case, it means that uncoded transmission of one bit should be compared to transmission of three codeword bits with only one third of the power for each bit. It turns out that the very simple repetition codes are not a good choice and more advanced codes need to be considered.

The overall aim of a digital communication system is to transmit as much information as possible from the transmitter side to the receiver side with as few errors as possible. The performance measure of the digital communication system is the average frequency or rate with which errors occur (the bit-error rate) and the rate with which information can be transmitted. Shannon showed that reliable communication can not be achieved at rates greater than the channel capacity [1]. For continuous AWGN channels, the capacity C in bits per second is defined by [5] S C = B log2 1+ , (1.7) BN0 where B is the bandwidth in hertz and S is the signal power. The capacity increases monotonically with increasing SNR (S/N0). Thus, for a fixed bandwidth, the capacity increases with an increase in the transmitted signal power. On the other hand, if the signal power is fixed, the capacity can be increased by increasing the bandwidth. Chapter 2 Error Control Coding

This chapter reviews some of the most important developments of error control codes. A brief historical overview is given in Section 2.1. Section 2.2 introduces linear block codes and discusses three important linear block codes in particular. Convolutional codes are considered in Section 2.3 and turbo codes in Section 2.4. We also brieﬂy introduce coded modulation in Section 2.5. While the focus of this thesis is concentrated on LDPC codes, a review of history allows us to view the progress of development from simple low- performing codes to today’s advanced iterative coding schemes. By introducing some well-known classes of codes, important concepts of error control coding are revealed. It is also interesting to see how the focus of research has shifted over the years.

2.1 Historical Development

Research in the field of error control coding started with the pioneering work of Claude Shannon in 1948 [1] on reliable communication over noisy transmission channels. He demonstrated that with proper encoding and decoding techniques the errors induced by a noisy channel can be reduced to any desired level without sacrificing the rate of information transmission, as long as the information rate is less than the capacity of the channel. However, the proof is non-constructive, since it does not give explicit encoders and decoders that can achieve this theoretical limit. Shannon’s results initiated large efforts by many researchers in design of good codes and practical decoding algorithms. Hamming codes, which belong to the class of linear block codes, were proposed already in the late 1940s, [6]. During the 1950s and 1960s, much coding research focused on developing the theory of efficient encoders and decoders. Especially linear block codes and convolutional codes,

9 10 Error Control Coding

ﬁrst introduced by Elias [7] in 1955, gained much interest. Gallager invented low-density parity-check (LDPC) codes already in 1962 [8], but his work was largely forgotten until the rediscovery of LDPC codes in the mid 1990s. LDPC codes are discussed in detail in Chapter 3.

After a number of years of coding research for binary-input channels, research in the 1970s and 1980s lead to new bandwidth-eﬃcient coding schemes. In fact, in the early 1970s it was believed that coding gain could only be achieved through bandwidth expansion and that coding would therefore not be a good solution for strongly bandwidth- limited communication systems [9]. A bandwidth-eﬃcient combination of coding and modulation, called coded modulation, was proposed by Ungerboeck and Csajka in 1976. Ungerboeck extended the idea further in [10–12]. Another coded modulation scheme was proposed around the same time by Imai and Hirakawa [13].

In the 1990s, a period of much interest in concatenated codes lead to the discovery of turbo codes. Berrou et al. [14] combined parallel concatenation including an interleaver with iterative decoding by soft-in soft-out (SISO) decoders. Simulations revealed the amazing performance of this coding scheme, which could approach the Shannon limit within less than one decibel. A few years after the discovery of turbo codes, Gallager’s LDPC codes were reinvented independently by MacKay and Neal [15] and Wiberg [16]. However, the ideas of iterative decoding were present already in Gallager’s original work. If simulation techniques had been more advanced when Gallager presented his work, the history of coding theory could have been very diﬀerent.

Research in the 2000s has lead to new methods for analyzing iterative coding schemes, like turbo and LDPC codes. Ten Brink [17] proposed a technique to aid the construction of good iteratively-decoded error-correcting codes, called extrinsic information transfer (EXIT) chart. Around the same time, Richardson and Urbanke [18] developed a general method for determining the thresholds of LDPC codes, called density evolution. The density evolution algorithm was simpliﬁed by Chung et al. [19], who introduced a Gaussian approximation of the message densities.

A detailed overview of the historical development of coding theory can be found in [20]. In the remaining sections of this chapter, some important classes of codes are introduced. For more details the reader is referred to textbooks in error control coding, e.g. [9,21,22]. 2.2. Linear Block Codes 11

2.2 Linear Block Codes

Let the row vector u =[u1 u2 ... uk]oflengthk denote a block of information bits, called the information word. There are, in total, qk different information words, where q ≥ 2is the size of the alphabet. In the following, only binary codes with q = 2 are considered and the arithmetic will be done modulo-2. The encoder maps each information word to adistinctcodewordv =[v1 v2 ... vn], represented by a row vector of length n,where n ≥ k. In other words, while there are 2n possible bit patterns, only a small subset of 2k bit patterns are selected to be codewords. Such a code with length n and 2k codewords is called an (n, k) block code. The n − k redundancy bits, called parity bits, that are added to the information word are determined by linear functions of the original k information bits. A binary block code is linear if and only if the sum of any two codewords (modulo-2) is also a codeword. The code rate is defined by the ratio R = k/n, which is bounded by 0 ≤ R ≤ 1. The mapping from an information word to a codeword may be performed by multiplication with a generator matrix. The generator matrix G is a k × n matrix, which specifies k vectors from which all codewords can be built. The encoding process can be represented by v = u · G. (2.1)

If the generator matrix consists of a k × k identity matrix followed by a k × (n − k) matrix, it has the special form corresponding to a systematic code. The codewords of a systematic code consist of the corresponding information word followed by n − k parity bits. A linear block code may also be speciﬁed by its parity-check matrix H of size (n − k) × n. The parity-check matrix is deﬁned such that any row in G is orthogonal to any row in H,thatis, G · HT = 0, (2.2) where 0 represents a zero vector or matrix of suitable size. An n-tuple v is a codeword in the code generated by G if and only if v · HT = 0.

Example: The (7,4) Hamming Code The (7,4) Hamming code is a linear block code of length n = 7, which transmits 7 codeword bits for every block of k =4in- formation bits. Assuming that the code is in systematic form, each codeword consists of four information bits followed by three parity bits, that is, [v1 v2 v3 v4 v5 v6 v7]=

[u1 u2 u3 u4 v5 v6 v7]. The parity bits v5,v6 and v7 are determined by linear functions of the information bits. These linear functions may be described by the generator matrix or 12 Error Control Coding

1 v5 u1 v6 00

u3 1 u2 u4 00

v7 1

a) b)

Figure 2.1: Graphical representation of encoding for the (7,4) Hamming code. by the graphical description shown in Figure 2.1, where the codeword bits are arranged in three intersecting circles. The parity bits should be chosen such that the (modulo-2) sum of all bits within each circle is 0 , that is, the parity bit should equal the parity of the information bits included in each circle. From Figure 2.1a), the encoding rule for a (7,4) Hamming code is found to be

v5 = u1 + u2 + u3

v6 = u1 + u3 + u4

v7 = u2 + u3 + u4 . (2.3)

As an example, Figure 2.1b) shows the encoding of the information u =[1010].The parity bits should be v5 =0,v6 =0,andv7 = 1, so that the sum of bits within each circle is zero. This yields the codeword v =[1010001].Now,assumethatthiscodeword is transmitted over a noisy channel and received as the word r =[1110001].Itis easy to verify that r does not correspond to a valid codeword, that is, the sum within each circle is not equal to zero and it does not fulfill the rules given by (2.3). The (7,4) Hamming code is capable of correcting one bit error and we may ask the question: is it possible to find one unique bit that lies inside all circles with bits not summing to zero, and outside all circles with bits summing to zero? If so, this bit is probably in error and the error may be corrected by flipping this bit. Stronger codes, with higher error-correcting capabilities, may be constructed by considering longer codewords. In this case, it is easier to represent the codes by their generator and parity-check matrices. The basic ideas of encoding and decoding remains the same nonetheless. The generator matrix corresponding to the graphical representation 2.2. Linear Block Codes 13 of encoding shown in Figure 2.1 is ⎡ ⎤ 1000110 ⎢ ⎥ ⎢ 0100101⎥ G = ⎢ ⎥ . (2.4) ⎣ 0010111⎦ 0001011

2.2.1 Decoding of Linear Block Codes

Let r =[r1 r2 ... rn] denote hard decisions on the received values at the output of the channel. The received vector r may be diﬀerent from the codeword v and the vector sum

e =[e1 e2 ... en]=r + v (2.5) is called the error vector. The easiest way to decode a linear block code is by syndrome decoding. When a vector is received, the decoder calculates the syndrome

T s =[s1 s2 ... sn−k]=r · H . (2.6)

The syndrome s = 0 if and only if r is a codeword. If r is a codeword, it is assumed that the information word corresponding to r was transmitted. If s = 0, it is desired that the decoder ﬁnds the codeword that diﬀers from r in the minimum number of bits. Since r = v + e and v · HT = 0, the syndrome can be written

s = e · HT . (2.7)

Among all possible error vectors that solve this system of equations, the most probable error vector is chosen as the true error vector. For binary symmetric channels the most probable error vector is the vector with the smallest number of ones. Thus, syndrome decoding corresponds to maximum-likelihood (ML) decoding, since the most likely codeword is found by the decoder. There are 2n−k possible distinct syndromes and each syndrome is mapped to the lowest weight error vector solving (2.7). However, since the number of distinct syndromes grows exponentially with n − k,anexhaustivesearchis practically impossible for long codes. Already for n − k = 20, the number of distinct syndromes are more than one million. 14 Error Control Coding

2.2.2 Error Detecting and Correcting Capabilities

The number of nonzero digits in a sequence is called the (Hamming) weight of the sequence. The minimum (Hamming) distance, denoted by dmin, of a linear block code is deﬁned as the smallest number of bits by which any two codewords diﬀer. It can be shown, for a linear code, that the minimum distance is equal to the weight of the minimum weight codeword (excluding the all-zero codeword). The minimum distance is connected to a code’s capabilities of detecting and correcting random errors.

Considering a block code with minimum distance dmin, no error vector with weight less than dmin can change one codeword into another. All such errors can be detected. If the error vector has weight equal to dmin, there exist at least one pair of codewords that diﬀers only by dmin bits. An error vector with non-zero digits in exactly these positions can not be detected, but if the code is long there may still be many other detectable error vectors of weight dmin or more. Let t denote the error correcting capability of the code, that is, assume that the code can correct any error vectors of weight less than or equal to t.Itcanbeshownthat dmin − 1 t = . (2.8) 2

Syndrome decoding maps each distinct syndrome to the lowest weight error vector corresponding to that syndrome. The number of correctable error patterns thus equals the number of distinct syndromes. For a t-error correcting code it is required that all possible error vectors of weight t or less are fewer or equal to the number of distinct syndromes, that is, n n n 1+ + + ...+ ≤ 2n−k . (2.9) 1 2 t Many block codes with error correction capability t can also correct some error vectors of weight higher than t. The only exception is the so-called perfect codes, where all errors of weight t or less can be corrected but any error of higher weight is mapped to another codeword. For perfect codes, the inequality (2.9) becomes an equality. During the development of linear block codes the focus of many researchers has been to search for codes of a given length that have the largest possible distance, which implies that a large number of errors can be guaranteed to be corrected. However, many codes can in fact correct a large fraction of error vectors with much higher weight than t. MacKay [5] stated that the minimum distance of a code, although widely worshiped by coding theorists, is not of fundamental importance to Shannon’s mission of achieving reliable communication over noisy channels. He also explained that state of the art 2.2. Linear Block Codes 15 decoders that approach the Shannon limit need to correct errors of much higher weight than t.

2.2.3 Important Linear Block Codes

Three important linear block codes are brieﬂy introduced in this section: Hamming codes, cyclic codes, and binary BCH codes.

Hamming Codes

Hamming codes [6] with codeword length n =2m − 1andk =2m − 1 − m information bits exist for all integers m ≥ 3. All Hamming codes have a minimum distance dmin =3, which implies that every Hamming code can correct one error or detect two errors. The parity-check matrix contains binary vectors describing the decimal numbers 1 to n as columns in any order. The systematic parity-check matrix may be written in matrix partitioned form as

H =[PIm] , (2.10) where P contains all length m binary vectors of weight two or more and Im denotes the identity matrix of size m × m. The generator matrix is given by T G = Ik P . (2.11)

Hamming codes are perfect codes and fulﬁll (2.9) with equality.

Example: The (15,11) Hamming Code A linear block code of length n =15 and k = 11 has the parity-check matrix ⎡ ⎤ 110100111011000 ⎢ ⎥ ⎢ 101010110110100⎥ H = ⎢ ⎥ . (2.12) ⎣ 011001101110010⎦ 000111011110001

Since the last four columns equal the identity matrix, the code is in systematic form. Also, all distinct vectors of length four (except the zero vector) are found as columns in H and therefore the code is a Hamming code. The ﬁrst 11 columns may be arranged in any order and the parity-check matrix will still describe a systematic Hamming code. The BER performance of this code is shown in Figure 2.7. 16 Error Control Coding

Cyclic Codes

Cyclic codes are popular due to their special structure that simpliﬁes the logic required for encoding and decoding and their ability to detect bursts of errors, that is, a number of consecutive errors in an otherwise error-free word. A code is cyclic if a cyclic shift of any codeword is also a codeword. All codewords in a cyclic code can be generated from one special codeword by shifting and addition (since the code is cyclic and linear). The special codeword that is used to generate the other codewords is usually called a generator sequence. By treating the bits in a codeword v =[v1 v2 ... vn] as the coeﬃcients of a polynomial, any codeword can be written as a code polynomial

2 n−1 v(X)=v1 + v2X + v3X + ...+ vnX . (2.13)

The generator sequence is denoted by a generator polynomial g(X). The cyclic shift of the generator sequence that gives the polynomial with the lowest degree is typically chosen as the generator polynomial. The lowest possible degree of the generator polynomial is n − k. Any cyclic codeword can be considered to be the product of the generator polynomial and some polynomial representing the information content of the codeword, which we denote by u(X). This multiplication v(X)=g(X)u(X) also corresponds to convolution of the sequences g =[g1 g2 ... gn−k+1]andu =[u1 u2 ... uk]. Non-systematic encoding can be accomplished by a linear shift register with n−k stages and feedback connections, while systematic encoding requires a few more steps. Decoding of cyclic codes consists of the same three steps as decoding of any linear block code; compute the syndrome, associate the syndrome with an error vector and correct the error by addition of the presumed error vector to the received vector. The syndrome of a systematic cyclic code can easily be calculated by dividing the received polynomial r(X) by the generator polynomial g(X). The remainder s(X) is the syndrome. The division can be accomplished eﬃciently by a shift register with feedback, similar to the encoder circuit. The mapping from syndrome to error vector may be performed by a lookup-table, but the structure of the cyclic code can also be used to implement a simpliﬁed logic at the cost of speed. Cyclic codes are often used for error detection and are especially good at detecting bursts of errors. Since the syndrome is calculated through division of the received polynomial by the generator polynomial, the error polynomial must have the generator polynomial as a factor to give a zero syndrome, corresponding to undetected errors. Thus, any error bursts of weight less than n − k (the degree of the generator polynomial) are detected. A commonly used cyclic redundancy check (CRC) code has a generator poly- 2.2. Linear Block Codes 17 nomial of degree 12 and codeword length n = 2047, of which 2035 bits carry information. It has minimum distance 4, but can detect any error bursts of length 11 or less.

Binary BCH Codes

Bose-Chaudhuri-Hocquenghem (BCH) codes were invented by Hocquenghem in 1959 and independently by Bose and Chaudhuri [23] in 1960. BCH codes are a class of cyclic codes and may be specified by a generator polynomial g(X). The generator polynomial is determined by its roots and can be constructed as the least common multiple of the minimum polynomials of the specified roots. The roots are chosen in such a way that the columns of the parity-check matrix have a high degree of linear independence, which gives a large distance, while also ensuring a good code rate. The following description of BCH codes assumes that the reader is somewhat familiar with finite fields and finite field arithmetics. Tutorials on this subject can be found in almost any textbook on error control coding, see for example [9,22]. Fix a finite field GF (2m). There exist binary BCH codes with length n =2m − 1and n − k ≤ mt parity bits. Let α be a primitive element in GF (2m), such that the powers m of α, α0 to α2 −2, form all the nonzero elements of the field. The generator polynomial of a t-error correcting BCH code is the lowest-degree polynomial that has the 2t roots

αm0 ,αm0+1, ..., αm0+2t−1 . (2.14)

A common choice is m0 = 1, which is also assumed here for simplicity. These roots of the generator polynomial will be denoted consecutive roots for short. Note that Reed-

Solomon codes are a special class of non-binary BCH codes, with m =1andm0 =1. The BCH codes can also be represented by their parity-check or generator matrix. The parity-check matrix may be derived in a straightforward way from the roots of the generator polynomial. If γ is a root of the generator polynomial it is also a root of any of the codeword polynomials, that is,

2 n−1 v(γ)=v1 + v2γ + v3γ + ...+ vnγ =0. (2.15)

The consecutive roots are roots of the generator polynomial and therefore also roots of any codeword polynomial v(X). Each root corresponds to one parity-check equation like (2.15), which can be combined to give the parity-check matrix. If there are j roots 18 Error Control Coding

Table 2.1: Binary polynomial representation of GF (16). Element Value Element Value 0 0000 α7 1011 α0 0001 α8 0101 α1 0010 α9 1010 α2 0100 α10 0111 α3 1000 α11 1110 α4 0011 α12 1111 α5 0110 α13 1101 α6 1100 α14 1001

γ1, ...,γj, the parity-check matrix may be written ⎡ ⎤ T T T 2T ··· n−1 1 γ1 γ1 γ1 ⎢ T ⎥ ⎢ T T 2T ··· n−1 ⎥ ⎢ 1 γ2 γ2 γ2 ⎥ H = ⎢ . . ⎥ . (2.16) ⎣ . . ⎦ T T T 2T n−1 1 γj γj ··· γj

It is not necessary to include all conjugate roots in the parity-check constraints, since v(γ) = 0 also implies that v(γ2) = 0 and so on. The roots may be considered simply as elements in GF (2m), but they may also be given a binary representation. The binary representation is typically a binary row vector, which needs to be transposed to a column vector when inserted in (2.16). The following example explains how to construct a BCH code and the corresponding parity-check matrix.

Example: Single-error correcting BCH code We construct a code with the parameters m =4,t =1,andn =2m − 1 = 15. The code is defined in GF (16), which is constructed from the minimal polynomial p(X)=X4 +X +1. Thus, a primitive element α of the field must fulfill α4 +α+1 = 0 and anypower of α can through this constraint be rewritten as a polynomial of degree three or less. The binary polynomial representation of the powers of α in GF (16) is given in Table 2.1, where the binary value corresponds to the coefficients of a degree three polynomial in α. Polynomial representations of the finite fields GF (2m), 3 ≤ m ≤ 10 can be found in [9, Appendix A]. The consecutive roots of this single-error correcting code are α1 and α2.Theseroots have the conjugate roots α4 and α8, since the next conjugate root α16 = α1 is already 2.2. Linear Block Codes 19 included. The generator polynomial is

g(X)=(X + α1)(X + α2)(X + α4)(X + α8) (2.17)

= X4 + X +1. (2.18)

This code has 4 parity-checks, since the degree of the generator polynomial is 4, and defines a (15, 11) cyclic block code. The parity-check matrix is defined by (2.16) and the consecutive roots α1 and α2.Sinceα2 is a conjugate root of α1, only the root α1 is needed to define H. The binary representations of the powers of α from Table 2.1 are inserted into (2.16) and we get ⎡ ⎤ 000100110101111 ⎢ ⎥ ⎢ 001001101011110⎥ H = ⎢ ⎥ . (2.19) ⎣ 010011010111100⎦ 100010011010111

From the parity-check matrix we see that this is a cyclic (15, 11) Hamming code. In fact, cyclic Hamming codes are just single-error correcting binary BCH codes.

The decoding of binary BCH codes can be divided into three main steps.

1. Calculate the syndrome of the received sequence r.

2. Calculate the error locator polynomial.

3. Calculate the roots of the error locator polynomial, to get error location positions.

The syndrome can be calculated by evaluating the received polynomial at all consecutive roots. If there are no errors, we have r(αi) = 0 for all consecutive roots αi, 1 ≤ i ≤ 2t. Again, it is not necessary to check all conjugate roots. If the syndromes are nonzero, the locations of the errors may be found through the error locator polynomial, at least if there are not more than t errors. Assume that there are l ≤ t errors in positions j1, ...,jl.Itcanbeshownthat

i i i j1 i jl Si = r(α )=v(α )+(α ) + ...+(α ) , 1 ≤ i ≤ 2t, (2.20) where v(αi) = 0. Solving the syndrome equations of (2.20) is the main part of the BCH decoding and any method for solving these equations is a BCH decoding algorithm. t Let Λ(X)=1+λ1X +...+λtX denote the error locator polynomial. The polynomial is deﬁned by Λ(X)=(1+αj1 )(1 + αj2 ) ···(1 + αjl ) (2.21) 20 Error Control Coding and the roots of the error locator polynomial are α−j1 , ...,α−jl . The coeﬃcients of Λ(X) can be determined from the equations (also called the Newton’s identities)

St+i + λ1St+i−1 + ...+ λt−1Si+1 + λtSi =0, 1 ≤ i ≤ t. (2.22)

When the coeﬃcients have been determined, the roots of Λ(X) are calculated and the errors can be corrected. The calculations in step 2 of the above decoding algorithm may be performed by Berlekamp’s iterative algorithm.

2.3 Convolutional Codes

Convolutional codes were introduced by Elias in 1955, [7]. They are diﬀerent from block codes in the sense that the encoder contains memory. The encoder output at a given time is therefore dependent on a number of previous information bits.

2.3.1 Encoding

The encoder of a convolutional code is easily implemented by a number of shift registers.

ArateR = k0/n0 convolutional encoder processes a block of k0 information bits at a time to produce a block of length n0, called a frame, that is one part of the code sequence. Let M denote the encoder memory order, that is, the maximum number of shift register stages in the path to any output bit. The memory constraint length, denoted by μ, deﬁnes the total number of shift register stages in the encoder. The encoding operation can be described by generator polynomials, where one polynomial represents the generation of one output bit from one information bit. For example, a simple convolutional code of rate R =1/2(n0 =2,k0 = 1) has the generator polynomials

g(1)(D)=D2 +1, (2.23)

g(2)(D)=D2 + D +1, (2.24) where D can be interpreted as a delay operator. The generator polynomials represent the impulse response of the encoder. Given an information sequence u =[u1,u2,u3,...], (1) (1) (1) (1) (2) (2) (2) (2) the two output sequences v =[v1 ,v2 ,v3 ,...]andv =[v1 ,v2 ,v3 ,...]can be obtained as the convolution of u with the two generator polynomials. The encoder described by the above generator polynomials is non-recursive and does not employ feedback in the encoding operation. The code is also non-systematic. Such an encoder is commonly referred to as a non-systematic convolutional (NSC) encoder. An- other common type of encoder is the recursive systematic convolutional (RSC) encoder. 2.3. Convolutional Codes 21

0/00 0/00 0/00 0/00 00 0/11 0/11 0/11 0/11 1/11 1/11 1/11 1/11 01 1/00 1/00 1/00 1/00 0/10 0/10 0/10 0/10 10 0/01 0/01 0/01 0/01 1/01 1/01 1/01 1/01 11 0/10 0/10 0/10 0/10

Figure 2.2: Trellis diagram of the rate 1/2 convolutional code described by the generator polynomials (2.23) and (2.24). The states are written on the left and the input/output bits corresponding to each transition are written close to each edge in the graph.

The generator polynomials of a RSC encoder are rational functions in D with binary coefficients. Every recursive systematic encoder can be transformed into a non-systematic non-recursive encoder. Both encoders generate the same code, that is, the same set of code sequences. The difference lies in the mapping of messages to code sequences. In particular, weight 1 information sequences are mapped to finite weight sequences by the NSC encoder, but to sequences of infinite weight by the RSC encoder.

2.3.2 Decoding

The decoding of a convolutional code is conveniently described by a code trellis. The trellis contains all 2μ possible states on the vertical axis and uses time on the horizontal axis to show the possible paths through the states. The edges in the trellis correspond to possible state transitions. Figure 2.2 shows the trellis for the generator polynomials given in (2.23) and (2.24). The states are written on the left and the input/output bits corresponding to each transitionarewritteninthegraph. Convolutional codes with short constraint length are generally decoded by the Viterbi algorithm or the BCJR algorithm. These algorithms are both optimal, but according to diﬀerent criteria. The Viterbi algorithm is a maximum-likelihood (ML) decoding algorithm, that is, the decoder selects the codeword that maximizes the conditional probability of the received sequence. Thus, decoding by the Viterbi algorithm leads to the lowest possible codeword error rate. On the other hand, the BCJR algorithm is a maximum a posteriori probability (MAP) algorithm, which maximizes the a posteriori probability that an information bit is correctly decoded, given the received sequence. 22 Error Control Coding

Therefore, the BCJR algorithm minimizes the probability of information bit error. If longer constraint lengths are desired, a suboptimal sequential decoding algorithm may be used instead of the Viterbi or BCJR algorithm. In sequential decoding, only the most likely paths through the trellis are stored. The decoder considers only extensions of these paths, which reduces the decoding complexity considerably. Two commonly used sequential decoding algorithms are the Fano algorithm [24] and the stack algorithm [25]. Similar to block codes, convolutional codes have a concept of distance that determine the error-correcting capabilities of the code. The most important distance measure for convolutional codes is the minimum free distance (dfree), that defines the minimum number of bits in which any two different paths differ.

The Viterbi Algorithm

ML decoding of a convolutional code implies that the received sequence should be compared with every possible code sequence, that is, the sequences corresponding to every possible path through the trellis. Fortunately, Viterbi [26] realized how to simplify the ML decoding without sacriﬁcing performance. In principle, the Viterbi algorithm relies on the fact that a path to a certain state that is non-optimal in one stage can never be optimal in the future. This means that only one path reaching each state needs to be stored. The complexity of the decoder is thereby largely reduced, since the number of states is only 2μ. In general, Viterbi decoding is employed for memory constraint lengths in the range 2 ≤ μ ≤ 10 [22].

The steps of the Viterbi algorithm performed for each received frame (n0 bits) are:

1. For each path (one incident to each state) at the start of the frame, compute the

distance between the received frame and the n0 bits corresponding to each of the 2k0 extensions of the path. This distance, for example calculated as the Hamming distance, is called the branch metric.

2. For each of the states at the end of the frame, construct the 2k0 paths which terminate in that state. Calculate the sum of branch metrics (the overall path metric) for each of those paths. Store the path with the lowest overall path metric. If several paths have the same metric, one of them is chosen arbitrarily.

Usually, any non-recursive encoder is cleared after each sequence by transmission of μ zeros, such that the encoder shift register ends in the zero state. For recursive encoders, the clearing data depends on the state of the encoder. If the encoder is cleared, the decoder chooses the path with the lowest metric, that also ends in the zero state, as the 2.3. Convolutional Codes 23 optimal path. The information bits corresponding to the optimal path may be stored during calculation of the path metrics or they can be found by following the optimal path through the trellis. If the received sequence r contains soft decisions (possibly quantized values) instead of hard decisions, the Viterbi algorithm performs better. A metric that takes the soft information into account should then be used instead of the Hamming distance.

The BCJR Algorithm

A MAP decoding algorithm for convolutional and linear block codes was proposed by Bahl et al. [27] and called the BCJR algorithm after its inventors. In the same paper, the authors also pointed out that linear block codes could be given a trellis representation. The complexity of the BCJR algorithm is however significantly higher than the complexity of the Viterbi algorithm, while giving only a minimal BER reduction. Due to the high complexity, the BCJR algorithm initially received very little attention. Later, with the invention of turbo codes (see Section 2.4), the MAP decoder gained much interest as a component decoder in the iterative turbo decoding scheme. The main ideas of the BCJR algorithm are presented in this section and the reader is referred to [28] for a good tutorial and to [9] for more details. We consider a binary-input continuous-output AWGN channel. Assume that the bits vi of the code sequence are represented by BPSK signals through the mapping vi =0→ +1 and vi =1→−1. The received sequence r is real-valued. The BCJR algorithm calculates the log a posteriori probability (LAPP) ratio of the code sequence bits, defined by P (vi =+1|r) L(vi)=log . (2.25) P (vi = −1|r) The decoder makes a decision on the bits by 0ifL(vi) > 0 vî = . (2.26) 1ifL(vi) < 0

By incorporating the code structure through the trellis, (2.25) may be written [28] ⎛ ⎞ P (si− = s ,si = s, r)/P (r) ⎜ 1 ⎟ ⎜ S+ ⎟ L(vi)=log , (2.27) ⎝ ⎠ P (si−1 = s ,si = s, r)/P (r) S−

+ where si is the state of the encoder at time i, S is the set of state transitions (s ,s) − caused by an information bit ui = 0 (mapped to +1) and S is the set of state transitions 24 Error Control Coding

(s ,s) caused by an information bit ui =1(mappedto−1). The probabilities in (2.27) are determined by a recursive calculation similar to the Viterbi algorithm. The main diﬀerence is that when two paths merge, a joint metric for arriving at the state by either route is calculated. The following steps summarize the BCJR algorithm.

1. Forward recursion: Compute a forward metric for each state at all time instances in the trellis.

2. Backward recursion: Compute a backward metric for each state at all time instances in the trellis.

3. Find bit values: For each time i in the trellis and for each transition in S−,add the forward metric to the metric of state s, and add the transition metric and the backward metric to the metric of state s. Do the same calculation for each transition in S+. These metrics are combined by (2.27) to ﬁnd the bit value.

2.4 Turbo Codes

Turbo codes, ﬁrst presented in 1993 by Berrou et al. [14], have exceptionally good performance and vastly outperformed previously known codes. Two fundamental ideas were combined in the development of turbo codes. The authors considered a parallel concatenation of two convolutional codes, separated by a pseudorandom interleaver, and they also introduced an iterative decoding scheme for this parallel concatenation. The result is an almost capacity-achieving code with moderate decoding complexity.

2.4.1 Encoding of Turbo Codes

Turbo codes are encoded by a parallel concatenation of convolutional component encoders. Figure 2.3 shows a schematic description of a parallel concatenated encoder. The information bits u are encoded by a component encoder to create a sequence of parity bits p1. The information bits are also interleaved by a random interleaver Π and encoded by a second component encoder, to produce a second sequence of parity bits p2. The function of the interleaver is to decorrelate the inputs to the component encoders, so that the parity bits are independent. The two component encoders are often identical. The information bits and the two parity sequences are multiplexed to create the codeword v. Recursive systematic convolutional (RSC) component codes have been shown to perform much better than non-recursive codes in turbo coding schemes. Typically, the RSC encoders have rate R =1/2 and then the parallel concatenated encoder has rate 2.4. Turbo Codes 25

information u

p RSC encoder 1 v =[up1 p2] Π

p2 RSC encoder

Figure 2.3: A parallel concatenated encoder.

R =1/3. If a higher rate is desired, puncturing of the parity bits (a number of bits are periodically deleted) to reduce coding overhead is often employed. The component encoders generally have a small memory constraint length μ. Good performance at moderate BERs is achieved with μ = 4 or less, [9]. To achieve performance close to the Shannon limit the interleaver should be large, typically several thousand bits or more. Pseudorandom interleavers give the best performance. Note that the interleaver makes the number of possible states in the overall code extremely large. Therefore, it is not possible to decode the parallel concatenation by trellis-based ML or MAP algorithms.

2.4.2 Turbo Decoding

Turbo codes are decoded iteratively and the bit level soft decisions calculated by one component decoder are used to improve the soft decisions calculated by the second component decoder. The best performance is achieved if a MAP decoder, e.g. the BCJR algorithm, is used as component decoder. The iterative decoding algorithm to be described here is suboptimal, but achieves near-ML performance nonetheless.

Using Bayes’ rule, the LAPP ratio L(vi) from (2.25) can be written P (r|vi =+1) P (vi =+1) L(vi)=log +log . (2.28) P (r|vi = −1) P (vi = −1)

The second term represents aprioriinformation about the codeword bits. The bit-values are typically equally likely, resulting in zero aprioriinformation. However, in an iterative decoding scheme, the first decoder outputs soft information regarding the information bits u,whichservesasaprioriinformation for the second decoder. In the same way, the first decoder receives soft information from the second decoder. The soft information that one decoder passes to the other is extrinsic information, meaning that one decoder only passes information that is not available to the other decoder. The first decoder 26 Error Control Coding

Π−1

parity sequence 2 parity sequence 1 a priori information + BCJR + BCJR Π − decoder − decoder

information

Figure 2.4: Iterative decoding of turbo codes, [22].

passes information about u that comes from the parity sequence p1, while the second decoder passes information coming from the parity sequence p2.TheLAPPL(vi)after decoding may be written as a sum of three terms,

L(vi)=Lchannel + Lapriori+ Lextrinsic , (2.29)

where Lchannel corresponds to the received channel log-likelihood ratio (LLR), Lapriori is the aprioriLLR and Lextrinsic is the extrinsic part of the LAPP. The extrinsic information passed to the next decoder can be calculated by subtracting Lchannel and Lapriori from

L(vi). Figure 2.4 describes the iterative decoding scheme for turbo codes. In the first iteration, the aprioriinformation to the first BCJR decoder is zero, since it is assumed that the bit-values are equally likely. The first decoder uses the channel LLR values of the information bits and parity sequence 1 to calculate LAPPs of these bits. The input to the first decoder is subtracted from its output LAPPs of the information bits to produce extrinsic information, which is aprioriinformation to the second decoder. The apriori LLRs and the channel LLRs of the information bits are interleaved by the interleaver Π to arrange the information bits in the right order. The second decoder calculates LAPPs from the aprioriLLRs and the channel LLRs of the information bits and parity sequence 2. The input to the second decoder is subtracted to produce extrinsic information. This information is deinterleaved and fed to the first decoder. The steps described here con- stitute one decoding iteration. The iterative decoding can be interrupted after a given number of iterations (typically on the order of 10-20) or when the LAPP estimates are assumed to be sufficiently reliable. The hard decisions on the information bits are found from the last estimate of LAPPs. 2.5. Coded Modulation 27

2.5 Coded Modulation

It is well-known that coding combined with binary modulation requires bandwidth expansion by a factor of 1/R, if the information rate should be the same as for uncoded transmission. By a combination of coding and modulation, called coded modulation, it is possible to achieve coding gain without bandwidth expansion. Coded modulation involves signal set expansion rather than bandwidth expansion. In principle, coded modulation applies more redundancy to bits in a symbol that are more likely to be in error, due to the mapping of symbols to the constellation points. Coded modulation may be implemented by convolutional codes, called trellis coded modulation, or by linear block codes, called block coded modulation.

2.5.1 Trellis Coded Modulation

Ungerboeck and Csajka introduced trellis coded modulation (TCM) in 1976, but the method was not widely reckoned until the publication of Ungerboeck’s more detailed follow-up paper [10] in 1982. Ungerboeck has also summarized the general principles of TCM in [11,12]. The main idea of TCM is to find a good convolutional code for higher order modulations by maximizing the minimum squared Euclidean distance (instead of Hamming distance) between code sequences. To achieve a large minimum squared Eu- clidean distance, the parity checks are applied on a per-symbol basis instead of applied to the bitstream before modulation. This allows construction of codes with a minimum Eu- clidean distance between code sequences that significantly exceeds the minimum distance between uncoded modulation signals, at the same rate, bandwidth and signal power. Ungerboeck’s approach to map the symbol values to the constellation points was to use set partitioning of the bit values within a symbol. For example, consider the 8- PSK constellation shown in Figure 2.5, where the symbols are labeled by binary vectors d =[d3 d2 d1]. Let bit d1 correspond to the principal set partition, with bits from adjacent constellation points separated into different sets. This step may be seen as a conversion from one 8-PSK constellation into two QPSK constellations, emphasized by dotted lines connecting their elements. Bit d2 partitions the closest points in each of these sets into different subsets, which corresponds to further partitioning of each QPSK constellation into two BPSK signal sets. Finally, bit d3 determines the point within the subset. Note that in this case the set partitioning mapping corresponds to natural mapping, which assigns labels in the order of their integer equivalents. After set partitioning, a convolutional code is designed to protect the set partition. The first TCM schemes were designed in a heuristic manner, but later Ungerboeck de- 28 Error Control Coding

000

111 001

010

110

011 101 d d 100 3 d2 1

Figure 2.5: 8-PSK constellation with set partitioning. duced a general structure of multilevel encoders followed by mapping of symbols to constellation points by set partitioning. This general structure allowed a computer search for the best TCM schemes and Ungerboeck showed improvements in the order of 6 dB compared to uncoded QPSK.

Example: Trellis Coded 8-PSK This example considers Ungerboeck’s ﬁrst TCM scheme for 8-PSK [11]. A rate R =2/3 TCM scheme is designed that can easily be compared with uncoded QPSK, since both systems allow transmission at the same rate and bandwidth. Consider the 8-PSK constellation with set partitioning shown in Figure 2.5.

Bit d3 is left uncoded, since the Euclidean distance between two constellation points that diﬀer only in bit d3 is the largest possible within this constellation. A rate R =1/2 convolutional code determines the two other bits that have smaller uncoded Euclidean distance. This coded modulation scheme achieves a 3 dB coding gain compared to uncoded QPSK.

2.5.2 Block Coded Modulation

Block coded modulation (BCM) considers a combination of block coding and channel signal sets to construct bandwidth-eﬃcient codes. Imai and Hirakawa [13] were the ﬁrst to consider BCM in 1976. They proposed a multilevel coding technique to construct BCM codes and a multistage decoding algorithm for these codes. The block diagram of a multilevel encoder for a modulation scheme with 2lm symbols is shown in Figure 2.6.

The information bits are partitioned into lm blocks with lengths corresponding to the 2.6. Performance Comparison 29

u1 v1 E1

u2 v2 u v partition E2 signal of information mapper

ulm vlm Elm

Figure 2.6: Block diagram of an encoder for multilevel signal transmission.

possibly diﬀerent rates of the component encoders E1 to Elm . The information blocks u1, ...,ulm are encoded by the component encoders and each component code Ci is a (n, ki) linear block code. The signal mapper combines one bit from each of the codewords v1, ...,vlm to one symbol with lm bits. Short BCM codes may be decoded by ML decoding algorithms based on the code trellis. Multilevel codes may also be decoded by a multistage decoding procedure that provides a good trade-oﬀ between error performance and decoding complexity. The multistage decoding algorithm decodes each component code with a soft-decision ML algorithm, one at a time. The decoded information from one component code is passed to the next decoding step. In principle, the Euclidean distance between possible words of one component decoder depends on the bits estimated in the earlier component decoders. Since errors may propagate from one component decoder to the next, the multistage decoding algorithm is suboptimal even though each component decoder is a ML decoder.

2.6 Performance Comparison

Reduced error probability may be achieved by adding redundancy to the information word, but adding redundancy implies that more bits need to be transmitted. For a fair comparison between codes of diﬀerent rates and uncoded transmission, the signal to noise ratio (SNR) is measured in energy per information bit over the one-sided noise power spectral density, that is, Eb/N0. Therefore, the beneﬁts of the code must at least com- pensate for the Eb/N0 loss, otherwise the overall performance is reduced. Typically, the bit-error rate (BER) or frame-error rate (FER) performance is plotted as a function of

Eb/N0. Coded communication systems can also be compared in terms of coding gain, 30 Error Control Coding

which defines the reduction of the Eb/N0 required to achieve a specific error probability, compared to uncoded transmission. When comparing different systems, it is also important to remember that a coded system with good performance generally requires a complex encoder and decoder. Figure 2.7 shows a comparison between the (15,11) Hamming code, an often used convolutional code with memory constraint length 6, and turbo and LDPC codes of different length. The BER of uncoded BPSK is also shown. The excellent performance of LDPC codes is shown here to motivate the more detailed description of these codes in Chapter 3. All codes except the (15, 11) Hamming code have rate R =1/2. The BER performance for LDPC codes and turbo codes of length 103,104 and 106 are taken from [29, Fig. 3]. The minimum required SNR to achieve reliable communication (the Shannon limit) with a coded system of rate R =1/2 over a continuous-output AWGN channel and BPSK modulation is 0.188 dB, [9]. The figure shows that the short Hamming code gives only a moderate coding gain −4 even for high Eb/N0 (around 1 dB at a BER of 10 ), compared to uncoded BPSK. The Hamming code is also the only code shown here that is decoded by hard decisions. In general, unquantized soft-decision ML decoding achieves an asymptotic coding gain of 3 dB compared to hard-decision ML decoding. The performance of the convolutional code is significantly better than the performance of the Hamming code. However, the introduction of turbo codes and LDPC codes revolutionized the coding performance. Even for short turbo and LDPC codes the BER performance is largely improved. Figure 2.7 shows that the coding gain of the LDPC code of length 103 is 6.5dBataBERof 10−4, compared to uncoded BPSK. This means that transmission at the BER of 10−4 can be achieved by LDPC codes of length 103 using less than 25% of the power needed to achieve the same BER in uncoded transmission. The turbo code outperforms the LDPC code for short codeword lengths. On the other hand, for very long codewords, the performance of the LDPC code comes very close to the Shannon limit and outperforms the turbo code. In fact, it can be shown that turbo codes are just special cases of LDPC codes [5, p. 581].

Figure 2.8 shows the general form of turbo and LDPC performance curves. At Eb/N0 above a certain value, the BER falls rapidly with increased Eb/N0. The range of Eb/N0 values that has this rapid reduction of BER is called the waterfall region. However, both turbo codes and LDPC codes have been shown to reach a region where the BER curve flattens, called the error floor region. Turbo codes usually reach the error floor at BERs around 10−5 −10−6 and the error floor typically arises from low-weight codewords. Appropriate interleaver design may reduce the number of low-weight codewords and thus 2.6. Performance Comparison 31

10−1

10−2 Uncoded BPSK

10−3 BER LDPC n =103 10−4 Convolutional

Hamming 3 Turbo n =10 (15,11)

−5 10 Turbo n =104 Shannon LDPC n =104 limit Turbo n =106 LDPC n =106 10−6 0 1 2 3 4 5 6 7 8 9 E /N (dB) b 0

Figure 2.7: Comparison between bit-error rates of several different codes. The figure shows the excellent performance of LDPC codes. All codes except the (15, 11) Hamming code have rate R =1/2. lower the error floor. LDPC codes also exhibit error floors, but generally they appear at lower BERs than for turbo codes. The error floor of an LDPC code has been shown to arise from certain combinatorial structures of the code, called trapping sets [30]. 32

waterfall region BER

error floor region

Eb/N0

Figure 2.8: General performance curve of turbo and LDPC codes. Chapter 3 Low-Density Parity-Check Codes

Low-density parity-check (LDPC) codes were originally discovered by Gallager in 1962 [8]. He introduced linear block codes speciﬁed by a very sparse parity-check matrix created in a random manner. The main contribution of Gallager was however a non-optimum but simple iterative decoding scheme for the LDPC codes, with complexity increasing only linearly with block length, that showed promising results. In the following years there were very few papers in this ﬁeld, but Tanner introduced the bipartite graph to this problem in 1981 and used it both for code construction and for decoding (with a generalization of Gallager’s iterative algorithm), [31]. Around 1996, LDPC codes were independently rediscovered by MacKay and Neal [15] and Wiberg [16]. Sipser and Spiel- man [32] are sometimes also mentioned as reinventors of Gallager’s work. In the last decade much research has been devoted to this area. However, the practical performance of Gallager’s original work from 1962 would have broken practical coding records up to 1993, [33]. A tutorial coverage of LDPC codes can be found for example in [34,35].

3.1 Fundamentals of LDPC Codes

LDPC codes are speciﬁed by a very sparse parity-check matrix, that is, the parity-check matrix should have a very small number of ones in each row and column compared to the codeword length. The codeword length and the number of information bits are denoted by n and k, respectively. The dimensions of the parity-check matrix are m × n,where m ≥ n − k denotes the number of parity bits and m equals n − k if the rows of H are linearly independent. The design code rate is R = k/n. As suggested by Tanner [31], an LDPC code can also be represented by a bipartite graph, called Tanner graph. The bipartite graph consists of two types of nodes and edges may only connect two nodes

33 34 Low-Density Parity-Check Codes of different types. One type of nodes, called variable nodes, corresponds to the bits of the codeword, while the other type, called check nodes, corresponds to the parity- check constraints. Every edge in the graph connecting a variable node with a check node corresponds to a one in the parity-check matrix. The m rows of the parity-check matrix specify the m check node connections, and the n columns of the parity-check matrix specify the n variable node connections. The number of edges connected to a node is called the degree of the node. LDPC codes where all nodes of the same type have the same degree are called regular LDPC codes, while LDPC codes where variable nodes and/or check nodes have different degrees are called irregular. Figure 3.1 shows an example of a Tanner graph for a regular LDPC code of length 10, with the parity-check matrix given by ⎛ ⎞ 1111000000 ⎜ ⎟ ⎜ ⎟ ⎜ 1000111000⎟ ⎜ ⎟ H = ⎜ 0100100110⎟ . (3.1) ⎜ ⎟ ⎝ 0010010101⎠ 0001001011 Note that the parity-check matrix given by (3.1) for the example code of length 10 is not sparse. In practice the codeword length ranges from at least a few hundred to several thousand bits or more, while the degrees of the nodes (corresponding to the column and row weights of the parity-check matrix) are only slightly higher than those shown in the example graph. Both the Tanner graph and the parity-check matrix give a full specification of one instance of an LDPC code on its own. By defining a degree distribution pair (λ(x), ρ(x)), an ensemble of LDPC codes may be described, [36]. The variable node and check node degree distributions are polynomials defined by

dvmax i−1 λ(x)= λix (3.2) i=2 and dcmax i−1 ρ(x)= ρix , (3.3) i=2 respectively. The maximum variable (check) node degree is denoted by dvmax (dcmax ). The coefficients of the variable (check) node degree distribution λi (ρi) represent the fraction of edges incident to variable (check) nodes of degree i. Since the degree distributions describe fractions of edges, the coefficients λi and ρi must be non-negative and the sum dvmax dcmax of the coefficients must equal one ( i=2 λi =1, i=2 ρi = 1). The regular code 3.1. Fundamentals of LDPC Codes 35

v1 c1 v2

v3 c2 v4 Variable Check nodes v5 nodes c3 v6

v7 c4 v8

v9 c5 v10

Figure 3.1: Bipartite graph describing a regular LDPC code of length 10. shown in Figure 3.1 is a realization of the code ensemble described by λ(x)=x and ρ(x)=x3. The number of variable nodes and check nodes of degree i are given by

λi/i n · (3.4) dvmax j=2 λj/j and ρi/i m · , (3.5) dcmax j=2 ρj/j respectively. The code rate may also be expressed by the degree distributions as dcmax n − m j ρj/j R = =1− =2 . (3.6) n dvmax i=2 λi/i Irregular LDPC codes have been shown to approach the Shannon limit closely, [29,37]. In the search for good irregular LDPC codes, it is typical to first search for an asymptotically good code ensemble. A specific code from this ensemble is then constructed according to the specified degree distributions. Usually, effort is spent to avoid short cycles in the bipartite graph, since cycles in the graph affects the performance under iterative decoding negatively. The length of the shortest cycle in the graph is called the girth. The construction algorithms for regular and/or irregular LDPC codes can be divided into two main categories: construction of random-like LDPC codes and construction of 36 Low-Density Parity-Check Codes structured LDPC codes. Random-like (pseudo-random) constructions include the first construction suggested by Gallager in [8] as well as [29,33,38–42]. Most random graphs are constructed under certain constraints on girth, degree distributions, etc. For example, the medium length codes constructed in [29] contain no cycles involving only degree-two variable nodes. In general, long random-like LDPC codes perform better than structured LDPC codes with comparable parameters. However, the encoding complexity is usually higher than for structured LDPC codes. A large amount of information is also required to specify the positions of the non-zero elements in a pseudo-random parity-check matrix. One drawback with LDPC codes has been the relatively high encoding complexity. While turbo codes can be encoded in linear time, LDPC encoding in general has quadratic complexity in the block length. However, Richardson and Urbanke showed that the constant factor in the complexity expression is very low, and therefore practically feasible encoders exist even for large block lengths, [43]. They also gave examples of optimized codes that can be encoded in linear time. Many algebraic and combinatorial constructions of structured LDPC codes have been suggested to reduce the encoding complexity and storage requirements, for example [44– 53]. Especially cyclic or quasi-cyclic LDPC codes are popular since the encoding can be implemented using simple shift registers and the complexity is linearly proportional to the number of parity bits of the code [44–49]. Algebraic constructions based on balanced incomplete block designs are presented in [51]. It has also been shown that for practical codeword lengths, well-designed structured LDPC codes can perform as well as random-like LDPC codes, [51]. An extensive reference list containing both random-like and structured LDPC codes is given in [53].

3.2 Decoding of LDPC Codes

The main advantage of LDPC codes is the simple iterative decoding algorithms that are available. LDPC codes may be decoded by a variety of decoding algorithms ranging from the low-complexity bit-flipping decoding algorithm proposed by Gallager, [8], to the more complex sum-product algorithm. The complexity of these decoding algorithms is linear in block length. In the following, we focus on the sum-product algorithm (SPA). It is very efficient for decoding LDPC codes, see for example [9, 33] or the tutorial paper on the sum-product algorithm written by Kschischang et al., [54]. The sum-product algorithm gives the same result as MAP decoding in the special case where the Tanner graph contains no cycles and the received symbols are independent of each other. However, sum-product decoding of LDPC codes in which the underlying graph has cycles, has also 3.2. Decoding of LDPC Codes 37 been shown to perform very well for long codes, see for example [15]. In short, the sum-product algorithm iteratively updates the a posteriori probabilities that a given bit in the codeword equals 0 or 1, given the received word. In the initialization each variable node is assigned the conditional probability of the codeword bit being a 0 or 1, given only the received value corresponding to that node. In order to calculate the conditional probabilities, the distribution of the noise must be known. The initial probabilities are then updated according to the sum-product rule to incorporate the structure of the code. In cycle-free graphs, the sum-product algorithm yields the correct a posteriori probabilities, that a given bit in the codeword equals 0 or 1 given the whole received word, when the algorithm terminates. The sum-product algorithm may be described either in the probability or the log- probability domain. The latter is usually chosen since it is numerically stable and may be more favorable for hardware implementation. Below, the sum-product algorithm in the log-probability domain is summarized. The conventional, flooding-schedule iterative decoder is considered.

Let v =[v1 v2 ... vn] be a codeword, where vi ∈{0, 1} for 1 ≤ i ≤ n.We v assume that the received channel samples are given by r = x + w,wherexi =(−1) i and w =[w1 w2 ... wn] are samples of white Gaussian noise with wi coming from the N 2 ≤ ≤ { }n { }m distribution (0,σw)for1 i n. Furthermore, let dv(i) i=1 and dc(j) j=1 denote the degrees of the variable nodes and check nodes, respectively, where dv(i) is the degree of the ith variable node and dc(j) is the degree of the jth check node. Figure 3.2 shows the message ﬂow through a variable node and a check node. The sum-product algorithm calculates approximations of the variable node LLRs P (xi =+1|r) Lv(i)=log . (3.7) P (xi = −1|r)

1. Initialization: Compute the channel bit LLRs for each variable node as Pr(xi =+1|ri) 2ri Lch(i)=log = . (3.8) 2 Pr(xi = −1|ri) σw

2. Check node update: Compute the messages from each variable node to all its neighboring check nodes. The message from variable node i to check node j is given by Lvc(i, j)=Lch(i)+ Lcv(j ,i) , (3.9) j= j 38 Low-Density Parity-Check Codes

where Lcv(j ,i) is the incoming message from check node j to variable node i.The

summation is over the dv(i) − 1 check node neighbors of variable node i, excluding check node j.

3. Variable node update: Compute the messages from each check node to all its neighboring variable nodes. The message from check node j to variable node i is given by −1 1 Lcv(j, i)=2tanh tanh Lvc(i ,j) , (3.10) 2 i= i

where Lvc(i ,j) is the incoming message from variable node i to check node j.The

product is over the dc(j) − 1 variable node neighbors of check node j, excluding variable node i.

4. Verify parity checks: Compute the LLR of each variable node as Lv(i)=Lch(i)+ Lcv(j, i) , (3.11) j

where the summation is over all dv(i) check node neighbors of variable node i.If

Lv(i) > 0, it is assumed that the corresponding codeword bit isv ˆi =0,otherwise T vˆi =1.Ifvˆ is a valid codeword, that is, vˆ · H = 0, the algorithm is halted. Otherwise, repeat the check node and variablenodeupdate(steps2-3)andverify the parity checks again, until a valid codeword is found or a preset maximum number of iterations is reached.

As described in step 4, the decoding of LDPC codes is typically stopped when a valid codeword is found. This stopping criterion may save computation time and gives a clear distinction of detected or undetected errors. Undetected errors occur if the decoder finds a valid codeword vˆ that is not equal to the true v. Detected errors occur if the algorithm runs for the maximum number of iterations (typically on the order of 100) without finding a valid codeword. In practice, undetected errors are rare. The error performance of an LDPC code under SPA decoding depends on several important parameters, [9]. The girth of the Tanner graph should be large enough so that short cycles are avoided. With many short cycles in the graph, the decoder may not converge to a codeword close to that found by the ML decoder. However, the girth should not be too large either. As shown in [55], codes with cycle-free Tanner graphs have a very poor tradeoff between rate and minimum distance for each fixed length. In [41], 3.2. Decoding of LDPC Codes 39

cj vi

Lvc(i, j) Lcv(j, i)

v c Lch(i) i j

dv(i) − 1 dc(j) − 1

Lcv(j ,i) Lvc(i ,j) 1 Lcv(j ,i) 1 Lvc(i ,j) dv(i)−1 dc(j)−1

Lcv(j2,i) Lvc(i2,j)

cj cj cj vi vi vi 1 2 dv(i)−1 1 2 dc(j)−1

a) b)

Figure 3.2: Message ﬂow through a) variable node i and b) check node j.

the authors argue that the negative effect of short cycles on performance is much less if the cycle is well connected, that is, if the cycle involves variable nodes of high degree. It has also been shown that the error-floor performance of the decoder is governed by a small number of likely error events related to certain structures in the Tanner graph, called trapping sets or near codewords, [30]. Known trapping sets have short cycles, and it is tempting to conjecture that avoiding short cycles is the best solution. However, it may be better to avoid the trapping sets directly in order to lower the error floors, [56]. Another parameter to consider is the minimum distance of the code. LDPC codes with a large minimum distance tend to have a low error floor. The number of minimum-weight codewords also affects the performance. Finally, the column and row weights of the parity-check matrix affect the error performance and the speed of decoding convergence. 40 Low-Density Parity-Check Codes

3.3 Density Evolution

LDPC codes exhibit a threshold for many channels and iterative decoders of interest, [8, 29, 36]. When the block length tends to infinity and the noise variance is less than a certain threshold, an arbitrarily small BER may be achieved. On the other hand, if the noise variance is above the threshold, the BER will be larger than a positive constant even if the number of decoder iterations approaches infinity. Richardson et al. used the expected behavior in the limit of infinitely long codes to determine the expected performance of LDPC codes, since it is difficult to analyze for codes of finite size, [18]. They developed an algorithm to iteratively calculate probability density functions (densities) of the messages, called density evolution. The algorithm may be used to analyze the performance of LDPC codes with given degree distributions by calculating their thresholds. Density evolution is applicable to memoryless binary-input output-symmetric channels [18], where the input is X ∈{±1} and the output Y depends only on the current input symbol and fulfills the symmetry condition

P (y|X =1)=P (−y|X = −1) . (3.12)

If these conditions are satisfied, the performance of the LDPC code is independent of the encoded data. For simplicity, the density evolution algorithm was derived assuming that the all-one codeword was transmitted, [18]. The density evolution algorithm was also derived under the assumption that the Tanner graph of the LDPC code is cycle-free, or at least has girth g>2l, if the performance after l decoder iterations is considered. A random graph can be considered to be cycle-free if the block length approaches infinity. Using the density evolution theory, Richardson et al. proved several important properties of infinitely long LDPC codes that are also very useful in practice when designing LDPC codes of finite size [18]:

n 1. Concentration: “Let Pe (l) be the expected fraction of incorrect messages which are passed in the lth iteration, where the expectation is over all instances of the code, the choice of the message, and the realization of the noise. For any δ>0, the probability that the actual fraction of incorrect messages which are passed in the lth n n iteration for any particular such instance lies outside the range (Pe (l)−δ, Pe (l)+δ) converges to zero exponentially fast in n.”

n ∞ 2. Convergencetocycle-freecase:“Pe (l) converges to Pe (l)asn tends to in- ∞ ﬁnity, where Pe (l) is the expected fraction of incorrect messages passed in the lth 3.3. Density Evolution 41

decoding round assuming that the graph does not contain cycles of length 2l or less.”

∞ 3. Density evolution and threshold determination: “Pe (l) is computable by a deterministic algorithm. Furthermore, there exists a channel parameter σ∗,the ∗ ∞ threshold, with the following property: if σ<σ then liml→∞ Pe (l)=0;if,onthe ∗ ∞ other hand, σ>σ then there exists a constant γ(σ) > 0 such that Pe (l) >γ(σ) for all l ≥ 1.”

The concentration statement tells us that all codes belonging to the same ensemble have almost the same performance. The important consequence is that for long codes it is enough to analyze the average behavior of the code ensembles of interest. However, for shorter LDPC codes the difference in performance is significant for different code realizations from the same ensemble [40–42]. Density evolution may be used to calculate thresholds and optimize degree distributions, but the computational complexity is very high (except for the binary erasure channel where density evolution becomes one-dimensional). However, Chung et al. presented an approximation of the density evolution algorithm [19], that is summarized in the following section. Since this approximation is generally employed in analysis of LDPC codes, a detailed description of the original density evolution algorithm is omitted here.

3.3.1 The Gaussian Approximation

The density evolution algorithm tracks densities of messages, representing each density by an infinite-dimensional vector. However, with only a relatively small loss in accuracy, the message density can be assumed to be a Gaussian density, simply represented by its mean and variance, [19]. Density evolution under the Gaussian approximation is easier to analyze and has much lower computational complexity than density evolution. It is therefore useful for the optimization of LDPC codes. With the Gaussian approximation, good irregular LDPC codes may be found using linear programming. The following derivation of density evolution under the Gaussian approximation for irregular LDPC codes follows that of [19]. Note that this derivation assumes that only the all-zero codeword is transmitted, corresponding to a sequence of ones after modulation. (l) Assume that the density of an LLR message Lcv from a check node to a variable node in the lth iteration is approximated by a Gaussian density specified by its mean (l) 2 (l) (mcv ) and variance ([σcv] ). For output-symmetric channels, all message densities must fulfill the symmetry condition f(x)=f(−x)ex,wheref(x) is the density of an LLR message, [29, Theorem 3]. This gives a relation between the mean and the variance, 42 Low-Density Parity-Check Codes

2 (l) (l) [σcv] =2mcv , which implies that only the mean of the messages needs to be tracked. (l) Furthermore, let mvc and mch denote the mean of an LLR message from a variable node to a check node, and the mean of the channel bit LLR, respectively. The density of an LLR message from a variable node to a check node in the lth iteration is approximated by a Gaussian mixture dvmax (l) (l) fvc(l)= λiN (mvc,i, 2mvc,i) , (3.13) i=2 (l) where mvc,i is the mean of the messages from variable nodes of degree i,givenby

(l) (l−1) mvc,i = mch +(i − 1)mcv , (3.14) and N (m, σ2) is a Gaussian density with mean m and variance σ2. The density of the messages from check nodes to variable nodes is determined from (3.10), which may be rewritten as (l) (l) Lcv (j, i) Lvc (i ,j) tanh = tanh , (3.15) 2 2 i= i where the product is over the dc(j)−1 variable node neighbors of check node j, excluding variable node i. For irregular LDPC codes, the LLR messages from variable/check nodes of the same degree are assumed to be independent and identically distributed (i.i.d.). Therefore, by taking the expected value of both sides, (3.15) simpliﬁes to

j− (l) (l) 1 Lcv,j Lvc E tanh = E tanh . (3.16) 2 2 where j now denotes the degree of the check node instead of the check node number itself.

Now, assume that the random variable Z ∈N(mz, 2mz). Then the expectation

E[tanh z/2] depends only on the mean mz,thatis, ! " 2 z 1 z − (z−mz) E tanh = √ tanh e 4mz dz . (3.17) 2 4πmz R 2 For convenience, deﬁne the function # (z−x)2 1 z − 1 − √ tanh e 4x dz, if x>0 φ(x)= 4πx R 2 (3.18) 1, if x =0.

Note that φ(x) is continuous and monotonically decreasing on [0, ∞) and has an inverse φ−1(x). 3.3. Density Evolution 43

Using (3.13) together with (3.17) and (3.18), we get (l) dvmax $ % Lvc (l) E tanh =1− λiφ m . (3.19) 2 vc,i i=2

(l) By inserting (3.19) into (3.16), the mean mcv,j of the Gaussian message from a check node of degree j toavariablenodeinthelth iteration is found to be ⎛ ⎞ j−1 dvmax $ % (l) −1 ⎝ (l) ⎠ mcv,j = φ 1 − 1 − λiφ mvc,i . (3.20) i=2

The average mean of all messages from check nodes to variable nodes is a weighted (l) average of the mean values mcv,j with ρj as weights,

dcmax (l) (l) mcv = ρjmcv,j (3.21) j=2 ⎛ ⎞ j−1 dcmax dvmax −1 ⎝ (l) ⎠ = ρjφ 1 − 1 − λiφ(mvc,i) . (3.22) j=2 i=2

Using (3.14), (3.22) can be written as a recursive equation ⎛ ⎞ j−1 dcmax dvmax & ' (l) −1 ⎝ (l−1) ⎠ mcv = ρjφ 1 − 1 − λiφ mch +(i − 1)mcv , (3.23) j=2 i=2 which is the fundamental equation of many designs of LDPC codes. For error-free decoding, when the noise variance is below the threshold, the recursive equation converges to ∞ as l →∞.

3.3.2 Mutual Information and EXIT Charts

Instead of tracking the mean of the Gaussian messages, one may track the mutual information (MI) that corresponds to the Gaussian approximation of the message density. Tracking MI generally gives the most accurate prediction of the decoding threshold and MI is also a good metric for many diﬀerent channels, [57] [58, Ch. 4.10.1]. The MI transfer characteristics have been used in [17, 59] to analyze the convergence of iterative decoding schemes. We derive a recursive equation, similar to (3.23), for tracking the MI evolution. The reader is referred to the tutorial introduction in [57] for a detailed derivation. 44 Low-Density Parity-Check Codes

Let IA,V be the average aprioriMI between the input messages and the codeword bits associated to the corresponding variable nodes. IE,V is the average extrinsic MI between the output messages and the corresponding codeword bits. Similarly on the check node side, deﬁne IA,C (IE,C) to be the average apriori(extrinsic) MI between the check node input (output) messages and the codeword bits corresponding to the variable nodes providing (receiving) the message. The evolution is initialized by the average MI between the received messages and the corresponding codeword bits, denoted by Ich.For the AWGN channel, it is given by Ich = J(σch), where

2 Eb σch =8R (3.24) N0 and Eb/N0 is the signal-to-noise ratio at which the analysis is performed. The function J(·) is deﬁned by ∞ 2 2 1 − (y−σ /2) −y − √ 2σ2 J(σ)=1 e log2(1 + e )dy (3.25) −∞ 2πσ2 $ % ∈N σ2 2 and computes the MI of a Gaussian random variable y 2 ,σ . An approximation of the J-function and its inverse, which simpliﬁes the implementation of the MI evolution, is found in [59]. For irregular LDPC codes, the MI evolution functions are given by [57] dvmax ! $ %" l l 2 ( ) −1 ( ) 2 IE,V = λiJ (i − 1) J IA,V + σch (3.26) i=2 and dcmax ! $ %" l l 2 ( ) −1 ( ) IE,C =1− ρjJ (j − 1) J 1 − IA,C . (3.27) j=2 Note that the extrinsic MI from the variable nodes is the aprioriMI for the check nodes, (l) (l) (l+1) (l) that is, IA,C = IE,V . Similarly, we have that IA,V = IE,C. The recursion is initialized by (1) IA,V =0. The exchange of extrinsic information may also be visualized in an extrinsic information transfer chart (EXIT chart), [17, 59]. An EXIT chart shows the characteristics of both the variable node and check node update and the functions (3.26) and (3.27) are plotted in the same diagram. However, when plotting the characteristics of the check node update, the axes are swapped. The MI evolution can now be visualized as a decoding trajectory. Figure 3.3 shows the EXIT chart foranirregularLDPCcodewithrateR =1/2, maximum variable node degree dvmax = 20, and a constant check node degree dc =8, 3.3. Density Evolution 45

IE,V (IA,V ,Eb/N0) 0.9

0.8

0.7

0.6

A,C IA,C (IE,C) 0.5 or I E,V I 0.4

0.3

0.2

0.1

0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 I or I A,V E,C

Figure 3.3: EXIT chart for a rate R =1/2 irregular LDPC code at Eb/N0 =2dB.

at Eb/N0 = 2 dB. The arrows show the decoding trajectory, starting with zero apriori information in the point (0, 0). A vertical and a horizontal arrow correspond to one decoder iteration. If the decoding process converges successfully, the trajectories reach the point (1, 1). To achieve convergence, there must be a tunnel between the two curves, that is, the characteristics of the variable node update IE,V (IA,V ,Eb/N0) must lie above the characteristics of the check node update IA,C (IE,C). The code threshold corresponds to the Eb/N0 at which the tunnel opens and the decoding converges faster the wider the tunnel is. 46 Low-Density Parity-Check Codes Part II

Introduction to the Speciﬁc Research Topics 48 Chapter 4

Overview of Clipping Mitigation Strategies in OFDM

Multi-carrier transmission is a method of transmitting data by splitting it into several components and sending each of these components over separate carrier signals. The available channel bandwidth is divided into a number of equal-bandwidth subchannels, where the bandwidth of each subchannel is suﬃciently narrow so that the frequency response characteristics of the subchannels are nearly ﬂat. Multi-carrier transmission techniques started evolving already in the 1960’s and have received much interest over the years. Multi-carrier transmission is also known as orthogonal frequency-division multiplexing (OFDM) or discrete multitone (DMT). For a detailed introduction to multicarrier transmission systems, the reader is referred to any of the textbooks on the subject, for example [60]. In this chapter, the basics of the OFDM system are presented. Then the clipping problem is discussed and an overview of current clipping mitigation methods is provided.

4.1 OFDM Basics

The main idea of OFDM is to convert an intersymbol interference (ISI) channel into parallel ISI-free subchannels, [61,62]. Each subchannel will have gain equal to the channel’s frequency response at the corresponding subchannel frequency. OFDM is implemented by an inverse fast Fourier transform (IFFT) at the transmitter side and an FFT at the { }N−1 receiver side. Let Xi i=0 denote the N message subsymbols of one data block, typically called the frequency-domain signal, where N is the number of subchannels in the OFDM

49 50 Overview of Clipping Mitigation Strategies in OFDM

AWGN ˆ X x y Y X IFFT P/S Linear S/P FFT equalizer N N filter N N N

Channel

Figure 4.1: The conventional OFDM system. system. The baseband signal, called the time-domain signal, can be written

N−1 1 j2πni/N xn = √ Xi · e n =0,...,N − 1 . (4.1) N i=0 Thus, the modulated OFDM vector can be expressed as

x = IFFT{X}. (4.2)

A cyclic prefix of length no less than the order of the channel is generally inserted between successive blocks. The cyclic prefix makes the linear convolution with the channel appear as circular convolution, which corresponds to multiplication in the FFT domain. The cyclic prefix also prevents interblock interference. For simplicity, the cyclic prefix is omitted in this presentation and it is assumed that there is no interblock interference. A block diagram describing the conventional OFDM system is shown in Figure 4.1. Figure 4.2showsasampleofanOFDMblockwithN = 64 subcarriers and BPSK modulation.

The frequency-domain signal Xn is shown in Figure 4.2a) and the absolute value of the complex time-domain signal xn is shown in Figure 4.2b). OFDM has already been included in several standards, e.g. digital audio/video broadcasting (DAB/DVB) and the local area mobile wireless networks (802.11a). The techniques have also been adopted by the ETSI broadcast radio access network commit- tees, [63]. OFDM enables simple equalization since each subchannel can be assumed to have a frequency-flat impulse response. However, there are several drawbacks with the OFDM system, such as large peak-to-average power ratio (PAPR) of the transmitted signal and sensitivity to channel fades due to reduced diversity, [60]. The PAPR is often very high for OFDM signals,√ since the peak of the complex envelope of the time-domain signal may be up to N times higher than the maximum absolute value of a message subsymbol Xi. A large PAPR makes power backoff necessary, otherwise the transmitter high-power amplifier (HPA) will be working in its non-linear region, resulting in non-linear distortions of the transmitted signal. It is important to use the linear region of the HPA to prevent 4.1. OFDM Basics 51

a) 1.5 1 0.5

n 0 X −0.5 −1 −1.5 0 10 20 30 40 50 60 70 n b)

| 2 n x | 1

0 0 10 20 30 40 50 60 70 n

Figure 4.2: A sample OFDM block with N =64subcarriers. spectral growth that may result in interference between the subchannels and out-of- band radiation. However, with a large power backoff, the range of the transmission is reduced. Also, the power consumption of a HPA depends largely on its peak power output rather than the average output power [64]. Occasional large peaks imply low power efficiency, which is a drawback especially in mobile applications where the battery lifetime is important. To avoid these problems, PAPR-reduction algorithms or deliberate time-domain amplitude clipping can be employed. The instantaneous PAPR of the kth OFDM block and the overall PAPR are defined by 2 xk ∞ PAPRk = (4.3) { 2} E xk 2 /N and 2 maxk xk PAPR = ∞ , (4.4) { 2} E xk 2 /N respectively, where xk is the kth block of the time-domain signal vector. However, in 52 Overview of Clipping Mitigation Strategies in OFDM practice it is more useful to know the distribution of the instantaneous PAPR. The complementary cumulative distribution function (CCDF) of the PAPR is commonly used as a performance measure, [63]. The instantaneous PAPR of the sample OFDM block shown in Figure 4.2b) is 7.7. Note also that the PAPR of the frequency-domain signal, shown in Figure 4.2a), is only 1.

4.2 Reduction of the Peak-To-Average Power Ratio

A number of methods for dealing with the high PAPR of the OFDM signal have been proposed. These techniques include clipping and ﬁltering [65–69], tone reservation [70,71], active constellation extension [64], multiple signal representation [72–75], etc. A tutorial overview of PAPR reduction techniques for OFDM can be found in [63]. A few of the available techniques are summarized here. The simplest method for PAPR reduction is probably digital clipping and ﬁltering of the time-domain signal, [65–69]. The amplitude of the time-domain signal is clipped and the signal after clipping is xn, |xn|≤A [xn] = (4.5) clipped jφ(x) Ae , |xn|

4.3 Reduction of Clipping Eﬀects in the Receiver

When the OFDM signal has been intentionally clipped to reduce the PAPR, receiver-side signal processing may be necessary to reduce the loss due to clipping. This is possible since the clipping noise is usually generated by a known process that can be recreated in the receiver, since the nonlinear clipping distortion is a deterministic function of the data. Therefore, by estimating the clipping noise, the receiver can partly remove the clipping distortion. Reduction of clipping eﬀects in the receiver has been considered in a number of papers, for example [69,77–86]. The idea of decision-aided reconstruction (DAR) of the clipped signal is introduced in [77] and a low complexity algorithm is derived. The DAR method is further developed in [78], where a maximum-likelihood detector for an uncoded OFDM system is proposed. Furthermore, decision-aided reconstruction methods for OFDM systems with convolutional coding were developed in [79, 81, 84]. In [79], the classical DAR procedure is combined with an iterative decoding of the channel code in a turbo-like fashion. Likelihoods of the noisy codeword bits are calculated and used in the Viterbi decoder to compute an estimate of the information bits. The clipped samples are then estimated by comparing the time-domain OFDM signal calculated from the estimation of the information bits with the maximum allowed time-domain signal amplitude. New likelihoods of the codeword bits are calculated from the estimate of the clipped samples. In [81], the classical DAR proposed in [77] is combined with the receiver methods of [78] and [84]. The proposed algorithm is applied to scalable video transmission. The work in [80] also builds on [77] and [78] and considers decision-aided reconstruction of clipped turbo-coded OFDM signals. A method based on iterative detection and soft decoding in the receiver, similar to [79], is developed. Bayesian theory is also employed for reconstruction of clipped OFDM signals [82, 83]. A Bayesian inference method is developed in [82], based on prior densities on the information symbols in the frequency domain, the noise and a binary variable indicating whether a sample is clipped or not. Using the Bayesian model, a maximum a posteriori estimator for the information symbols is derived. A minimum mean-square error (MMSE) estimator of a clipped Gaussian process is derived in closed form in [83]. This estimator can for example be applied to clipped OFDM signals in an AWGN channel. 4.4. Spread OFDM 55

AWGN

SpreadingIFFT Linear FFT Wiener N N filter N filter N

Channel

Figure 4.3: The spread OFDM system.

Another iterative method for reconstruction of clipped OFDM signals is proposed in [85]. An approximation of the distortion signal is first calculated using an assumed statistical model of the clipping distortion. The approximated distortion is subtracted from the received signal and the resulting signal is clipped and filtered to provide an approximation of the remaining distortion. The last step is repeated to give a better approximation. In [86], clipped OFDM signals with insufficient cyclic prefix are recovered iteratively. ISI and intercarrier interference caused by the insufficient cyclic prefix is removed in the same iteration as the clipped signal is reconstructed.

4.4 Spread OFDM

In a multipath environment, some OFDM subcarriers may be completely lost due to deep fades. Therefore, even though most subcarriers may be detected without errors, the overall BER is usually dominated by a few subcarriers with bad SNR. To avoid this domination by the weakest subcarriers, it is of interest to increase the diversity. Extra diversity can be added by introducing dependence among symbols on different subcarriers. This is performed by spreading, also called linear precoding, which was initially introduced by Wornell, [87]. In spread OFDM (SOFDM), the vector to be modulated by the IFFT is first multiplied by a spreading matrix, see Figure 4.3. Wiener filtering may be implemented at the receiver side to reduce the intercarrier interference introduced by the precoder, resulting in a minimum mean square error (MMSE) receiver. For a unitary spreading matrix, the Wiener filter is shown to be simply scalar channel equalization followed by the inverse of the spreading matrix, [88]. Spreading may also reduce the PAPR of the time-domain OFDM signal [89–91]. Since spreading is data-independent, no block-based optimization is performed, which yields low-complexity implementation, and no side information is needed. In [92], channel independent precoders that minimize the uncoded bit error rate (BER) in an OFDM system are derived. It is shown, for QPSK signaling and an MMSE 56 Overview of Clipping Mitigation Strategies in OFDM receiver, that the class of optimal precoders is the unitary matrices with all elements having the same magnitude. Examples of this class are the discrete Fourier transform (DFT) matrix and the Walsh-Hadamard (WH) matrix. Using the DFT matrix as a precoder gives back the single-carrier system, but with a cyclic prefix. This system still enables simple equalization, but to the cost of implementing both the IFFT and FFT in the receiver. The WH matrix has been shown to reduce the PAPR of the signal, [89], as well as increasing the frequency diversity. Debbah et al. proposed a new method to reduce the intercarrier interference in the receiver for WH spreading, [93]. They achieve near maximum-likelihood performance, still with a reasonably low complexity. However, three different spreading matrices are compared in [90] and it is shown that WH spreading gives the smallest PAPR reduction compared to conventional OFDM. Joint coding-precoding is suggested in [94]. The authors show that high orders of diversity can be achieved at a reasonably low complexity by combining coding and spreading. They also state that maximum likelihood decoding is necessary to achieve maximum diversity gain from spreading. Small sized precoders may however be used, without a significant performance loss, to reduce the complexity. In [95], the subcarriers are grouped into subsets of less correlated subchannels and each group has its own small spreading matrix. However, optimal subcarrier grouping and precoder design as suggested in [95] requires channel state information, which may not be available.

4.5 Coded OFDM

The application of error-correction coding to the OFDM system is essential to reduce the BER. Several codes, such as trellis coded modulation, Reed-Solomon codes and BCH codes, have been suggested for use with OFDM, see for example the overview of coded OFDM (COFDM) in [96]. Figure 4.4 shows the block diagram of a coded OFDM system. Typically, the channel encoder produces a codeword with n bits, where n is a multiple of N, the number of OFDM subchannels. The codeword is divided into blocks of length N, that are serially fed to the OFDM IFFT modulator. Prior to the channel decoder, several received OFDM blocks are buﬀered to produce a received word. Several coding techniques to reduce the PAPR have been presented, [97–102]. The basic idea of these techniques is to avoid the use of codewords with high PAPR. The coding techniques that achieve both PAPR reduction and error correction are attractive. However, there are still no good solutions available on how to obtain low PAPR and error correction with a reasonable coding rate for a large number of subcarriers, [102]. 4.5. Coded OFDM 57

AWGN

Channel IFFT Linear FFT Channel encoder filter decoder

Channel

Figure 4.4: Block diagram of a coded OFDM system.

After the invention of turbo codes, these codes were suggested also for COFDM systems, [80, 103–106]. An overview of COFDM is given in [103] together with a comparison of turbo coded OFDM to already existing COFDM schemes. A turbo coded OFDM system with PAPR reduction through SLM without side information is proposed in [106]. The turbo encoder uses different interleavers to generate different sequences and the sequence with the lowest PAPR is transmitted. In the receiver, no side information about the interleaver is available, but the decoder calculates the reliability of each possible sequence (corresponding to the different interleavers) and chooses the most reliable candidate. Another scheme for PAPR reduction using random-like codes is suggested in [105]. In [80], deliberate amplitude clipping of the OFDM signal is combined with turbo coding to achieve an OFDM system with low PAPR and good performance in terms of BER. The clipping distortion is modeled as an attenuation and additive noise and this model is incorporated into the initial calculation of channel LLRs, as described in the earlier paper [107] by the same authors. However, although this modification significantly improves the performance, there is still a substantial performance degradation compared to the performance of the system in an ideal AWGN channel without clipping. The performance of the turbo coded OFDM system with clipping is further improved by a recursive reconstruction of the clipped signal [80]. LDPC coded OFDM was first considered in [108,109]. It was shown [108] that LDPC COFDM outperforms turbo COFDM on a frequency-selective fading channel, while the turbo COFDM performs better on an AWGN channel. The LDPC COFDM scheme was further developed in [109]. LDPC COFDM has been considered for optical communication systems, [110]. It is suggested as a universal coded modulation technique that is suitable for use in a variety of optical channels. Optimization of LDPC codes for OFDM is discussed in [111–113]. In order to optimize the LDPC code according to the quality of the different subchannels, channel state information is needed at the transmitter side. It is also important that the channel does not change or changes very slowly, since a redesigned code must also be transmitted to 58

AWGN

Channel SpreadingIFFT Linear FFT Wiener Channel encoder N filter N filter decoder

Channel

Figure 4.5: Block diagram of a coded spread OFDM system. the receiver. However, for some wireline applications, like xDSL transmissions [111] or power-line DMT transmissions [113], the channel variations may be small enough. The main idea is to specify which bits of the codeword that should be sent through each subchannel. In [112], the degree distributions are optimized as in [19] (see Section 3.3.1), and good positions for the information bits are then found according to the quality of the subchannels. The variable node degree distribution may also be optimized in con- junction with the bit positions, by dividing the variable node degree distribution into subdegree distributions [111,113]. By building a rectangular approximation of the channel spectrum, the frequency tones can be grouped into subbands according to their SNR. The subdegree distributions can be optimized by linear programming. The main differences between the approaches in [111] and [113] are in the optimization of the subdegree distributions. In [114], linear precoding and error-correction are compared in terms of their ability to deal with channel fades. The increase in diversity by linear precoding and coded OFDM is discussed. The authors conclude that coded OFDM often incurs high complexity and/or large decoding delay, while the diversity gain is rather small. On the other hand, the complexity added by linear precoding is small compared to the gains in diversity that may be achieved. However, today the complexity added by error-correction is usually not a problem and the best solution may be a combination of linear precoding and error correction, as shown in Figure 4.5. Such a combination was first considered in [104], where it was shown that the diversity of the combined system is the product of the individual diversities of the component coder and precoder. The decoding complexity of the system is only a linear multiple of the sum of their individual complexities. Serener et al. have investigated LDPC coded spread OFDM in [115–118]. The performance of spread OFDM with LDPC coding in outdoor and indoor environments was analyzed in [115] and [116], respectively. In [117,118], the same authors extend their work further and optimize LDPC codes for OFDM and SOFDM in correlated channels. They show that the error floors of LDPC COFDM may be lowered by introducing spreading to the system. Chapter 5 Overview of Unequal Error Protection Codes

In many communication scenarios, such as wireless networks and transport of multimedia data, sufficient error protection is often a luxury. In these systems it may be wasteful or even infeasible to provide uniform protection for all information bits. Instead, it is more efficient to protect the most important information more than the rest. This implies improving the performance of the more important bits by sacrificing some performance of the less important bits. There are several examples where unequal error protection (UEP) is desired. • In wireless networks, protocol information is often more important than the payload data, even though the final objective is to deliver payload data. The protocol information should therefore be more protected than the payload data.

• Source coded multimedia data, such as video or speech, often has different importance levels. It is of interest to have better protection of the coarse resolution than the fine resolution. If the channel conditions are good, it is desired that all data is recovered. However, under bad channel conditions, it may still be possible to have a coarse reconstruction of the data. This is typical for one-way broadcast media, such as terrestrial broadcasting of digital high-definition television (HDTV), where a reverse link used to adjust the protection according to the channel conditions is not possible. To achieve UEP, the most obvious method is to use separate codes with different protection for the different classes of bits, depending on their importance. This approach is called the time-sharing method. However, in most applications, the number of important bits is low compared to the total number of bits. If separate codes are used,

59 60 Overview of Unequal Error Protection Codes the most important bits are protected by a short code. This may not be efficient, since longer codes generally have better performance and a code with very low rate might be needed to get low enough BER for the most important bits. Another drawback of the time-sharing method is the increased complexity introduced by separate encoders and decoders for the different classes of bits. Some fundamental limits of unequal error protection are presented in [119,120]. For example, it is shown in [119] that for data-rates approaching capacity, even a single bit cannot achieve a positive error exponent, that is, it is not possible to make the error probability of a particular bit decay exponentially fast with block length. However, [120] shows that good UEP codes exist when only a small fraction (approaching zero in the limit) of bits are highly protected. These codes have perfect protection in the sense that even when the channel capacity is arbitrary close to zero, it is possible to get arbitrarily small error probability of the important bits without increased error probability of the other bits. With today’s capacity achieving codes one might argue that UEP codes are not of interest when the SNR is above the threshold, because the average BER (the probability that a bit is not recovered when the decoding stops) goes to zero if the code is long enough and the number of decoder iterations is large enough. Thus, capacity achieving codes are asymptotically better than UEP codes, since all bits have an asymptotically vanishing BER. However, UEP is desired for codes with short or moderate length that are usable in practice, since error-free communication of all bits requires a relatively high SNR when using such codes. This chapter gives an overview of UEP codes from the first developments in the 1960’s until today’s achievements. Error control codes achieving UEP may be constructed in many different ways, such as coded modulation with UEP, UEP turbo codes, etc. The last section focuses particularly on irregular UEP-LDPC codes, which are the UEP codes considered in the appended papers. In the following sections it is assumed that the codeword bits are divided into groups with different protection, called protection classes.

5.1 UEP Codes

UEP codes were ﬁrst studied by Masnick and Wolf in 1967, [121]. They described the properties of linear UEP codes, derived bounds on the number of parity bits needed and proved that systematic cyclic codes could not have UEP. Cyclic UEP codes were desired due to their low-complexity encoding and decoding. However, shortly thereafter it was shown through an example that non-systematic cyclic codes with UEP exist, [122]. 5.1. UEP Codes 61

This paper was followed up by a more thorough description of a class of non-systematic UEP codes that may be easily decoded by majority-logic, [123]. Another early UEP scheme was developed in [124]. A typical application of UEP codes at this time, which is mentioned in several papers, is the unequal importance of the bits representing decimal numbers. When transmitting digital data, errors in the higher order digits of numbers are usually more costly than errors in the lower order digits. Therefore, unequal error protection of the bits would better correspond to their importance and thereby improve the perceived quality of the data. Linear UEP codes were further studied by Boyarinov and Katsman, [125]. They suggested iterative and concatenated designs of UEP codes and provided fast decoding algorithms for these codes. Another class of UEP codes based on cyclic codes was developed in [126] and further investigated in [127]. Hagenauer proposed a family of rate-compatible punctured convolutional codes, [128]. Starting from low-rate convolutional codes, periodic puncturing produces codes of different rates that perform well relative to the best known codes of comparable complexity. The code rate can also be changed within a data frame, which makes these codes suitable for applications requiring UEP. Another popular way of achieving UEP is coded modulation. In a band-limited environment, coding and modulation should be integrated to use the available bandwidth more efficiently. UEP codes may be integrated with modulation solely for this purpose, but the modulation may also be employed to enhance the UEP capability of the system. One common and simple way to achieve UEP by modulation is to use constellations with non-uniformly spaced signal points, [129–131]. Typically, neighboring signal points are assigned the same bit pattern for the important bits and different patterns for the less important bits. After the invention of capacity-achieving codes, turbo codes have also been used in coded modulation to obtain powerful UEP codes without sacrificing bandwidth [132, 133]. The appended papers C and D are the first papers that consider design of bandwidth-efficient UEP-LDPC codes. Turbo UEP codes for binary modulation were invented and further investigated by [134–136]. The main idea is to use a standard turbo code as a basis and perform non-uniform puncturing of the parity bits from the parallel encoders. Typically, turbo codes use uniform puncturing of the parity bits, which leads to almost equal error protection of all information bits. UEP capability may be achieved by increasing the number of redundancy bits correlated with the most important bits and reducing the number of redundancy bits correlated with the less important bits, using non-uniform puncturing. By UEP capability we mean reduced BER of the most important bits, while the BER 62 Overview of Unequal Error Protection Codes of other bits is only slightly increased. It is shown in [136] that to increase the UEP capability, the encoder interleavers should be chosen to guarantee isolation of the different protection classes. Iterative decoding was also proposed in [137] to increase the performance of a UEP coding scheme based on a product-like combination of Reed-Muller codes.

5.2 UEP-LDPC Codes

The outstanding performance of LDPC codes makes them an interesting choice for UEP schemes and LDPC codes with UEP started to evolve in parallel with UEP turbo codes. Several methods for constructing UEP-LDPC codes have been presented, [120,138–147]. Many of these methods are based on the irregularity of the variable and/or check node degree distributions and we treat them in a separate section. The other methods are all different. Vasic et al. proposed a class of UEP-LDPC codes based on cyclic difference families, [138]. The resulting LDPC codes are irregular, but we consider them here since the codes have a special structure that is different from the other irregular UEP-LDPC codes. However, the performance of this class of codes was shown to be outperformed by the codes suggested in [140]. In [141], UEP capability is achieved by a combination of two Tanner graphs of different rates. A high-rate LDPC code gives extra protection to the more important bits, while a lower-rate LDPC code protects all bits. Density evolution formulas for the binary erasure channel are derived for the combination of codes, which allows the design of high performance UEP-LDPC codes. Combinations of an LDPC code with Reed-Solomon codes have been considered in [148] and [149], which both apply the proposed codes to image transmission applications. UEP may also be provided by non-binary LDPC codes, [145]. Another completely different UEP scheme was proposed in [146]. The UEP-LDPC codes are based on the algebraic Plotkin construction and are decoded in multiple stages. Both the irregularity of the resulting parity-check matrix as well as the multistage decoding algorithm provide UEP capability. This UEP scheme was further developed in [150] and applied to JPEG2000 image transmission. It is also argued in [146] that irregular UEP-LDPC codes will not provide UEP unless a very small number of decoder iterations is performed. 5.3. Irregular UEP-LDPC Codes 63

5.3 Irregular UEP-LDPC Codes

It is explained in [39] that from the point of view of a variable node, it is best to have high degree, because the more information it gets from its check nodes the more accurately it can judge what its correct value should be. On the contrary, from the point of view of a check node, it is best to have low degree, since the lower the degree of a check node is, the more valuable is the information it can transmit back to its neighboring variable nodes. It has also been widely observed that the connection degrees of the variable nodes aﬀect the bit-error rate. These observations are the basis for the design of irregular UEP-LDPC codes. All irregular LDPC codes can in principle be UEP-LDPC codes. By mapping the more important bits to the higher degree variable nodes of a standard LDPC code, some UEP behavior is achieved. However, the UEP capability may be enhanced by modiﬁcation of the code ensemble degree distributions. The following irregular UEP- LDPC design schemes are all variants of the standard code design, which enhance the UEP properties of the code. Note that only the design of the degree distributions together with mapping of information bits to certain variable nodes are considered to enhance the UEP capability of these codes. The encoding and decoding operations are performed exactly as for non-UEP LDPC codes. Irregular UEP-LDPC codes have been studied in [120,139,140,142–144,147]. The designs considered in [139,140,142,143,147] rely solely on irregular degrees of the variable nodes, while the check nodes have a concentrated degree distribution.

Partially regular ensembles with a constant check node degree (dc) are considered in [139, 140]. The variable nodes are divided into several protection classes according to their degrees, such that the variable nodes in one class all have the same degree. A schematic description of a partially regular ensemble with 4 protection classes is shown in Figure 5.1. A more protected class has a higher variable node degree than a less C1 C2 C3 C4 protected class, that is, dv >dv >dv >dv . Only two classes of information bits and one class of parity bits are considered in [139,140]. The authors derive density evolution formulas for the binary erasure channel and use them to optimize the degrees, so that the important classes have low error rates, while the overall performance is comparable to other codes. Another UEP-LDPC code design based on irregular variable node degrees is proposed in [142,143]. An extended parameterization of the LDPC code irregularity is considered and a distinct variable node degree distribution for each protection class is allowed. A hierarchical optimization procedure is suggested to optimize the degree distributions. 64 Overview of Unequal Error Protection Codes

n − k

4 2 C3 C C1 C d dv dv dv v

nC1 nC2 nC3 nC4

Figure 5.1: Tanner graph of partially regular ensembles with 4 protection classes.

• The design algorithm is very general and allows an arbitrary number as well as arbitrary sizes of the protection classes.

• Optimal degree distributions are easily calculated using linear programming.

• The degree distributions are optimized for the AWGN channel.

• There is little loss in overall performance even though the most protected classes have improved performance. For some parameter settings, improvements of all protection classes have been reported. 66

• Only the design of the degree distributions and the construction of the code according to these distributions diﬀer from standard LDPC codes. This means that the same encoder and decoder may be used even though a UEP-LDPC code is employed, which keeps the system complexity low.

When designing irregular UEP-LDPC codes, the UEP capability increases with the maximum variable node degree. However, [29] states that with larger degrees, the codeword needs to be longer in order to approach the asymptotic performance predicted by density evolution. This means that the maximum degree should be limited when optimizing the degree distributions. There is a trade oﬀ between the UEP capability that can be achieved by high variable node degrees and the performance loss when the codeword is not long enough for the high degrees. Chapter 6 Research Contributions

This thesis contributes to two diﬀerent research areas within coding theory through the research presented in Part III. This chapter summarizes the appended research papers and discusses the contributions made. The work is related to contributions by other authors, presented in Chapters 4 and 5.

6.1 Clipping Mitigation in LDPC Coded OFDM Sys- tems

As discussed in Chapter 4, the main drawbacks of the OFDM system are the high peak- to-average power ratio (PAPR) and the sensitivity to channel fades. Channel coding is critical to reduce the error probabilities and a coded OFDM system is generally employed. An LDPC coded OFDM system is considered in Papers A and B.

Paper A - Performance of LDPC Coded Spread OFDM with Clipping

Authors: S. Sandberg, C. de Fr´ein, J. P. LeBlanc, B. Vasic and A. D. Fagan Reproduced from: Proceedings of the 8th International Symposium on Communication Theory and Applications (ISCTA) 2005, UK, pp. 156-161.

Paper A presents the performance of an LDPC coded spread OFDM system. The aim of spreading is to reduce the eﬀects of frequency-selective fading by introducing dependency among the bits transmitted over diﬀerent subchannels. The presented system employs Walsh-Hadamard spreading, that also reduces the PAPR [89]. Joint implementation of

67 68 Research Contributions the IFFT and the Walsh-Hadamard spreading matrix in the transmitter can also be performed with a smaller number of operations than if the transformations were computed separately [151]. Simulations for an ETSI indoor wireless channel model show that the gain by spreading is substantial in an LDPC coded OFDM system, thereby conﬁrming the results by Serener et al. [115, 116]. The contribution of Paper A is the investigation of an LDPC coded spread OFDM system with clipping. If clipping is considered, the reduced PAPR leads to lower BERs, since the probability of clipping is reduced and there is less clipping noise. Simulation results show that the coding gain by joint LDPC coding and Walsh-Hadamard spreading is larger in a system with clipping than without. A later paper, [90] (2007), compared the PAPR reduction achieved by three diﬀerent spreading matrices and among these the Walsh-Hadamard spreading matrix achieved the least PAPR reduction. Therefore, if a slightly higher complexity can be tolerated, the performance may be further increased by choosing another spreading matrix.

Paper B - Receiver-oriented Clipping-eﬀect Mitigation in OFDM - A Worthy Approach?

Authors: S. Sandberg, J. P. LeBlanc and B. Vasic Reproduced from: Proceedings of the 10th International OFDM-Workshop 2005, Hamburg.

Paper B analyzes the performance of two different methods to reduce the clipping distortion effects in the receiver. In the first method, the clipping noise is characterized as in [107] and the channel bit LLRs fed to the LDPC decoder are modified accordingly. The second method is Bayesian estimation of the signal before clipping [83]. Simulations on an LDPC coded OFDM system show unexpectedly that the methods, which significantly improve the BER performance in uncoded OFDM systems at high SNRs [83,107], achieve almost no performance improvement when LDPC codes are applied. The redundancy added by the LDPC code seem to be effective also in mitigating the clipping distortion. These results suggest that clipping mitigation in LDPC coded OFDM should be concentrated to the transmitter. A similar paper [80] appeared only a few months after the submission of Paper B. The authors proposed an iterative method, combining a modification of channel bit LLRs fed to a turbo decoder with recursive reconstruction of the clipped signal. This iterative scheme improves the performance significantly, but there is still a substantial performance degradation compared to an ideal AWGN channel without clipping. 6.2. Design of Unequal Error Protection LDPC Codes 69

6.2 Design of Unequal Error Protection LDPC Codes

The second research area of this work is design of UEP-LDPC codes. Papers C and D consider design of UEP-LDPC codes for higher order constellations (HOCs). Paper E analyzes the UEP behavior of LDPC codes constructed by diﬀerent construction algorithms.

Paper C - Design of Unequal Error Protection LDPC Codes for Higher Order Constellations

Authors: N. von Deetzen and S. Sandberg Reproduced from: Proceedings of the IEEE International Conference on Communications, 2007, Scotland, pp. 926-931.

Paper C presents a UEP-LDPC code design for HOCs based on optimization of the variable node degree distribution. This is the first work on design of UEP-LDPC codes for HOCs and it can be seen as a combination of the UEP-LDPC code design proposed in [143] and the design of LDPC codes for HOCs presented in [113,152]. In higher order modulation, different bits in a symbol may have different error probabilities. The variable node degree distribution to be optimized is divided into sub-degree distributions both for the protection classes and for the classes of bits with different inherent protection resulting from the modulation. This allows for a flexible code design allowing any modulation scheme and arbitrary number and sizes of the protection classes. Typically, HOCs provide inherent UEP. Design of LDPC codes for HOCs (e.g. [113, 152]) aim at reducing the overall BER by providing overall equal error protection through channel coding. The design strategy presented in Paper C and extended in Paper D provides a method that uses the UEP from the modulation to create a code with other UEP properties, usually specified by the source coding unit. Simulations with 8-PSK modulation show that the proposed design method improves the overall BER performance, while slightly reducing the UEP capability compared to UEP-LDPC code design for BPSK. Furthermore, it should be noted that the UEP-LDPC code design for HOCs may also be directly applied to another problem, even though it was not the aim of the code design and is not mentioned in the paper. Design of LDPC codes for OFDM channels [111,112] and DMT channels [113] has been considered, under the assumption that the channel is constant or varies very slowly (see Section 4.5). Both [111] and [113] divide the variable node degree distribution into sub-degree distributions in the same way as is done in Paper C. The design algorithm proposed in Paper C accounts for differences in error probability 70 Research Contributions between the received bits before decoding. Such differences may arise from different error probabilities of bits in a symbol (as in design for HOCs) or from different subchannels in an OFDM system having different SNRs. The design algorithm proposed in Papers C and D may therefore also be used to design UEP-LDPC codes for OFDM channels.

Paper D - Design of Bandwidth-Eﬃcient Unequal Error Protec- tion LDPC Codes

Authors: S. Sandberg and N. von Deetzen Accepted for publication in IEEE Transactions on Communications, Jan. 29, 2009

Paper D is an extension of Paper C. An important addition is that the effects of one of the design-parameters, the threshold offset , on BER performance and UEP capability have been investigated. Simulation results for 8-PSK show that the proposed algorithm gives the best performance and UEP capability if the threshold offset is chosen to =0.1 dB. This contradicts the assumption in Paper C that a higher gives more UEP capability. Paper D also shows that the UEP capability does not vanish after a large number of decoder iterations, as has been claimed by several authors. Moreover, Paper D compares the BER performance of the proposed design with an existing LDPC code design for HOCs without UEP capability. Simulations with =0.1 dB show that the proposed design slightly reduces the overall BER, while also increasing the UEP capability. Finally, simulations for 64-QAM show that a higher modulation index yields greater performance gain of the HOC-UEP scheme compared to the UEP scheme.

Paper E - On the UEP Capabilities of Several LDPC Construc- tion Algorithms

Authors: N. von Deetzen and S. Sandberg Submitted to: IEEE Transactions on Communications, Nov. 18, 2008

It is commonly known that the connection degree of a variable node affects its error probability and this is the basis for design of irregular UEP-LDPC codes. However, some authors argue that this holds only for a very small number of decoding iterations [146], while others show good UEP capability also after a large number of iterations, as in Pa- per D. Paper E analyzes construction algorithms for the parity-check matrices of LDPC 6.2. Design of Unequal Error Protection LDPC Codes 71 codes, with respect to their capabilities of providing UEP. By construction of several codes with identical degree distributions, it is shown that the choice of code construction algorithm is critical and highly affects the performance and the UEP properties of the code. These results explain disagreements in earlier research. The differences in UEP capability are explained by structural differences in the parity- check matrices. Detailed check node degree distributions for the protection classes are introduced to describe the number of connections between different protection classes. The UEP capability of an LDPC code is enhanced by addition of constraints on the detailed check node degree distributions during the code construction. However, the trade off between UEP capability and overall BER performance is clearly seen in the simulation results. When the number of connections between different protection classes is reduced by addition of constraints on the detailed check node degree distributions, the UEP capability is increased, while the overall BER performance is reduced. 72 Research Contributions References

[1] C. E. Shannon, “A mathematical theory of communication,” Bell System Technical Journal, 1948.

[2] J. G. Proakis, Digital Communications. New York: McGraw-Hill, 4th ed., 2001.

[3] J. B. Anderson, Digital transmission engineering. New York: IEEE Press, Prentice Hall, 1999.

[4]E.A.LeeandD.G.Messerschmitt,Digital Communication. Massachusetts: Kluwer Academic Publishers, 2nd ed., 1996.

[5] D. J. C. MacKay, Information theory, inference and learning algorithms. Cambridge University Press, 2003.

[6] R. W. Hamming, “Error detecting and error correcting codes,” Bell System Tech- nical Journal, vol. 26, pp. 147–160, April 1950.

[7] P. Elias, “Coding for noisy channels,” IRE Convention Record, no. 4, pp. 37–46, 1955.

[8] R. G. Gallager, “Low density parity check codes,” IRE Transactions on Information Theory, pp. 21–28, Jan. 1962.

[9] S. Lin and D. J. Costello, Error control coding, second edition. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 2004.

[10] G. Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Transac- tions on Information Theory, vol. 28, pp. 55–67, Jan. 1982.

[11] G. Ungerboeck, “Trellis-coded modulation with redundant signal sets Part I: In- troduction,” IEEE Communications Magazine, vol. 25, pp. 5–11, Feb. 1987.

73 74 References

[12] G. Ungerboeck, “Trellis-coded modulation with redundant signal sets Part II: State of the art,” IEEE Communications Magazine, vol. 25, pp. 12–21, Feb. 1987.

[13] H. Imai and S. Hirakawa, “A new multilevel coding method using error correcting codes,” IEEE Transactions on Information Theory, vol. 23, pp. 371–377, May 1977.

[14] C. Berrou, A. Glavieux, and T. Thitimajshima, “Near Shannon limit error- correcting coding and decoding: turbo codes,” in Proc. ICC 1993, pp. 1064–1070, May 1993.

[15] D. J. C. MacKay and R. M. Neal, “Good codes based on very sparse matrices,” 5th IMA Conference on Cryptography and Coding. (Lecture notes in computer science), pp. 100–111, 1995.

[16] N. Wiberg, Codes and decoding on general graphs.Link¨oping, Sweden: Dissertation no. 440, Dept. of Elect. Eng. Link¨oping Univ., 1996.

[17] S. ten Brink, “Convergence behavior of iteratively decoded parallel concatenated codes,” IEEE Transactions on Communications, vol. 49, pp. 1727–1737, Oct. 2001.

[18] T. Richardson and R. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,” IEEE Transactions on Information Theory, vol. 47, pp. 599–618, Feb 2001.

[19] S.-Y. Chung, T. Richardson, and R. Urbanke, “Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation,” IEEE Trans- actions on Information Theory, vol. 47, pp. 657–670, Feb. 2001.

[20] A. Calderbank, “The art of signaling: ﬁfty years of coding theory,” IEEE Trans- actions on Information Theory, vol. 44, pp. 2561–2595, Oct 1998.

[21] S. B. Wicker and S. Kim, Fundamentals of codes, graphs, and iterative decoding. Norwell, MA, USA: Kluwer Academic Publishers, 2002.

[22] P. Sweeney, Error control coding, from theory to practice. Wiley, 2002.

[23] R. C. Bose and D. K. Ray-Chaudhuri, “On a class of error-correcting binary group codes,” Information and Control, vol. 3, pp. 68–79, 1960.

[24] R. Fano, “A heuristic discussion of probabilistic decoding,” IEEE Transactions on Information Theory, vol. 9, pp. 64–74, Apr 1963. References 75

[25] F. Jelinek, “A fast sequential decoding algorithm using a stack,” IBM J. Res. Dev., vol. 13, pp. 675–685, Nov. 1969.

[26] A. Viterbi, “Error bounds for convolutional codes and an asymptotically optimum decoding algorithm,” IEEE Transactions on Information Theory, vol. 13, pp. 260– 269, Apr 1967.

[27] L. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of linear codes for minimizing symbol error rate (corresp.),” IEEE Transactions on Information Theory, vol. 20, pp. 284–287, Mar 1974.

[28] W. E. Ryan, “A turbo code tutorial,” in Proc. IEEE Globecom’98, [Online]. Avail- able: http://www.ece.arizona.edu/∼ryan, 1998.

[29] T. Richardson, M. Shokrollahi, and R. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Transactions on Information The- ory, vol. 47, pp. 619–637, Feb. 2001.

[30] T. Richardson, “Error ﬂoors of LDPC codes,” in Proc. 41st Annual Allerton Conf. on Commun., Control, and Computing, vol. 41, pp. 1426–1435, Oct. 2003.

[31] M. Tanner, “A recursive approach to low complexity codes,” IEEE Transactions on Information Theory, pp. 533–547, Sep. 1981.

[32] M. Sipser and D. Spielman, “Expander codes,” IEEE Transactions on Information Theory, vol. 42, pp. 1710–1722, Nov 1996.

[33] D. J. C. MacKay, “Good error-correcting codes based on very sparse matrices,” IEEE Transactions on Information Theory, pp. 399–431, March 1999.

[34] W. E. Ryan, “An introduction to LDPC codes,” 2003. Available online at http: //www.ece.arizona.edu/∼ryan/NewFolder/ryan-crc-ldpc-chap.pdf.

[35] A. Shokrollahi, “LDPC codes: An introduction,” 2003. Available online at http: //ipm.ac.ir/swc/2002/amin/Amin2.pdf.

[36] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, D. A. Spielman, and V. Ste- mann, “Practical loss-resilient codes,” in STOC ’97: Proceedings of the twenty- ninth annual ACM symposium on Theory of computing, (New York, NY, USA), pp. 150–159, ACM, 1997. 76 References

[37] S.-Y. Chung, J. Forney, G.D., T. Richardson, and R. Urbanke, “On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit,” IEEE Communications Letters, vol. 5, pp. 58–60, Feb 2001.

[38] M. Yang, W. Ryan, and Y. Li, “Design of eﬃciently encodable moderate-length high-rate irregular LDPC codes,” IEEE Transactions on Communications, vol. 52, pp. 564–571, April 2004.

[39] M. Luby, M. Mitzenmacher, M. Shokrollahi, and D. Spielman, “Improved low- density parity-check codes using irregular graphs,” IEEE Transactions on Infor- mation Theory, vol. 47, pp. 585–598, Feb 2001.

[40] X.-Y. Hu, E. Eleftheriou, and D. M. Arnold, “Regular and irregular progressive edge-growth Tanner graphs,” IEEE Transactions on Information Theory, vol. 51, pp. 386–398, Jan. 2005.

[41] T. Tian, C. Jones, D. Villasenor, and R. Wesel, “Selective avoidance of cycles in irregular LDPC code construction,” IEEE Transactions on Communications, vol. 52, pp. 1242–1247, Aug. 2004.

[42] D. Vukobratovic and V. Senk, “Generalized ACE constrained progressive edge- growth LDPC code design,” IEEE Communications Letters, vol. 12, pp. 32–34, Jan. 2008.

[43] T. J. Richardson and R. L. Urbanke, “Eﬃcient encoding of low-density parity-check codes,” IEEE Transactions on Information Theory, pp. 638–656, Feb. 2001.

[44] Y. Kou, S. Lin, and M. Fossorier, “Low-density parity-check codes based on ﬁnite geometries: a rediscovery and new results,” IEEE Transactions on Information Theory, vol. 47, pp. 2711–2736, Nov 2001.

[45] M. Fossorier, “Quasicyclic low-density parity-check codes from circulant permutation matrices,” IEEE Transactions on Information Theory, vol. 50, pp. 1788–1793, Aug. 2004.

[46] J. Chen, R. Tanner, J. Zhang, and M. Fossorier, “Construction of irregular LDPC codes by quasi-cyclic extension,” IEEE Transactions on Information The- ory, vol. 53, pp. 1479–1483, April 2007.

[47] S. Kim, J.-S. No, H. Chung, and D.-J. Shin, “Quasi-cyclic low-density parity- check codes with girth larger than 12,” IEEE Transactions on Information Theory, vol. 53, pp. 2885–2891, Aug. 2007. References 77

[48] G. Liva, W. Ryan, and M. Chiani, “Quasi-cyclic generalized LDPC codes with low error ﬂoors,” IEEE Transactions on Communications, vol. 56, pp. 49–57, January 2008.

[49] L. Zeng, L. Lan, Y. Tai, S. Song, S. Lin, and K. Abdel-Ghaffar, “Transactions papers - Constructions of nonbinary quasi-cyclic LDPC codes: A finite field approach,” IEEE Transactions on Communications, vol. 56, pp. 545–554, April 2008.

[50] B. Vasic and O. Milenkovic, “Combinatorial constructions of low-density parity- check codes for iterative decoding,” IEEE Transactions on Information Theory, pp. 1156–1176, June 2004.

[51] B. Ammar, B. Honary, Y. Kou, J. Xu, and S. Lin, “Construction of low-density parity-check codes based on balanced incomplete block designs,” IEEE Transac- tions on Information Theory, vol. 50, pp. 1257–1269, June 2004.

[52] J. Xu, L. Chen, L. Zeng, L. Lan, and S. Lin, “Construction of low-density parity- check codes by superposition,” IEEE Transactions on Communications, vol. 53, pp. 243–251, Feb. 2005.

[53] J. Xu, L. Chen, I. Djurdjevic, S. Lin, and K. Abdel-Ghaﬀar, “Construction of regular and irregular LDPC codes: Geometry decomposition and masking,” IEEE Transactions on Information Theory, vol. 53, pp. 121–134, Jan. 2007.

[54] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum- product algorithm,” IEEE Transactions on Information Theory, pp. 498–519, Feb. 2001.

[55] T. Etzion, A. Trachtenberg, and A. Vardy, “Which codes have cycle-free tanner graphs?,” IEEE Transactions on Information Theory, vol. 45, pp. 2173–2181, Sep 1999.

[56] M. Ivkovic, S. K. Chilappagari, and B. Vasic, “Eliminating trapping sets in low- density parity-check codes by using Tanner graph covers,” IEEE Transactions on Information Theory, vol. 54, pp. 3763–3768, Aug. 2008.

[57] G. Liva, S. Song, L. Lan, Y. Zhang, S. Lin, and W. E. Ryan, “Design of LDPC codes: A survey and new results,” J. Comm. Software and Systems, 2006. Available at http://elib.dlr.de/47266/. 78 References

[58] T. Richardson and R. Urbanke, Modern Coding Theory. Cambridge University Press, March 2008.

[59] S. ten Brink, G. Kramer, and A. Ashikhmin, “Design of low-density parity-check codes for modulation and detection,” IEEE Transactions on Communications, vol. 52, pp. 670–678, April 2004.

[60] A. R. S. Bahai, B. R. Saltzberg, and M. Ergen, Multi-carrier digital communications: theory and applications of OFDM (2nd ed.). Springer, New York, USA., 2004.

[61] J. Cimini, L., “Analysis and simulation of a digital mobile channel using orthogonal frequency division multiplexing,” IEEE Transactions on Communications, vol. 33, pp. 665–675, Jul 1985.

[62] J. Bingham, “Multicarrier modulation for data transmission: an idea whose time has come,” IEEE Communications Magazine, vol. 28, pp. 5–14, May 1990.

[63] S. H. Han and J. H. Lee, “An overview of peak-to-average power ratio reduction techniques for multicarrier transmission,” IEEE Wireless Communications, vol. 12, pp. 56–65, April 2005.

[64] B. S. Krongold and D. L. Jones, “PAR reduction in OFDM via active constellation extension,” IEEE Transactions on Broadcasting, pp. 258–268, Sep. 2003.

[65] X. Li and J. Cimini, L.J., “Eﬀects of clipping and ﬁltering on the performance of OFDM,” IEEE Communications Letters, vol. 2, pp. 131–133, May 1998.

[66] J. Armstrong, “Peak-to-average power reduction for OFDM by repeated clipping and frequency domain ﬁltering,” Electronics Letters, vol. 38, pp. 246–247, Feb 2002.

[67] H. Ochiai and H. Imai, “Performance analysis of deliberately clipped OFDM signals,” IEEE Transactions on Communications, vol. 50, pp. 89–101, Jan 2002.

[68] K. R. Panta and J. Armstrong, “Eﬀects of clipping on the error performance of OFDM in frequency selective fading channels,” IEEE Transactions on Wireless Communications, pp. 668–671, March 2004.

[69] F. Peng and W. Ryan, “MLSD bounds and receiver designs for clipped OFDM channels,” IEEE Transactions on Wireless Communications, vol. 7, pp. 3568–3578, Sep. 2008. References 79

[70] J. Tellado and J. Cioﬃ, “Eﬃcient algorithms for reducing PAR in multicarrier systems,” IEEE International Symposium on Information Theory, 1998. Proceedings. 1998, pp. 191–, Aug 1998.

[71] H. Entekhabi, M. Sharif, and V. Tarokh, “On the peak to average power reduction of OFDM signals using reserved subcarriers,” Personal, Indoor and Mobile Radio Communications, 2008. PIMRC 2008. IEEE 19th International Symposium on, pp. 1–5, Sep. 2008.

[72] S. M¨uller and J. Huber, “OFDM with reduced peak-to-average power ratio by optimum combination of partial transmit sequences,” Electronics Letters, vol. 33, pp. 368–369, Feb 1997.

[73] R. B¨auml, R. Fischer, and J. Huber, “Reducing the peak-to-average power ratio of multicarrier modulation by selected mapping,” Electronics Letters, vol. 32, pp. 2056–2057, Oct 1996.

[74] M. Breiling, S. H. M¨uller-Weinfurtner, and J. B. Huber, “Peak-power reduction in OFDM without explicit side information,” 5th International OFDM-Workshop 2000, Hamburg, Germany, Sep. 2000.

[75] A. Jayalath and C. Tellambura, “SLM and PTS peak-power reduction of OFDM signals without side information,” IEEE Transactions on Wireless Communica- tions, vol. 4, pp. 2006–2013, Sep. 2005.

[76] Z. Wang, X. Ma, and G. B. Giannakis, “OFDM or single-carrier block transmissions?,” IEEE Transactions on Communications, pp. 380–394, March 2004.

[77] D. Kim and G. Stuber, “Clipping noise mitigation for OFDM by decision-aided reconstruction,” IEEE Communications Letters, vol. 3, pp. 4–6, Jan 1999.

[78] J. Tellado, L. Hoo, and J. Cioﬃ, “Maximum-likelihood detection of nonlinearly distorted multicarrier symbols by iterative decoding,” IEEE Transactions on Com- munications, vol. 51, pp. 218–228, Feb 2003.

[79] G. Gelle, M. Colas, and D. Declercq, “Turbo decision aided reconstruction of clipping noise in coded OFDM,” IEEE 5th Workshop on Signal Processing Advances in Wireless Communications, 2004, pp. 591–595, July 2004.

[80] H. Nikopour, A. Khandani, and S. Jamali, “Turbo-coded OFDM transmission over a nonlinear channel,” IEEE Transactions on Vehicular Technology, vol. 54, pp. 1361–1371, July 2005. 80 References

[81] H. Zhang, X.-G. Xia, Q. Zhang, and W. Zhu, “Iterative decision-aided clipping compensation and its application to scalable video transmission with multiband OFDM,” IEEE Transactions on Vehicular Technology, vol. 56, pp. 756–765, March 2007.

[82] D. Declercq and G. Giannakis, “Recovering clipped OFDM symbols with Bayesian inference,” IEEE International Conference on Acoustics, Speech, and Signal Pro- cessing, 2000. ICASSP ’00. Proceedings. 2000, vol. 1, pp. 157–160 vol.1, 2000.

[83] P. Banelli, G. Leus, and G. B. Giannakis, “Bayesian estimation of clipped Gaus- sian processes with application to OFDM,” Proceedings of the European Signal Processing Conference (EUSIPCO), Sep. 2002.

[84] H. Chen and A. Haimovich, “Iterative estimation and cancellation of clipping noise for OFDM signals,” IEEE Communications Letters, vol. 7, pp. 305–307, July 2003.

[85] R. AliHemmati and P. Azmi, “Iterative reconstruction-based method for clipping noise suppression in OFDM systems,” IEE Proceedings- Communications, vol. 152, pp. 452–456, Aug 2005.

[86] U.-K. Kwon, G.-H. Im, and E.-S. Kim, “An iteration technique for recovering in- suﬃcient cyclic preﬁx and clipped OFDM signals,” IEEE Signal Processing Letters, vol. 14, pp. 317–320, May 2007.

[87] G. W. Wornell, “Spread-response precoding for communication over fading channels,” IEEE Transactions on Information Theory, pp. 488–501, March 1996.

[88] M. Debbah, P. Loubaton, and M. de Courville, “Spread OFDM performance with MMSE equalization,” IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001., pp. 2385–2388, May 2001.

[89] M. Park, H. Jun, and J. Cho, “PAPR reduction in OFDM transmission using Hadamard transform,” ICC 2000 - IEEE International Conference on Communi- cations, pp. 430–433, June 2000.

[90] Y. K. Min and V. Jeoti, “PAPR analysis of precoding-based OFDM systems,” International Conference on Intelligent and Advanced Systems, 2007. ICIAS 2007., pp. 457–461, Nov. 2007.

[91] S. Slimane, “Reducing the peak-to-average power ratio of OFDM signals through precoding,” IEEE Transactions on Vehicular Technology, vol. 56, pp. 686–695, March 2007. References 81

[92] Y.-P. Lin and S.-M. Phoong, “BER minimized OFDM systems with channel independent precoders,” IEEE Transactions on Signal Processing, pp. 2369–2380, Sep. 2003.

[93] M. Debbah, M. de Courville, and P. Maill´e, “Multiresolution decoding algorithm for Walsh-Hadamard linear precoded OFDM,” 7th International OFDM-Workshop 2002, Hamburg, Germany, Sep. 2002.

[94] Z. Wang, S. Zhou, and G. B. Giannakis, “Joint coding-precoding with low- complexity turbo-decoding,” IEEE Transactions on Wireless Communications, pp. 832–842, May 2004.

[95] Z. Liu, Y. Xin, and G. Giannakis, “Linear constellation precoding for OFDM with maximum multipath diversity and coding gains,” IEEE Transactions on Commu- nications, vol. 51, pp. 416–427, March 2003.

[96] W. Zou and Y. Wu, “COFDM: an overview,” IEEE Transactions on Broadcasting, vol. 41, pp. 1–8, Mar 1995.

[97] A. Jones, T. Wilkinson, and S. Barton, “Block coding scheme for reduction of peak to mean envelope power ratio of multicarrier transmission schemes,” Electronics Letters, vol. 30, pp. 2098–2099, Dec 1994.

[98] A. Jones and T. Wilkinson, “Combined coding for error control and increased robustness to system nonlinearities in OFDM,” IEEE 46th Vehicular Technology Conference, 1996. ’Mobile Technology for the Human Race’.,, vol. 2, pp. 904–908 vol.2, Apr-1 May 1996.

[99] S. Shepherd, J. Orriss, and S. Barton, “Asymptotic limits in peak envelope power reduction by redundant coding in orthogonal frequency-division multiplex modulation,” IEEE Transactions on Communications, vol. 46, pp. 5–10, Jan 1998.

[100] M. Sharif and B. Hassibi, “Existence of codes with constant PMEPR and related design,” IEEE Transactions on Signal Processing, vol. 52, pp. 2836–2846, Oct. 2004.

[101] M. Olfat and K. Liu, “Low peak to average power ratio cyclic Golay sequences for OFDM systems,” IEEE International Conference on Communications, 2004, vol. 2, pp. 993–997 Vol.2, June 2004. 82 References

[102] O. Muta and Y. Akaiwa, “Peak power reduction method based on structure of parity-check matrix for LDPC coded OFDM transmission,” IEEE 65th Vehicular Technology Conference, 2007. VTC2007-Spring., pp. 2841–2845, April 2007.

[103] B. Le Floch, M. Alard, and C. Berrou, “Coded orthogonal frequency division multiplex [TV broadcasting],” Proceedings of the IEEE, vol. 83, pp. 982–996, Jun 1995.

[104] S. Zhou, Z. Wang, N. Bapat, and G. Giannakis, “Turbo decoding of error control coded and unitary precoded OFDM,” Vehicular Technology Conference, 2002. VTC Spring 2002. IEEE 55th, vol. 3, pp. 1237–1241 vol.3, 2002.

[105] G. Yue and X. Wang, “A hybrid PAPR reduction scheme for coded OFDM,” IEEE Transactions on Wireless Communications, vol. 5, pp. 2712–2722, Oct. 2006.

[106] Y.-C. Tsai, S.-K. Deng, K.-C. Chen, and M.-C. Lin, “Turbo coded OFDM for reducing PAPR and error rates,” IEEE Transactions on Wireless Communications, vol. 7, pp. 84–89, Jan. 2008.

[107] H. Nikopour and S. H. Jamali, “On the performance of OFDM systems over a cartesian clipping channel - a theoretical approach,” IEEE Transactions on Wire- less Communications, pp. 2083–2096, Nov. 2004.

[108] H. Futaki and T. Ohtsuki, “Low-density parity-check (LDPC) coded OFDM systems,” IEEE Vehicular Technology Conference, 2001. VTC 2001 Fall. VTS 54th, vol. 1, pp. 82–86 vol.1, 2001.

[109] H. Futaki and T. Ohtsuki, “Performance of low-density parity-check (LDPC) coded OFDM systems,” IEEE International Conference on Communications, 2002. ICC 2002., vol. 3, pp. 1696–1700 vol.3, 2002.

[110] I. Djordjevic, B. Vasic, and M. Neifeld, “LDPC-coded OFDM for optical communication systems with direct detection,” IEEE Journal of Selected Topics in Quantum Electronics, vol. 13, pp. 1446–1454, Sep.-Oct. 2007.

[111] V. Mannoni, G. Gelle, and D. Declercq, “A linear criterion to optimize irregular LDPC codes for OFDM communications,” Vehicular Technology Conference, 2005. VTC 2005-Spring. 2005 IEEE 61st, vol. 1, pp. 678–682 Vol. 1, May-1 June 2005.

[112] A. de Baynast, A. Sabharwal, and B. Aazhang, “LDPC code design for OFDM channel: graph connectivity and information bits positioning,” International Sym- posium on Signals, Circuits and Systems, 2005. ISSCS 2005., vol. 2, pp. 649–652 Vol. 2, July 2005. References 83

[113] A. Sanaei and M. Ardakani, “LDPC code design for nonuniform power-line channels,” EURASIP Journal on Advances in Signal Processing, vol. 2007, pp. Article ID 76146, 9 pages, 2007. doi:10.1155/2007/76146.

[114] Z. Wang and G. Giannakis, “Linearly precoded or coded OFDM against wireless channel fades?,” IEEE Third Workshop on Signal Processing Advances in Wireless Communications, 2001. (SPAWC ’01). 2001, pp. 267–270, 2001.

[115] A. Serener, B. Natarajan, and D. M. Gruenbacher, “Performance of spread OFDM with LDPC coding in outdoor environments,” IEEE Vehicular Technology Confer- ence, pp. 318–321, Oct. 2003.

[116] A. Serener and D. M. Gruenbacher, “LDPC coded spread OFDM in indoor environments,” Proceedings of the 3rd IEEE International Symposium on Turbo Codes & Related Topics, France, pp. 549–552, Sep. 2003.

[117] A. Serener, B. Natarajan, and D. Gruenbacher, “Optimized LDPC codes for OFDM and spread OFDM in correlated channels,” IEEE 65th Vehicular Technology Con- ference, 2007. VTC2007-Spring., pp. 2276–2280, April 2007.

[118] A. Serener, B. Natarajan, and D. Gruenbacher, “Lowering the error ﬂoor of optimized short-block-length LDPC-coded OFDM via spreading,” IEEE Transactions on Vehicular Technology, vol. 57, pp. 1646–1656, May 2008.

[119] S. Borade, B. Nakiboglu, and L. Zheng, “Some fundamental limits of unequal error protection,” IEEE International Symposium on Information Theory, 2008. ISIT 2008., pp. 2222–2226, July 2008.

[120] H. Pishro-Nik, N. Rahnavard, and F. Fekri, “Nonuniform error correction using low- density parity-check codes,” IEEE Transactions on Information Theory, vol. 51, pp. 2702–2714, July 2005.

[121] B. Masnick and J. Wolf, “On linear unequal error protection codes,” IEEE Trans- actions on Information Theory, vol. 13, pp. 600–607, Oct 1967.

[122] W. Gore and C. Kilgus, “Cyclic codes with unequal error protection,” IEEE Trans- actions on Information Theory, vol. 17, pp. 214–215, Mar 1971.

[123] C. Kilgus and W. Gore, “A class of cyclic unequal error protection codes,” IEEE Transactions on Information Theory, vol. 18, pp. 687–690, Sep 1972. 84 References

[124] D. Mandelbaum, “Unequal error protection codes derived from diﬀerence sets,” IEEE Transactions on Information Theory, vol. 18, pp. 686–687, Sep 1972.

[125] I. Boyarinov and G. Katsman, “Linear unequal error protection codes,” IEEE Transactions on Information Theory, vol. 27, pp. 168–175, Mar 1981.

[126] M.-C. Lin and S. Lin, “Cyclic unequal error protection codes constructed from cyclic codes of composite length,” IEEE Transactions on Information Theory, vol. 34, pp. 867–871, Jul 1988.

[127] M.-C. Lin, C.-C. Lin, and S. Lin, “Computer search for binary cyclic UEP codes of odd length up to 65,” IEEE Transactions on Information Theory, vol. 36, pp. 924– 935, Jul 1990.

[128] J. Hagenauer, “Rate-compatible punctured convolutional codes (RCPC codes) and their applications,” IEEE Transactions on Communications, vol. 36, pp. 389–400, Apr 1988.

[129] L.-F. Wei, “Coded modulation with unequal error protection,” IEEE Transactions on Communications, vol. 41, pp. 1439–1449, Oct 1993.

[130] A. Calderbank and N. Seshadri, “Multilevel codes for unequal error protection,” IEEE Transactions on Information Theory, vol. 39, pp. 1234–1248, Jul 1993.

[131] M. Sajadieh, F. Kschischang, and A. Leon-Garcia, “Modulation-assisted unequal error protection over the fading channel,” IEEE Transactions on Vehicular Tech- nology, vol. 47, pp. 900–908, Aug 1998.

[132] S. Olivieri and M. Martini, “Dynamic bit-interleaved turbo-coded modulation for unequal error protection,” IEEE Global Telecommunications Conference, 2001. GLOBECOM ’01., vol. 6, pp. 3267–3271, 2001.

[133] M. Aydinlik and M. Salehi, “Turbo coded modulation for unequal error protection,” IEEE Transactions on Communications, vol. 56, pp. 555–564, April 2008.

[134] A. Barbulescu and S. Pietrobon, “Rate compatible turbo codes,” Electronics Let- ters, vol. 31, pp. 535–536, Mar 1995.

[135] F. Burkert, G. Caire, J. Hagenauer, T. Hindelang, and G. Lechner, “Turbo decoding with unequal error protection applied to GSM speech coding,” Global Telecommu- nications Conference, 1996. GLOBECOM ’96., vol. 3, pp. 2044–2048, Nov 1996. References 85

[136] G. Caire and E. Biglieri, “Parallel concatenated codes with unequal error protection,” IEEE Transactions on Communications, vol. 46, pp. 565–567, May 1998.

[137] G. Buch and F. Burkert, “Concatenated Reed-Muller codes for unequal error protection,” IEEE Communications Letters, vol. 3, pp. 202–204, Jul 1999.

[138] B. Vasic, A. Cvetkovic, S. Sankaranarayanan, and M. Marcellin, “Adaptive error protection low-density parity-check codes for joint source-channel coding schemes,” IEEE International Symposium on Information Theory 2003. ISIT 2003. Proceed- ings., pp. 267–267, July 2003.

[139] N. Rahnavard and F. Fekri, “Unequal error protection using low-density parity- check codes,” International Symposium on Information Theory, 2004. ISIT 2004. Proceedings., pp. 449–, July 2004.

[140] N. Rahnavard, H. Pishro-Nik, and F. Fekri, “Unequal error protection using partially regular LDPC codes,” IEEE Transactions on Communications, vol. 55, pp. 387–391, Mar. 2007.

[141] N. Rahnavard and F. Fekri, “New results on unequal error protection using LDPC codes,” IEEE Communications Letters, vol. 10, pp. 43–45, Jan 2006.

[142] C. Poulliat, D. Declercq, and I. Fijalkow, “Optimization of LDPC codes for UEP channels,” in Proc. ISIT 2004, p. 451, June 2004.

[143] C. Poulliat, D. Declercq, and I. Fijalkow, “Enhancement of unequal error protection properties of LDPC codes,” EURASIP Journal on Wireless Communications and Networking, vol. 2007, pp. Article ID 92659, 9 pages, 2007. doi:10.1155/2007/92659.

[144] L. Sassatelli, W. Henkel, and D. Declercq, “Check-irregular LDPC codes for unequal error protection under iterative decoding,” in 4th International Symposium on Turbo-codes and related topics (D. D. L. Sassatelli, W. Henkel, ed.), 2006.

[145] A. Goupil and D. Declercq, “UEP non-binary LDPC codes: A promising framework based on group codes,” IEEE International Symposium on Information Theory, 2008. ISIT 2008., pp. 2227–2231, July 2008.

[146] V. Kumar and O. Milenkovic, “On unequal error protection LDPC codes based on Plotkin-type constructions,” IEEE Transactions on Communications, vol. 54, pp. 994–1005, June 2006. 86 References

[147] W. Ruihua, Y. Liuguo, Y. Quan, and L. Jianhua, “Design of irregular LDPC code with unequal error protection property and its performance analysis on image transmissions,” Journal of Electronics (China), vol. 22, pp. 658–662, Nov 2005.

[148] N. Thomos, N. Boulgouris, and M. Strintzis, “Product code optimization for deter- minate state LDPC decoding in robust image transmission,” IEEE Transactions on Image Processing, vol. 15, pp. 2113–2119, Aug. 2006.

[149] C. Zhong and J. Havlicek, “LDPC codes for robust transmission of images over wireless channels,” Conference Record of the Thirty-Fifth Asilomar Conference on Signals, Systems and Computers, 2001., vol. 1, pp. 797–800 vol.1, 2001.

[150] L. Pu, M. W. Marcellin, B. Vasic, and A. Bilgin, “Unequal error protection and progressive decoding for JPEG2000,” Image Communications, vol. 22, no. 3, pp. 340– 346, 2007.

[151] P. Marti-Puig and J. Sala-Alvarez, “A fast OFDM-CDMA user demultiplexing architecture,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2000. ICASSP ’00., pp. 3358–3361, June 2000.

[152] H. Sankar, N. Sindhushayana, and K. Narayanan, “Design of low-density parity- check (LDPC) codes for high order constellations,” in Proc. Globecom 2004, pp. 3113–3117, Nov. 2004. Part III

Research Papers 88 Paper A Performance of LDPC Coded Spread OFDM with Clipping

Authors: Sara Sandberg, Cormac de Fr´ein, James P. LeBlanc, Bane Vasic and Anthony D. Fagan

Reformatted version of paper originally published in: Proceedings of the 8th International Symposium on Communication Theory and Appli- cations (ISCTA) 2005, UK, pp. 156-161.

c 2005, ISCTA 2005. Reprinted with permission.

89 90 PERFORMANCE OF LDPC CODED SPREAD OFDM WITH CLIPPING

Sara Sandberg, Cormac de Fr´ein, James P. LeBlanc, Bane Vasic and Anthony D. Fagan

Abstract

We present the performance of OFDM systems with coding, spreading, and clipping. Low-Density Parity-Check (LDPC) codes give coding gains and spreading by the Walsh Hadamard transform gives gains in terms of increased frequency diversity as well as reduced peak-to-average power ratio (PAPR) of the transmitted OFDM signal. By evaluating both the IFFT transform (OFDM) and the Walsh Hadamard transform in a single step, the number of operations needed for the spread OFDM system is actually less than for the conventional OFDM system. Reducing the PAPR is important in systems with clipping since it is related to the probability of clips. Each clip introduces clipping noise to the system which reduces the performance. Results of a clipped OFDM system with LDPC coding and spreading for an ETSI indoor wireless channel model are presented and compared with other systems. It is shown that there is a gain by spreading for LDPC coded OFDM systems, and especially for systems with clipping.

1 Introduction

In wireless communications, the channel is often time-varying due to relative transmitter- receiver motion and reflections. This time-variation, called fading, reduces system performance. With a high data rate compared to the channel bandwidth, multipath propagation becomes frequency-selective and causes intersymbol interference (ISI). A multicarrier OFDM system is known to transform a frequency-selective fading channel into parallel flat-fading subchannels if a cyclic prefix is used for preventing inter-block interference. The receiver complexity is thereby significantly reduced, since the equalizer can be implemented as a number of one-tap filters. In such a system, the data transmitted on some of the carriers might be strongly attenuated and could be unrecoverable at the receiver. Lately, spread spectrum techniques have been combined with the conventional OFDM to better exploit frequency diversity, [1,2]. This combination implies spreading information across all (or some of) the carriers by precoding with a unitary matrix and is in the 92 Paper A following referred to as spread OFDM (SOFDM). Another way to resist data corruption on fading subchannels is to use error-correcting codes. In [3], joint precoding and coding of the OFDM system is suggested with convolutional codes or turbo-codes and it is shown that there is a significant performance gain offered by introducing precoding to the coded transmission. In the last decade low- density parity-check (LDPC) codes, first invented by Gallager in 1962 [4], have attracted attention, see e.g. [5]. Serener et al. investigate the performance of SOFDM with LDPC coding in [6, 7]. One of the major drawbacks with the OFDM system is its high peak-to-average power ratio (PAPR). A high PAPR corresponds to a high probability of clipping in the power amplifier in the transmitter or, alternatively, a large input power backoff. This implies reduced signal power, degrading bit error rate and for clipping even spectral spreading. There has been much research in the area of reducing the PAPR for OFDM systems, [8,9]. It is shown in [10] that precoding by the Walsh Hadamard (WH) matrix reduces the PAPR of the OFDM signal and the associated reduced probability of clipping distortion will increase the performance of the system. This precoding scheme has also been suggested for spreading, [1]. Surprisingly, the joint WH spreading and OFDM modulation can be performed by one single transformation that requires less operations than the IFFT alone, [11]. In this paper, the total performance gain of the WH spreading is investigated for an OFDM system with LDPC coding and clipping. In particular, the gain in bit-error-rate performance is analyzed for an ETSI channel model and results for clipped OFDM signals are provided. The conventional and SOFDM system as well as the channel model are described in Section 2 and the application of LDPC codes to OFDM in Section 3. Results of the WH spreading applied to clipped and non-clipped OFDM signals follow in Section 4. Finally, Section 5 gives some concluding remarks.

2 The OFDM System

The OFDM modulation is obtained by applying Inverse FFT (IFFT) to N message subsymbols, where N is the number of subchannels in the OFDM system. The baseband signal can be written

N−1 1 j2πnk/N xn = √ Xk · e ,n=0,...,N − 1(1) N k=0 Paper A 93

v X x y Y Xˆ IFFT P/S h S/P FFT EQ N N N N N

Figure 1: Conventional OFDM.

where Xk is symbol number k in a random message stream, also called the frequency domain signal. Thus the modulated OFDM vector can be expressed as

x = IFFT{X} (2)

A block diagram describing the conventional OFDM system is shown in Figure 1. h is the channel impulse response and v is a vector of uncorrelated complex Gaussian random variables with variance σ2. The output from the FFT is

Y = CX + W (3)

where the diagonal matrix C = diag(c1,c2,...,cN ) gives the frequency domain channel attenuations and W is the FFT of the noise v. The elements of W are still uncorrelated complex Gaussian random variables with variance σ2 due to the unitary property of the FFT. The zero-forcing equalizer is considered for conventional OFDM, but Wiener equalizers are also used. A practical implementation of OFDM usually uses a cyclic preﬁx in order to avoid inter-symbol interference (ISI). In SOFDM, the frequency domain signal is multiplied by a spreading matrix Θ before it is fed to the IFFT, Figure 2. The spreading considered here is the WH matrix that can be generated recursively for sizes a power of two [12]. The (2 × 2) WH matrix is given by 1 11 WH2 = √ . (4) 2 1 −1 The (2m × 2m) WH matrix is given in terms of the (m × m) WH matrix, 1 WHm WHm WH2m = √ . (5) 2 WHm −WHm

In the following Θ is assumed to be the (N × N) WH matrix. The WH transform is an orthogonal linear transform that can be implemented by a butterﬂy structure as the IFFT and since the WH and IFFT transforms can be combined and calculated with less complexity than the IFFT alone this means that the transmitter complexity is reduced 94 Paper A

v ˆ X X˜ x˜ y˜ Y˜ X Θ IFFT h FFT G N N N N

Figure 2: Spread OFDM. by applying WH spreading. At the receiver side, Wiener filtering is performed and the output of the Wiener filter is the vector Xˆ given by Xˆ = GY˜ . G is defined as

H 2 T G = argminW W Xˆ − X =Θ F (6) where F is a diagonal matrix, c∗ c∗ F = diag 1 ,..., N (7) 2 2 2 2 |c1| + σ |cN | + σ and (·)∗ denotes conjugation. This means that the receiver consists of scalar channel equalization followed by the transpose of the spreading matrix, which in the case of unitary spreading equals the inverse. In the following, the noise power σ2 and the frequency domain channel attenuations ci are assumed to be known. The wireless channel model used in this work is the ETSI indoor wireless channel model for HIPERLAN/2, [13]. Wireless channels can be modeled as Rayleigh fading channels. When a signal is sent from the transmitter across a wireless channel, it trav- els via many paths (due to reflections, refractions or diffractions) to the receiver. The different path lengths result in time delays or phase differences between the multipath components, which leads to constructive and destructive interference. The received signal jφy y can be written y = rye where ry is the amplitude and φy is the phase. The received amplitude is Rayleigh distributed with the probability density function

ry ry p(ry)= exp(− )0≤ ry < ∞ (8) δ 2δ and the received phase is uniformly distributed with 1 p(φy)= 0 ≤ φy < 2π (9) 2π where δ is the mean power of the waveform. A sample wireless channel response is shown in Figure 3. Two deep spectral nulls are apparent in this example. In Conventional OFDM, each subchannel is assigned one subsymbol. In the example channel given above, the subsymbols transmitted on the subchannels in the close vicinity of the two deep spectral nulls, will have a high probability of error. However, using the Paper A 95

−10

−20 Subchannel Power Gain (dB)

−30 8 16 24 32 40 48 56 64 Subchannel Index

Figure 3: Example of a wireless channel response following the ETSI indoor wireless channel model for HIPERLAN/2. spreading employed in this work, the information transmitted on each subchannel will be a linear combination of the original N subsymbols. This means that instead of a few subsymbols being severely aﬀected by spectral nulls, several subsymbols are lightly aﬀected. This approach leads to improved BER.

3 LDPC codes for OFDM and SOFDM

Error control codes used in this paper belong to a class known as low-density parity-check (LDPC) codes [4]. An LDPC code is a linear block code and it can be conveniently described through a graph commonly referred as a Tanner graph [14]. Such a graphical representation facilitates a decoding algorithm known as the message-passing algorithm. A message-passing decoder has been shown to virtually achieve Shannon capacity when long LDPC codes are used. In the next paragraph we will describe a speciﬁc class of LDPC codes used here. For more details on message passing decoding the reader is referred to an excellent introduction by Kschischang et al. [15].

Consider a linear block code C of length n deﬁned as the solution-space (in F2)ofthe system of linear equations Hx = 0,whereH is an m × n binary matrix. The bipartite graph representation of C is denoted by G. G contains a set of n variable nodes and a 96 Paper A

v ˜ ˜ ˆ cˆ m c X X x˜ y˜ Y X LDPC LDPC Modu− Θ IFFT FFT G Encoding lation h Decoding N N N N

Figure 4: SOFDM with LDPC coding. set of m check nodes, i.e. nodes corresponding to equations in Hx = 0. A variable node is connected with a check node if it belongs to a corresponding equation. More precisely, the i-th column of H corresponds to a variable node xi of the graph G,andthej-th row of the matrix corresponds to a check node Sj of G. The choice of a parity check matrix that supports the message-passing algorithm is a problem that has been extensively studied in recent years, and many random [16] and structured codes have been found [17]. We have chosen codes from a family of rate-compatible array codes, [18, 19], because they support a simple encoding algorithm and have low implementation complexity. The general form of the parity check matrix can be written as ⎛ ⎞ I I I ... P ,m/k I ... I ⎜ 1 ⎟ ⎜ ⎟ ⎜ 0 IP2,3 ... P2,m/k P2,(m+1)/k ... P2,n/k ⎟ ⎜ ⎟ H = ⎜ 00 IP3,m/k P3,(m+1)/k ... P3,n/k ⎟ (10) ⎜ . . . . ⎟ ⎝ 00 0 ...... ⎠

00 0 0 IPm/k,(m+1)/k ... Pm/k,n/k where each of submatrices Pi,j,isapowerofak×k permutation matrix (see [19]). Notice that the row and column weights of H vary, i.e. the code is irregular. A column j in a set of m/k leftmost columns has the weight j, while the rest of the columns have weight m/k. Figure 4 shows a block diagram describing the SOFDM system with LDPC coding, denoted by LDPC-SOFDM. The message bits are first encoded to codewords of length n and modulated to n/2 QPSK symbols. The modulation symbols are ± Es/2 ± j Es/2, where Es is the symbol energy. The modulated codeword is partitioned into blocks of N samples, where we assume that n ≥ 2N, and each block is multiplied by the spreading matrix Θ. The spread signal is sent through the OFDM system and the Wiener filter. Both soft information from the output of the Wiener filter and the SNR for each subchannel are fed to the decoder. Since spreading averages SNR of different subchannels, a theoretical SNR taking the spreading into account is computed and used for the decoding. The theoretical SNR can be calculated from the output of the Wiener filter, which is Xˆ = GY˜ =ΘT FCΘX +ΘT FW. (11) Paper A 97

Component k of Xˆ can be written

Xˆk = αXk + βk + δk (12) where N 1 α = μi (13) N i =1 N N βk = ΘjkΘjiμj Xi (14) i=1,i= k j=1 and N ∗ ci δk = Θik Wi. (15) | |2 2 i ci + σ =1 √ 2 2 2 Θij is element (i, j) in the WH matrix, Θij = ±1/ N,andμi = |ci| /(|ci| + σ ). For large N the interference-plus-noise at the output of the Wiener ﬁlter (βk + δk)canbe considered to be Gaussian, [2], and approximated as noise. The theoretical SNR of subchannel k with spreading is

Var{αXk} SNRk = = (16) Var{βk} +Var{δk}

2 α Es = $ % (17) 2 2 N N σ2 N |ci| ΘjkΘjiμj Es + 2 2 2 i=1,i= k j=1 N i=1 (|ci| +σ ) For conventional OFDM (Θ = I), the SNR is simply

2 Var{ciXk} |ci| Es SNRk = = (18) σ2 σ2

4Results

To show the performance of LDPC-SOFDM, simulations are performed with rate 0.8 lattice codes and a codeword length of 1024. The maximum column weight of the parity- check matrix is 3. The performance is an average over diﬀerent channel realizations of the ETSI indoor wireless channel model and the channel realizations are normalized to have energy N,thatis, N 2 |ci| = N (19) i=1 Figure 5 shows the BER performance of diﬀerent OFDM systems without clipping. The OFDM system has 64 subchannels and the channel is assumed to be constant during 98 Paper A

−1 10 Conv. OFDM SOFDM LDPC−OFDM −2 10 LDPC−SOFDM

−3 10 BER −4 10

−5 10

−6 10 10 15 20 25 SNR (dB)

Figure 5: Performance of conventional OFDM, SOFDM, LDPC-OFDM, and LDPC-SOFDM for the case of 64 subchannels.

the transmission of one codeword. Both SOFDM and LDPC coding give a large gain compared with conventional OFDM, but there is also a gain by spreading of about 2.7 dB of the LDPC coded system at 10−5 bit-error-rate. Figure 6 shows that the performance does not change much with the number of subcarriers, which is also the size of the precoder. However, with a large number of subchannels, like 512, interleaving is necessary for LDPC-OFDM to get this performance.

A comparison to the work in [2] for convolutional codes is shown in Figure 7. The frequency domain channel attenuations ci in [2] are assumed to be independent identically distributed circular complex Gaussian random variables with variance 1. A convolutional encoder with constraint length 7 is used and the code rate is 3/4. The spreading and equalizer is the same as in this paper and an OFDM system with 64 subchannels is used. We compare this with our LDPC-SOFDM system with rate 0.8 for the same channel parameters as in [2]. Figure 7 shows that the LDPC-SOFDM system performs better than the system with the convolutional code.

Figure 8 shows the reduction of PAPR that is the result of the WH spreading. The instantaneous PAPR for the kth OFDM block, and the overall PAPR are deﬁned, re- Paper A 99

−2 10 N=64 LDPC−OFDM N=64 LDPC−SOFDM N=512 LDPC−OFDM −3 10 N=512 LDPC−SOFDM

−4 10 BER

−5 10

−6 10 10 11 12 13 14 15 SNR (dB)

Figure 6: Performance of LDPC-OFDM and LDPC-SOFDM for both 64 and 512 subchannels.

0 10 Conv−SOFDM LDPC−SOFDM

−1 10

−2 10 BER −3 10

−4 10

−5 10 5 6 7 8 9 10 11 SNR (dB)

Figure 7: Comparison with a convolutional coded SOFDM system. 100 Paper A spectively, by

2 x(k) ∞ PAPRk = (20) { 2} E x(k) 2 /N

2 maxk x(k) PAPR= ∞ (21) { 2} E x(k) 2 /N where x(k)isthekth block of the time domain signal vector. However, in practice it is more useful to know the distribution of the instantaneous PAPR. In Figure 8 the overall PAPR has decreased by 1.1 dB and the mean instantaneous PAPR has decreased by 0.8 dB by spreading. In many systems clipping occurs in the power ampliﬁer and a reduction of PAPR jφ reduces the number of clips. If xi = rie i denotes the input complex signal, the clipping jφ of the baseband signal can be modeled ass ˇi =ˇrie i ,with ri, for ri ≤ Amax rˇi = (22) Amax, for ri >Amax where Amax is the maximum output amplitude. The clipping ratio γ is deﬁned as

Amax γ = √ . (23) Es Figure 9 shows the BER in a case where the signal is clipped with a clipping ratio of 2 dB. The total gain of spreading is around 4 dB at 10−5 bit-error-rate, compared to 2.7 dB when there is no clipping, since the PAPR reduction by spreading has reduced the number of clips. The eﬀect of clipping noise to the SNR is not taken into account in the decoder. This result suggests that spreading is of extra value in systems where there is a high probability of clipping.

5 Conclusions

In this paper the BER performance of LDPC-SOFDM is investigated. The spreading considered is the WH transform which actually can reduce the complexity of the system. Results for the ETSI indoor wireless channel model show that using LDPC-SOFDM instead of LDPC-OFDM in a system with clipping gives a gain of 4 dB at a bit-error-rate of 10−5. The gain is due to increased frequency diversity as well as reduced PAPR. The performance is also investigated for diﬀerent number of subchannels and the results show that systems with diﬀerent number of subchannels perform almost the same. However, a large number of subchannels increases the PAPR which in turn increases the probability of Paper A 101

(hist(PAPR)) 2 10 log 0 0 5 10 15 20 25 30 35 (a) PAPR 6

(hist(PAPR)) 2 10 log 0 0 5 10 15 20 25 30 35 (b) PAPR

Figure 8: Histograms of the PAPR for an OFDM system with 64 subchannels, with (a) and without (b) spreading.

−2 10 LDPC−OFDM LDPC−SOFDM LDPC−OFDM 2 dB clip LDPC−SOFDM 2 dB clip −3 10

−4 10 BER

−5 10

−6 10 10 11 12 13 14 15 16 SNR (dB)

Figure 9: Performance of LDPC-OFDM and LDPC-SOFDM with 64 subchannels. The performance is shown both for no clipping and for a clipping ratio of 2 dB. 102 Paper A clips. Our results conﬁrm that spreading enhances the performance of the OFDM system for the ETSI channel model and show that especially the performance in a system with clipping is increased, while the complexity is reduced.

References

[1] Y.-P. Lin and S.-M. Phoong, “BER minimized OFDM systems with channel independent precoders,” IEEE Transactions on Signal Processing, pp. 2369–2380, Sep. 2003.

[2] M. Debbah, P. Loubaton, and M. de Courville, “Spread OFDM performance with MMSE equalization,” IEEE International Conference on Acoustics, Speech, and Sig- nal Processing, 2001., pp. 2385–2388, May 2001.

[3] Z. Wang, S. Zhou, and G. B. Giannakis, “Joint coding-precoding with low- complexity turbo-decoding,” IEEE Transactions on Wireless Communications, pp. 832–842, May 2004.

[4] R. G. Gallager, “Low density parity check codes,” IRE Transactions on Information Theory, pp. 21–28, Jan. 1962.

[5] D. J. C. MacKay, “Good error-correcting codes based on very sparse matrices,” IEEE Transactions on Information Theory, pp. 399–431, March 1999.

[6] A. Serener, B. Natarajan, and D. M. Gruenbacher, “Performance of spread OFDM with LDPC coding in outdoor environments,” IEEE Vehicular Technology Confer- ence, pp. 318–321, Oct. 2003.

[7] A. Serener and D. M. Gruenbacher, “LDPC coded spread OFDM in indoor environments,” Proceedings of the 3rd IEEE International Symposium on Turbo Codes & Related Topics, France, pp. 549–552, Sep. 2003.

[8] P. V. Eetvelt, G. Wade, and M. Tomlinson, “Peak to average power reduction for OFDM schemes by selective scrambling,” Electronics Letters, pp. 1963–1964, Oct. 1996.

[9] K. Yang and S.-I. Chang, “Peak-to-average power control in OFDM using standard arrays of linear block codes,” IEEE Communications Letters, pp. 174–176, April 2003. Paper A 103

[10] M. Park, H. Jun, and J. Cho, “PAPR reduction in OFDM transmission using Hadamard transform,” ICC 2000 - IEEE International Conference on Communi- cations, pp. 430–433, June 2000.

[11] P. Marti-Puig and J. Sala-Alvarez, “A fast OFDM-CDMA user demultiplexing architecture,” Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2000. ICASSP ’00., pp. 3358–3361, June 2000.

[12] N. Ahmed and K. R. Rao, Orthogonal transforms for digital signal processing. Springer Verlag, 1975.

[13] ETSI, “Channel models for HIPERLAN/2 in diﬀerent indoor scenarios,” COST 256 TD(98), April 1998.

[14] M. Tanner, “A recursive approach to low complexity codes,” IEEE Transactions on Information Theory, pp. 533–547, Sep. 1981.

[15] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum- product algorithm,” IEEE Transactions on Information Theory, pp. 498–519, Feb. 2001.

[16] D. J. C. MacKay, “Relationships between sparse graph codes,” Proc. of IBIS 2000, Japan, 2000. Available online at http://www.inference.phy.cam.ac.uk/mackay/ abstracts/ibis.html.

[17] B. Vasic and O. Milenkovic, “Combinatorial constructions of low-density parity- check codes for iterative decoding,” IEEE Transactions on Information Theory, pp. 1156–1176, June 2004.

[18] E. Eleftheriou and S. Ol¸¨ cer, “Low-density parity-check codes for digital subscriber lines,” Proceedings, IEEE International Conference on Communications, pp. 1752– 1757, Apr. 2002.

[19] A. Dholakia and S. Ol¸¨ cer, “Rate-compatible low-density parity-check codes for digital subscriber lines,” Proc., IEEE International Conference on Communications, pp. 415–419, Jun. 2004. 104 Paper B Receiver-oriented Clipping-eﬀect Mitigation in OFDM - A Worthy Approach?

Authors: Sara Sandberg, James P. LeBlanc and Bane Vasic

Reformatted version of paper originally published in: Proceedings of the 10th International OFDM-Workshop 2005, Hamburg.

c 2005, OFDM-Workshop 2005. Reprinted with permission.

105 106 RECEIVER-ORIENTED CLIPPING-EFFECT MITIGATION IN OFDM - A WORTHY APPROACH?

Sara Sandberg, James P. LeBlanc and Bane Vasic

Abstract The high peak-to-average power ratio (PAR) in Orthogonal Frequency Division Multi- plexing (OFDM) modulation systems can signiﬁcantly reduce performance and power eﬃciency. A number of methods exist that combat large signal peaks in the transmitter. Recently several methods have emerged that alleviate the clipping distortion in the receiver. We analyze the performance of two receiver clipping mitigation methods in an OFDM system with Cartesian clipping and low-density parity-check (LDPC) coding. Surprisingly, the cost of completely ignoring clipping in the receiver is minimal, even though we assume that the receiver has perfect knowledge of the channel. The results suggest that clipping mitigation strategies should be concentrated to the transmitter.

1 Introduction

It is well known that OFDM signals may suffer from a high peak-to-average power ratio (PAR), tending towards a Gaussian distribution for a large number of sub-carriers. There is a vast research literature on transmit-oriented signal processing and coding methods to mitigate problems associated with clipping due to these high PAR values, see [1–3]. However, many such methods aimed at lowering PAR come at the cost of increased transmitter complexity or lowering transmission rate. Thus, one may consider allowing such occasional clips due to high PAR transmit signals and suffer the associated loss due to clipping. Of course, one would still seek receiver methods to minimize such clipping loss. A number of recent papers document the loss of untreated clipping, for example [4,5]. Furthermore, there is on-going research on understanding the clipping effects in the received signal, as well as ways to mitigate their effects, [6, 7]. In this paper we make an attempt to quantify in the best practical sense how much the clipping loss can be mitigated. Surprisingly, even when using near optimal methods 108 Paper B

v mˆ m c QPSK X x Z QPSK r LDPC LDPC yzFFT Modu− IFFT Demodu− Decoding Encoding lation lation N N

Figure 1: LDPC coded OFDM system with clipping. which involve signiﬁcant calculations (i.e. they lie beyond near-term practicality) we ﬁnd that the mitigation ability of such extreme processing is rather limited. To evaluate the limits of receiver oriented processing, we choose to combine best known practical coding methods, low-density parity-check (LDPC) codes, with the exact distribution of the clipping distortion. This combination could be expected to represent the best practically achievable system performance. It is assumed throughout the paper that the variance of the additive white Gaussian noise (AWGN) is perfectly known. Another way to combat the clipping distortion in the receiver is to estimate the signal before clipping from the clipped signal. In [7], a Bayesian estimator for the Cartesian clipper is derived. In this paper we show that for an OFDM system with LDPC coding, the ability to counter clipping gives an improvement of only 0.1 dB at 10−6 bit-error-rate. To wit, the cost of completely ignoring clipping in the receiver appears to be minimal. These results are intended to give insight into the potentially unrecoverable nature of the clipping phenomena and imply directing attention to transmit-oriented strategies.

2 System Description and Channel Model

The system model used throughout the paper is an OFDM system with Cartesian clipping and AWGN. LDPC codes are utilized for error-correction and the encoded bits are QPSK- modulated, see Figure 1. In this section the OFDM system, the channel model and the LDPC codes will be discussed in turn. The OFDM modulation is obtained by applying Inverse FFT (IFFT) to N QPSK modulated message (or codeword) subsymbols, where N is the number of subchannels in the OFDM system. The complex baseband signal (also called the time-domain signal) can be written N−1 1 j2πnk/N xn = √ Xk · e ,n=0,...,N − 1(1) N k=0 where Xk is subsymbol number k in the message stream, also called the frequency domain signal. Thus the OFDM modulated signal vector can be expressed as

x = IFFT{X}. (2) Paper B 109

I Q I Q The time-domain signal xn can be represented by xn + jxn ,wherexn and xn are the in-phase/quadrature (I/Q) components. A practical implementation of OFDM usually uses a cyclic prefix in order to avoid inter-symbol interference (ISI). I Q In this paper we focus on the Cartesian clipper that clips xn and xn separately and I Q in the followingx ñ denotes either xn or xn ,sothat˜xn is always a real sequence. The distorted signal is modeled as the output from the ideal clipper ⎧ ⎨⎪ −x0, xñ ≤−x0 yñ = g(˜xn)= xñ, −x < xñ < +x (3) ⎩⎪ 0 0 +x0, xñ ≥ +x0 where x0 is the clipping level andy ñ is the clipped I or Q component. The complex I Q I Q clipped signal is yn = yn + jyn = g(xn)+jg(xn ). Following the clipper, complex white 2 2 Gaussian noise v with variance 2σv (each noise component has variance σv ) is added to the signal. In the receiver, the clipped and noisy signal is demodulated by the OFDM FFT demodulator. Error control codes used in this paper belong to a class known as LDPC codes [8]. An LDPC code is a linear block code and it can be conveniently described through a graph commonly referred to as a Tanner graph [9]. Such a graphical representation facilitates a decoding algorithm known as the message-passing algorithm. A message-passing decoder has been shown to virtually achieve Shannon capacity when long LDPC codes are used. In the next paragraph we will describe a specific class of LDPC codes used here. For more details on message passing decoding the reader is referred to an excellent introduction by Kschischang et al. [10].

Consider a linear block code C of length n deﬁned as the solution-space (in F2)of the system of linear equations Hx = 0,whereH is an m × n binary matrix. The choice of a parity check matrix that supports the message-passing algorithm is a problem that has been extensively studied in recent years, and many random [11] and structured codes have been found [12]. We have chosen codes from a family of rate-compatible array codes, [13, 14], because they support a simple encoding algorithm and have low implementation complexity. The general form of the parity check matrix can be written as ⎛ ⎞ I I I ... P ,m/k I ... I ⎜ 1 ⎟ ⎜ ⎟ ⎜ 0 IP2,3 ... P2,m/k P2,(m+1)/k ... P2,n/k ⎟ ⎜ ⎟ H = ⎜ 00 IP3,m/k P3,(m+1)/k ... P3,n/k ⎟ (4) ⎜ . . . . ⎟ ⎝ 00 0 ...... ⎠

00 0 0 IPm/k,(m+1)/k ... Pm/k,n/k 110 Paper B

where each of the submatrices Pi,j is a power of a k × k permutation matrix, see [14]. Notice that the row and column weights of H vary, i.e. the code is irregular. In the message-passing decoder, log-likelihood ratios (LLRs) are updated and passed between nodes. The wanted LLRs are the a posteriori probabilities that a given bit in c equals 0 or 1 given the whole received word r. The initial LLR for the ith node (or codeword bit) is calculated as

Pr(ci =0|ri) LLRi =log (5) Pr(ci =1|ri) where Pr(A|B) denotes the conditional probability of A given B. When the LLRs are updated by the message-passing algorithm, they will better approximate the wanted LLRs.

3 Characterization of Clipping Noise

The description of the clipping distortion in this section mainly follows the characterization given in the recent paper [6]. The time-domain OFDM signal xn is the sum of several statistically independent subcarriers and the I/Q components can be approximated by Gaussian processes, invoking the central limit theorem, if the number of subcarriers is large. The pdf ofx ñ is therefore assumed to be Gaussian with mean zero and variance 2 2 σx,where2σx is the power of each complex frequency-domain symbol Xn. Knowing that the time-domain signal is approximately Gaussian, the Bussgang theorem can be applied, [15]. It states that the output of the clipper can be expressed as yñ = αxñ +˜wn,where˜wn is uncorrelated with the clipper input signalx ñ and α is an at- ˜ tenuation. The notation (·)n denotes as before either the I or Q component of a complex signal. The attenuation α is dependent only on the input backoff (ibo) of the clipper and for the Cartesian clipper defined in (3) the attenuation is given by √ α =1− 2Q( ibo)(6) where the Q-function and the ibo is defined as ∞ 1 2 Q(x)= √ e−x /2 (7) x 2π x2 ibo = 0 (8) 2 σx

The part of the clipped signal that has no correlation with the input signal, wn, is called the clipping noise. The pdf of the I/Q component of the time-domain clipping noise is Paper B 111 derived in [6] and is given by ⎧ ⎪ G(˜w;+x ,α2σ2), w˜ ≤−βx ⎪ 0 x 0 ⎪ 2 2 ⎨⎪ G(˜w; −x0,α σx) 2 2 pw˜(˜w)= +G(˜w;0,β σ ) (9) ⎪ x ⎪ G 2 2 − ⎪ + (˜w;+x0,α σx), βx0 < w<˜ +βx0 ⎩ 2 2 G(˜w; −x0,α σx), w˜ ≥ +βx0 where β =1− α and 1 (x − μ)2 G(x, μ, σ2)=√ exp − (10) 2πσ2 2σ2 2 is the Gaussian pdf with mean μ and variance σ .Thepdfpw˜(˜w)oftheI/Qcomponent of the clipping noise can be re-written as a function of the clipping parameters ibo and x0 only. Before LDPC decoding, the clipped and noisy received signal zn is demodulated by the FFT. The AWGN after the FFT will have the same variance as before the FFT, due to the unitary property of the FFT. This will not be the case for the clipping noise since it is not assumed to be Gaussian. The frequency-domain clipping noise over the kth subchannel, which is the output of the FFT when the input is the time-domain clipping noise, is N−1 1 I Q −j2πkn/N Wk = √ (wn + jwn )e (11) N n=0

The W˜ k (I or Q component) is a sum of 2N approximately i.i.d. random variables and its pdf is the convolution of 2N probability density functions. To avoid the convolutions the pdf of W˜ k can be calculated as the product of the characteristic function ofw ñ in different points. The characteristic function ofw ñ (the Fourier transform of its pdf) is given by, [6], 2 2 2 √ −β σxω /2 Φw˜(ω)=e 1 − 2Q( ibo + jωβσx) 2 2 2 √ −α σ ω /2 jωx0 + e x e 2Q( ibo + jωβσx) (12) where {·} denotes the real part. The characteristic function of the frequency-domain clipping noise can now be written as N−1 | | cos(2√πkn/N) ΦW˜ k (ω)= Φw˜ ω n=0 N |sin(2πkn/N)| Φw˜ √ ω (13) N 112 Paper B

˜ using (11) and (12), [6]. The pdf of Wk is the inverse Fourier transform of ΦW˜ k (ω), but it is usually easier to ﬁnd the cumulative distribution function by a numerical method called the Fourier series method, [16]. Since the clipping noise pdf is only dependent on the ibo of the clipper, this pdf must be calculated only once for a system with a given ibo and then a lookup table can be created. In the system discussed here, the clipping noise pdf is convolved with the AWGN pdf to give the pdf of the total noise for subchannel j, denoted puj (uj). If the channel is frequency ﬂat, the pdf of the total noise will be the same for all subchannels. The initial log-likelihood ratios for the decoder are calculated from this pdf according to (5), while accounting for the attenuation α. The conditional probability Pr(ci = c|ri)wherec is either 0 or 1 can be written | pri|c(ri ci = c)Pr(ci = c) Pr(ci = c|ri)= (14) pri (ri) | where pri (ri) is the pdf of the received signal and pri|c(ri ci = c) is the conditional density of the received signal given the corresponding codeword bit. In the case where 0 and 1 are equally likely to be transmitted, the LLR for the ith node is | pri|c(ri ci =0) LLRi =log . (15) | pri|c(ri ci =1)

The LLRs can be calculated from the pdf of the total noise and the attenuation α by − puj (ri ασx) LLRi =log (16) puj (ri + ασx) since each codeword bit is represented by (−1)ci before applying the IFFT.

4 Bayesian Estimation

Another way to counter the clipping in the receiver is to estimate the time-domain signal xn from the clipped and noisy signal zn. In [7], a Bayesian estimator for signals clipped by the Cartesian clipper and distorted by AWGN is derived. The Bayesian estimator is the optimal estimator of xn given zn in the Minimum Mean-Square Error (MMSE) sense, given by ∞

x˜ôpt = E{x˜|z˜} = xp˜ x˜|z˜(˜x, z˜)dx˜ (17) −∞ where px˜|z˜ is the conditional pdf of the I/Q componentx ñ givenz ñ. A straight forward derivation (see [7]) leads to a closed form expression of the Bayesian estimator for Paper B 113

Gaussian input signals, 2 2 2 x0 (˜z−x0) (˜z+x0) 1 σx − 2 − 2 − 2 x˜ˆopt = e 2σx e 2σv − e 2σv pz˜(˜z) 2πσv

2 σ2 − z˜ √ √ x 2(σ2+σ2) + ze˜ v x [Q( 2γ2) − Q( 2γ1)] 2 2 3 2π(σv + σx) z˜2 $ % σxσv − 2 2 −γ2 −γ2 e 2(σv +σx) e 2 − e 1 + 2 2 (18) 2π(σv + σx) where

2 2 σx(x0 − z˜)+σv x0 γ1 = (19) 2 2 σxσv 2(σx + σv )

2 2 −σx(x0 +˜z) − σv x0 γ2 = . (20) 2 2 σxσv 2(σx + σv )

The time-domain signal xn is estimated from the clipped and noisy zn and the output from the estimatorx ôpt is then demodulated by the FFT. The log-likelihood ratios that are needed for LDPC decoding should be calculated from the pdf of the estimation error after the FFT (X−Xôpt) and the power of the AWGN, but in this case we assume that the log-likelihood ratios can be approximated by the ratios calculated only from the AWGN. It will be shown in the next section that the performance loss of ignoring the clipping effects when calculating the LLRs is minor, and this validates the approximation.

5 Results and Discussion

The performance of the LDPC coded OFDM system with Cartesian clipping is shown in Figure 2 for an ibo of 4 dB. The rate of the LDPC code is 0.8, the codeword length is 1024 and 64 OFDM subchannels are used. The simulation shows that the performance gains of the two investigated receiver clipping mitigation methods are negligible. The best performance is obtained for the Bayesian estimation, but the improvement is only around 0.1 dB at 10−6 bit-error-rate and it seems like the LDPC decoder is not very sensitive to having the exact log-likelihood ratios. The signal-to-noise ratio (SNR) used in this paper is the ratio of transmitted signal power (after clipping) per message bit to the power of the AWGN, that is, 2 1 σy SNR = (21) 2 r σv 114 Paper B

10−1 Clipping ignored Exact noise dist. 10−2 Bayesian est. No clipping

10−3 BER 10−4

10−5

10−6 1 2 3 4 5 6 7 SNR (dB)

Figure 2: Performance of LDPC coded OFDM with Cartesian clipping (ibo =4dB). where r is the rate of the LDPC code and the power of the transmitted clipped signal is 2 2 σy =2σy˜, with ( [6]) 2 √ √ x 2ibo − ibo σ2 0 − Q ibo − e 2 iboQ ibo . y˜ = ibo 1 2 ( ) π +2 ( ) (22)

Figure 3 shows the pdf of the clipping noise, the pdf of the AWGN and the pdf of the total noise (the convolution) for an SNR of 6 dB and an ibo of 4 dB. It is seen in the figure that the clipping noise does not affect the total noise much, and the total noise has almost the same pdf as the AWGN. The log-likelihood ratios that are the input to the LDPC decoder are calculated from the total noise pdf and the attenuation according to (16), and in this case where the total noise pdf and the AWGN pdf looks almost the same, there will be a minor improvement in performance when the exact LLRs are used instead of just ignoring the effect of the clipping noise. The attenuation α is 0.89 for this example. The reason why the effect of the attenuation is minimal is that with a scaling, all LLRs are affected in the same manner. In systems without error correction, an improvement by the receiver clipping mitigation methods is observed for high SNR, see [6, 7]. When using LDPC codes, these high SNRs are never used since even a moderate SNR gives low enough bit-error-rate, at least in a wireless system. It seems like the redundancy added by the LDPC code is effective also in mitigating the clipping distortion. The results suggest that at the receiver side in Paper B 115

3 Clipping noise AWGN 2.5 Clipping noise + AWGN

1.5

Probability density 1

0.5

0 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 Noise value

Figure 3: Pdf of clipping noise, AWGN and the sum of the clipping noise and the AWGN for an ibo of 4 dB and an SNR of 6 dB. a system with LDPC coding, the clipping distortion can be assumed to be more AWGN with just a negligible performance loss.

6 Conclusions

Unexpectedly, it is shown that for bit-error-rates of down to 10−6 that are reasonable in a wireless system, the clipping mitigation methods using extensive calculations and a priori information give little improvement on the performance of an LDPC coded OFDM system, compared to just ignoring the clipping eﬀects. These results can be explained in part by noticing that the changes in the initial log-likelihood ratios due to the clipping mitigation methods are small. Our results should imply directing attention to transmit- oriented clipping mitigation strategies.

References

[1] B. S. Krongold and D. L. Jones, “PAR reduction in OFDM via active constellation extension,” IEEE Transactions on Broadcasting, pp. 258–268, Sep. 2003. 116 Paper B

[2] M. Breiling, S. H. M¨uller-Weinfurtner, and J. B. Huber, “Peak-power reduction in OFDM without explicit side information,” 5th International OFDM-Workshop 2000, Hamburg, Germany, Sep. 2000.

[3] M. Park, H. Jun, and J. Cho, “PAPR reduction in OFDM transmission using Hadamard transform,” ICC 2000 - IEEE International Conference on Communi- cations, pp. 430–433, June 2000.

[4] K. R. Panta and J. Armstrong, “Eﬀects of clipping on the error performance of OFDM in frequency selective fading channels,” IEEE Transactions on Wireless Communications, pp. 668–671, March 2004.

[5] H. Ochiai and H. Imai, “Performance analysis of deliberately clipped OFDM signals,” IEEE Transactions on Communications, vol. 50, pp. 89–101, Jan 2002.

[6] H. Nikopour and S. H. Jamali, “On the performance of OFDM systems over a cartesian clipping channel - a theoretical approach,” IEEE Transactions on Wireless Communications, pp. 2083–2096, Nov. 2004.

[7] P. Banelli, G. Leus, and G. B. Giannakis, “Bayesian estimation of clipped Gaussian processes with application to OFDM,” Proceedings of the European Signal Process- ing Conference (EUSIPCO), Sep. 2002.

[8] R. G. Gallager, “Low density parity check codes,” IRE Transactions on Information Theory, pp. 21–28, Jan. 1962.

[9] M. Tanner, “A recursive approach to low complexity codes,” IEEE Transactions on Information Theory, pp. 533–547, Sep. 1981.

[10] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum- product algorithm,” IEEE Transactions on Information Theory, pp. 498–519, Feb. 2001.

[11] D. J. C. MacKay, “Relationships between sparse graph codes,” Proc. of IBIS 2000, Japan, 2000. Available online at http://www.inference.phy.cam.ac.uk/mackay/ abstracts/ibis.html.

[12] B. Vasic and O. Milenkovic, “Combinatorial constructions of low-density parity- check codes for iterative decoding,” IEEE Transactions on Information Theory, pp. 1156–1176, June 2004. Paper B 117

[13] E. Eleftheriou and S. Ol¸¨ cer, “Low-density parity-check codes for digital subscriber lines,” Proceedings, IEEE International Conference on Communications, pp. 1752– 1757, Apr. 2002.

[14] A. Dholakia and S. Ol¸¨ cer, “Rate-compatible low-density parity-check codes for digital subscriber lines,” Proc., IEEE International Conference on Communications, pp. 415–419, Jun. 2004.

[15] A. Papoulis, Probability, random variables, and stochastic processes. McGraw-Hill, 3rd ed., 1991.

[16] J. Abate and W. Whitt, “Fourier-series method for inverting transforms of probability distributions,” Queuing Systems, vol. 10, pp. 5–88, 1992. 118 Paper C Design of Unequal Error Protection LDPC Codes for Higher Order Constellations

Authors: Neele von Deetzen and Sara Sandberg

Reformatted version of paper originally published in: Proceedings of the IEEE International Conference on Communications, 2007, Scotland, pp. 926-931.

c 2007, IEEE. Reprinted with permission.

119 120 DESIGN OF UNEQUAL ERROR PROTECTION LDPC CODES FOR HIGHER ORDER CONSTELLATIONS

Neele von Deetzen and Sara Sandberg

Abstract

We present an optimization method for unequal error protection (UEP)-LDPC codes with higher order constellations. By modifying the density evolution algorithm under the Gaussian approximation, we propose a flexible code design algorithm for a variable number of protection classes and arbitrary modulation schemes with Gray mapping. Our results show that appropriate code design for higher order constellations reduces the overall bit-error rate. Furthermore, the influence on the UEP capability of the code, that is, the difference in bit-error rate between the protection classes, is investigated.

1 Introduction

Coded modulation is a well-known technique which optimizes the coding scheme given the modulation in order to improve the performance of transmission systems [1–4]. Usually, the modulation alphabet is successively partitioned into smaller subsets, where each partitioning level is assigned a label. These labels are protected by separate channel codes with certain protection capabilities. The codes have to be designed carefully depending on the modulation scheme and its partitioning or labeling strategy. According to [5], the optimal way of designing the codes is to match the different code rates to the capacities of the partitioning steps. This means that, for a given signal-to-noise ratio (SNR) and given modulation scheme and partitioning, the code rates of the single codes are fixed. However, there are also other design approaches with similar results, [6, 7]. The corresponding channel codes can be block codes, convolutional codes, or concatenated codes. In our approach, we use low-density parity-check (LDPC) codes, which were presented by Gallager in [8]. LDPC codes are block codes with a sparse parity-check matrix H that can be conveniently described through a graph commonly called a Tanner graph [9]. Such a graphical representation facilitates a decoding algorithm known as the message-passing 122 Paper C algorithm. For more details on message-passing decoding, the reader is referred to an introduction by Kschischang et al. [10]. Optimization of LDPC codes as separate codes for each level in multilevel coding has been investigated in [11] amongst others. In this paper, only one code is used for all levels instead of separate codes. A longer code (with better performance) can then be used while keeping the delay fixed. The task is to design certain local properties of the code to match the higher order constellations and assign bit positions of the modulation scheme to the codeword bits. Achieving local properties in the codeword may be done by designing the variable and/or check node degree distribution of the code in an irregular way [12–15]. The connection degree of the variable nodes affects the bit-error rate (BER). The message bits are divided into several classes depending on the connection degree and each class has different BER after decoding, that is, the code provides unequal error protection (UEP). In [16], bits from the most protected modulation level are mapped to variable nodes with the best protection, that is, the variable nodes with highest connection degrees. No other optimization is performed. In [7], the different amount of protection for each modulation level is taken into account in the initialization of the density evolution algorithm that is employed to optimize the degree distribution of the code. In our approach, we design UEP-LDPC codes while accounting for the unequal error protection which is already inherent in modulation. Regarding the modulation schemes, any conventional scheme like M-QAM or M-PSK as well as more complex schemes called hierarchical constellations [17] with Gray labeling may be used. The paper is organized as follows. Section 2 presents the overall system model, while models for the modulator are given in Section 3. Section 4 contains the main part of this paper which includes a general description of irregular LDPC codes and the standard code optimization as well as extensions for UEP. We also explain the optimization of the degree distribution for higher order constellations and give an algorithm for the code design. In Section 5, some simulation results are discussed.

2SystemModel

In this section, we describe the system model of the transmission scheme. Usually, in multilevel coding, the information bits are demultiplexed into lm parallel streams, where 2lm = M is the constellation size of the modulation scheme. The diﬀerent bit streams are encoded separately and are assigned to the lm partitioning steps of a modulation scheme. In our case, each independent and identically distributed (i.i.d.) information bit is assigned to one of Nc − 1 protection classes, which are usually deﬁned by the source Paper C 123

C1 M1

ui ∈ Ck UEP− s

− REMUX k =1,...,Nc 1 LDPC CNc−1 MNs

CNc

Figure 1: UEP-LDPC coded modulation scheme. coding unit and do not have to be of equal size. We apply only one code C, providing

Nc protection classes at its output (see Figure 1), where all parity bits correspond to the least protected class CNc . The bits of the protection classes are remultiplexed and assigned to certain bit positions of the modulator, which correspond to modulation classes

M1, ..., MNs . The bit assignment will be described in Section 4.1. In the following we assume an additive white Gaussian noise (AWGN) channel with noise variance σ2.

3 Modulation

Let us assume a modulation scheme with M =2lm signal points, labeled by binary vectors d =(dlm−1, ...d1,d0). In order to design codes for higher order constellations, we investigate the error probabilities of the individual bits. The example of 8-PSK is chosen here, but the scheme can also be designed for any other constellation. Using the union bound, the approximate symbol-error rate expression for 8-PSK is given as [18], $ % π Ps, −PSK =erfc 3 · Eb/N sin , (1) 8 0 8 where erfc(·) is the complementary error function. The individual and average bit-error probabilities depend on the partitioning and labeling strategy. We will only consider Gray labeling, since it leads to the lowest overall bit-error probability. Furthermore, the bits are almost independent of each other which is important for the LDPC decoder performance. For Gray labeling, a symbol error typically results in only one bit error and, thus, ˜ ≈ one can assume that the average bit-error rate is Pb Ps/ log2(M). The expressions for 124 Paper C the bit-error probabilities of the individual bits in the symbol are $ % 1 π Pb,d ≈ · erfc 3 · Eb/N sin , (2) 0 2 0 8 $ % 1 π Pb,d = Pb,d ≈ · erfc 3 · Eb/N sin . (3) 1 2 4 0 8

From these different bit-error probabilities, one can determine equivalent noise variances of the single bit positions corresponding to the case of BPSK. We define the noise vector 2 2 2 σ =[σ1 ... σNs ] to be a vector that contains the equivalent noise variances for each separate bit-error rate ordered with the lowest variance first. We assume that there are

Ns distinct equivalent noise variances, where Ns ≤ lm. The equivalent noise variances may be calculated from the individual bit-error rates by

2 1 σj = & ' . (4) −1 2 2 erfc (2Pb,dj )

Note that these expressions are obtained by applying the union bound. The approximations are assumed to be appropriate for our purposes but can be replaced by more exact formulas. In the following, we assume that Ns equivalent BPSK channels are employed instead of the higher order constellation channel. We claim that this approximation meets our requirements since the system employs Gray mapping.

4UEP-LDPCCodes

As a channel code, we choose a UEP-LDPC code. There are different methods for achieving UEP with LDPC codes, the probably most obvious one is puncturing a certain amount of the code bits before modulation. The receiver does not have any knowledge about these bits and assumes all signals of the input alphabet with equal probability. Two other possibilities for obtaining UEP were presented in [14] and [15]. Both approaches use irregular LDPC codes and optimize the irregularities of the code in order to obtain several classes of protection within the codeword. More precisely, the authors in [14] optimize the irregular variable node (also called bit node) degree distribution while keeping the check node degree distribution fixed, whereas in [15], the check node degree distribution is adapted, keeping the variable node degree distribution fixed. We will follow the approach from [14]. The next section gives a general description of UEP-LDPC codes by considering degree distributions. Paper C 125

4.1 General Description

LDPC codes are block codes with a sparse parity-check matrix H of dimension (n−k)×n, where R = k/n denotes the code rate and k and n are the lengths of the information word and the codeword. The codes can be represented as a bipartite graph, called Tan- ner graph. The graph consists of two types of nodes, variable nodes and check nodes, which correspond to the bits of the codeword and to the parity-check constraints, respectively. A variable node is connected to a check node if the bit is included in the parity-check constraint. For regular LDPC codes, all variable nodes and check nodes have ﬁxed variable node degree and check node degree, respectively. However, irregular LDPC codes are known to approach capacity closer than regular LDPC codes. The irregular variable node and check node degree distributions may be deﬁned by the polynomials

dvmax dcmax i−1 i−1 λ(x)= λix and ρ(x)= ρix i=2 i=2

where dvmax and dcmax are the maximum variable and check node degree [19]. The degree distributions describe the proportion of edges connected to nodes with a certain degree. In order to optimize the degree distribution of an irregular LDPC code, the decoding behavior has to be investigated. Using a message-passing algorithm, the messages along the edges of the graph are updated iteratively. The mutual information messages at the input of a variable node and a check node at iteration l can be computed by means of density evolution using the Gaussian approximation [20] to be

dcmax (l−1) −1 (l−1) xu =1− ρjJ((j − 1)J (1 − xv )) , (5) j=2

dvmax (l) 2 −1 (l−1) x = λiJ( +(i − 1)J (x )) , (6) v σ2 u i=2 with J(·) computing the mutual information x = J(m)by

−z J(m)=1− E{log2(1 + e )} (7) 2 1 −z − (z−m) 4m =1− √ log2(1 + e ) · e dz 4πm R for a consistent Gaussian random variable z ∼N(m, 2m). These update rules are valid only when all bits belong to one modulation class with noise variance σ2. 126 Paper C

For the case of UEP-LDPC codes, we follow the approach from [14] and deﬁne an overall check node degree distribution and diﬀerent variable node degree distributions for the Nc protection classes, i.e.,

dvmax Ck Ck i−1 λ (x)= λi x for k =1...Nc . (8) i=2 Since the variable node degree distributions give proportions of edges connected to variable nodes of certain degrees, the constraint

Nc dvmax Ck λi =1 (9) k=1 i=2 must be fulfilled. Different variable node degree distributions lead to a modified update rule for the messages from variable nodes to check nodes

N dv c max 2 x(l) = λCk J( +(i − 1)J −1(x(l−1))) . (10) v i σ2 u k=1 i=2 The update rule for the messages from check nodes to variable nodes stays the same since the check node degree distribution is constant. This paper considers the design of UEP-LDPC codes for higher order constellations, where the individual bits in the symbol may have different error probabilities. The aim of the code design is to reduce the overall BER by taking these different error probabilities into account. The design algorithm should also give the possibility to trade overall BER for UEP capability. The natural way of assigning bits from modulation classes to protection classes to achieve UEP, is to use the best protected bits from the modulation, that is, modulation class M1, for protection class C1 and continue like that until all bits have been assigned to a protection class. However, this assignment is not necessarily expected to give a degree distribution with the lowest possible threshold, where the threshold is defined as the lowest Eb/N0 for which density evolution converges. As is discussed later on, there is always a tradeoff between a low threshold and good UEP capability. By introducing different variable node degree distributions also for each modulation class, linear programming may be used to assign bits from the modulation classes to the protection classes.

4.2 Notations

We consider a UEP-LDPC code with Nc protection classes. The proportions of each class are given by the normalized lengths of each class corresponding to the information bits, Paper C 127

α =[α1, ..., αNc−1]. The proportion distribution of the bits in the codeword belonging − to the protection classes is given by p =[α1R, ..., αNc−1R, (1 R)]. Ns is the number of different bit-error rates for the bits in a symbol and we will describe the bits with a distinct bit-error rate as belonging to one modulation class. β =[β1, ...,βNs ] defines the proportion of bits that belongs to each modulation class. The vector λ contains the overall variable node degree distribution, both for dif- Ck ferent protection classes and different modulation classes. Let λMj ,i be the proportion of edges connected to variable nodes of degree i that belongs to modulation class Ck Ck Ck T Mj andprotectionclassCk. We also define λ =[λ ,...,λ ] and λ = Mj Mj ,2 Mj ,dvmax ! "T T C T T C T λC1 ,...,λ Nc , ..., λC1 ,...,λ Nc , where (·)T denotes the transpose. λCk is a M1 M1 MNs MNs Mj

(dvmax − 1 × 1) vector and λ is a vector of size ((dvmax − 1) · Nc · Ns × 1). The vector T ρ =[ρ2,...,ρdcmax ] describes the check node degree distribution. For later purposes, we T T also deﬁne 1/dv =[1/2, 1/3, ..., 1/dvmax ] ,1/dc =[1/2, 1/3, ..., 1/dcmax ] and 1 to be an all-ones vector of appropriate length.

4.3 Optimization of the Degree Distribution for Higher Order Constellations

For higher order constellations (HOC), the update rule (10) has to be modified to take different noise variances for different variable nodes into account. Similar to [7], the update rule may be written

N N dv c s max 2 (l) Ck − −1 (l−1) xv = λMj ,iJ( 2 +(i 1)J (xu )) . (11) σj k=1 j=1 i=2

Equations (5) and (11) can now be combined to yield the mutual information evolution of the LDPC code (l) 2 (l−1) xv = F (λ,ρ,σ ,xv ) . (12)

(l) (l−1) (l−1) If xv >xv for any xv ,thenλ and ρ describe a code for which density evolution converges for the noise variance vector σ2. UEP capability may be obtained by optimizing each protection class after another by linear programming, starting with the best protected class and ﬁxing the degree distributions of the already optimized classes during the optimization of the following classes, [14]. It is well-known that a higher connectivity of a variable node leads to better protection. Thus, the optimization target is to ﬁnd a variable node degree distribution for the whole code that maximizes the average variable node degree of the class being 128 Paper C

optimized. The target function for protection class Ck canbeformulatedas

Ns dvmax Ck max λM ,i . (13) λ j j=1 i=2

This target function results in a degree distribution with UEP capability, but the only requirement on the assignment of the code bits to the modulation classes is that density evolution must converge for the given degree distribution. In order to achieve UEP, one would assign as many bits as possible from better modulation classes to the protection class being optimized. This can be done by introducing a scaling factor kj for the modulation classes, where the only requirement is k1 >k2 >...> kNs > 0. For simplicity, kj might be chosen as kj = Ns − j +1. Thefactor kj will appear later in the target function (16) of the algorithm where it has the effect that the linear programming algorithm, if possible while fulfilling all constraints, will use modulation classes with low noise variance for the best protected classes. When designing good LDPC code ensembles, the stability condition which ensures convergence of the density evolution for mutual information close to one should be fulfilled [19]. The stability condition gives an upper bound on the number of degree-2 variable nodes. For a BPSK scheme, where all bits are affected by the same noise variance, we have [19] 1 −r − x − 1 2 2σ2 >e = P0(x)e dx = e (14) λ (0)ρ (1) R with P0(x) being the message density corresponding to the received values and λ (x)and ρ(x) being the derivatives of the degree polynomials. It is straightforward to see that C Ns Nc ( k) dcmax · − λ (0) = j=1 k=1 λj,2 and ρ (1) = m=2 ρm (m 1). In our case, the bits are affected 2 by channel noise with different variances σj (see (4)) and, thus, different densities. We use the average density, which is given by utilizing the modulation class proportions β,

N N s s − 1 −r − x 2σ2 2 j e = βj · P0,j(x)e dx = βj · e . (15) R j=1 j=1

We are very well aware that this is an approximation but assume appropriateness for the ensemble of code constructions with given β.

4.4 Optimization Algorithm

The optimization algorithm proposed here is a modiﬁcation of the hierarchical optimization algorithm presented in [14] for higher order constellations. The optimization is Paper C 129

performed at Eb/N0 = δ + (this will be the threshold of the optimized code), where δ is the lowest possible threshold in dB for the given ρ and dvmax ,and is the offset from the lowest threshold that gives freedom in the choice of λ to enable design of a UEP code. The algorithm can be divided into two parts, global optimization and local optimization. In the global optimization, the local optimization is executed class after class for agivenEb/N0. In the local optimization, λ is optimized to maximize the scaled average variable node degree of class Ck while using the best possible modulation class, assuming C C1 k−1 ∀ that classes C1, ..., Ck−1 have already been optimized and that λMj , ..., λMj , j, are fixed. In order to find a maximum average degree of the class being optimized, the algorithm starts by setting the minimum variable node degree of this class to some maximum value, conveniently the maximum variable node degree of the code, and tries to find a solution. In case of failure, the minimum variable node degree is successively reduced until the algorithm succeeds in finding an overall degree distribution which fulfills the constraints. The global optimization can be stated as follows.

2 1) Fix Eb/N0 = δ + and calculate σ .

2) Find λ by performing local optimization for each protection class.

For the local optimization of class Ck, a linear programming routine is executed, which requires deﬁnition of the check node degree distribution ρ, Eb/N0 = δ + in dB, and the maximum variable node degree dvmax .

(k) 1) Initialization dvmin = dvmax

2) While optimization failure

a) Optimize Ns dvmax Ck max kj λ (16) λ Mj ,i j=1 i=2

under the constraints [C1] − [C6].

[C1] Rate constraint

Ns Nc T Ck 1 T λ 1/dv = ρ 1/dc (17) Mj 1 − R j=1 k=1

[C2] Proportion distribution constraints 130 Paper C

i) Ns Nc T λCk Mj 1 = 1 (18) j=1 k=1

ii) ∀k ∈{1,...,Nc − 1},

Ns T R Ck T λ 1/dv = αk ρ 1/dc (19) Mj 1 − R j=1

iii) ∀j ∈{1,...,Ns − 1},

Nc T Ck 1 T λ 1/dv = βj ρ 1/dc (20) Mj 1 − R k=1

[C3] Convergence constraints, see (12)

F (λ,ρ,σ2,x) >x (21)

[C4] Stability condition, see (14) and (15) ⎡ ⎤ −1 Ns Nc Ns dcmax 2 Ck −1/2σ λ < ⎣ βje j · ρm m − ⎦ Mj ,2 ( 1) (22) j=1 k=1 j=1 m=2

[C5] Minimum variable node degree constraint

∀i

[C6] Previous optimization constraints

C ∀k

(k) (k) b) dvmin = dvmin − 1

End

4.5 Code Construction

When the optimal degree distribution of the variable nodes is found, a parity-check matrix is constructed by the Approximate Cycle Extrinsic message degree (ACE) algorithm [21]. The ACE algorithm constructs a parity-check matrix following a given variable node degree distribution while selectively avoiding small cycle clusters that are isolated from the rest of the graph. The ACE algorithm has good performance in the error-ﬂoor region forirregularLDPCcodes. Paper C 131

5 Simulation Results

In this section, simulation results for an example with 8-PSK are presented. We denote our scheme by higher order constellation UEP (”HOC-UEP”), which is a UEP-LDPC 2 2 code optimized for the diﬀerent σj from the modulation. The noise vector σ is calculated accordingto(4),withNs =2andβ =[2/3, 1/3] for Gray-labeled 8-PSK. The HOC- UEP scheme is compared to a UEP-LDPC code optimized for BPSK [14], but used for 8-PSK. This scheme, that is denoted by “UEP”, designs the code for an average σ2 and assigns the bits following the natural bit assignment. The variable node degree distributions are optimized for R =1/2, Nc =3,α =[0.3, 0.7], dvmax =30and ρ(x)=0.00749x7 +0.99101x8 +0.00150x9, which is found by numerical optimization in [19] to be a good check node degree distribution for dvmax = 30. Table 1 shows the degree distributions given by the two design algorithms. For the UEP scheme, we arbitrarily choose =0.1dBtoallowforsomeunequalerror protection. The resulting degree distributions λCk are given for each protection class

Ck. The minimum threshold δ of the HOC-UEP code is 0.27 dB lower than of the corresponding UEP code. Thus, we design the HOC-UEP code for =0.37 dB in order to have the same thresholds for both schemes. The degree distributions of the HOC-UEP Ck scheme, λMj , are also given in Table 1. For comparison, the degree distributions for both algorithms are also shown for the minimum thresholds, that is, =0dB. Finite length codeword simulations with n = 4096 and 50 decoding iterations are performed using the equivalent BPSK channels. Simulations verify that 8-PSK modulation and demodulation give almost exactly the same results as simulations with the equivalent BPSK channels. We assume that a soft demapper provides the message passing decoder with the channel log-likelihood ratios (LLRs) in any case using higher order constellation modulation and demodulation. Note that the channel LLRs are computed using the 2 appropriate noise variances σj of the modulation classes. Figure 2 shows the overall BER after 50 decoding iterations. By design, the overall BERs for the codes with = 0 dB are higher than for the corresponding codes with = 0 dB. This is because the thresholds of the codes are increased in order to allow an increased average variable node degree of the most protected classes. Figure 2 also shows that for high Eb/N0, the overall BERs of the HOC-UEP codes are lower than for the UEP codes. The overall BER of the HOC-UEP =0.37 dB code is lower than the overall BER of the UEP =0.1 dB code, even though they are designed for the same threshold. For an overall BER of 10−5, there is a gain of around 0.7 dB by the HOC-UEP scheme. 132 Paper C

Table 1: Degree distributions for the UEP and HOC-UEP schemes.

C1 C2 C3 =0dB

UEP λ7 =0.0799 λ3 =0.1790 λ2 =0.2103

λ8 =0.0948 λ6 =0.0737 λ3 =0.0181

λ30 =0.3029 λ7 =0.0414

HOC-UEP M1 λ9 =0.1703 λ3 =0.1673 λ2 =0.1240

λ10 =0.0555

λ30 =0.1811

HOC-UEP M2 λ30 =0.0854 λ4 =0.0225 λ2 =0.0878

λ5 =0.0738 λ3 =0.0022

λ7 =0.0117 λ4 =0.0183 =0.1dB

UEP λ11 =0.1783 λ3 =0.2041 λ2 =0.1841

λ12 =0.1184 λ4 =0.0393 λ3 =0.0575

λ30 =0.2183 =0.37 dB

HOC-UEP M1 λ16 =0.5255 λ3 =0.0187 λ2 =0.2174

λ17 =0.0088

HOC-UEP M2 λ3 =0.1929 λ3 =0.0075

λ4 =0.0293

The BER performances of the individual protection classes C1 and C2 for the UEP scheme are shown in Figure 3. The UEP capability, that is, the diﬀerence in BER between class C1 and C2, is increased with increasing .For = 0 dB, the UEP capability is accomplished by assignment of high degree variable nodes to the most protected classes.

Figure 4 shows the BER performance of protection classes C1 and C2 for the HOC- UEP scheme. The results show that the HOC-UEP =0.37 dB code has more UEP capability than the HOC-UEP = 0 dB code. A comparison of the UEP capability for the UEP scheme and the HOC-UEP scheme suggests that a high is needed for the HOC-UEP scheme in order to achieve UEP. However, a high does not seem to aﬀect the overall BER of the HOC-UEP scheme much for high Eb/N0, see Figure 2. Comparing the individual protection classes of the HOC-UEP =0.37 dB and the −5 UEP =0.1dBschemeatBER10 , we gain 0.1 dB for protection class C1 and Paper C 133

0 10 UEP ε = 0 dB UEP ε = 0.1 dB −1 10 HOC−UEP ε = 0 dB HOC−UEP ε = 0.37 dB

−2 10

−3 10

−4 10 Overall BER

−5 10

−6 10

−7 10 1 1.5 2 2.5 3 3.5 E /N (dB) b 0

Figure 2: Overall bit-error rate performance.

0 10 UEP C1 ε = 0 dB UEP C2 ε = 0 dB −1 10 UEP C1 ε = 0.1 dB UEP C2 ε = 0.1 dB

−2 10

−3 10 BER −4 10

−5 10

−6 10

−7 10 1 1.5 2 2.5 3 3.5 E /N (dB) b 0

Figure 3: Bit-error rate performance of protection class C1 and C2 for the UEP scheme. 134 Paper C

0 10 HOC−UEP C1 ε = 0 dB HOC−UEP C2 ε = 0 dB −1 10 HOC−UEP C1 ε = 0.37 dB HOC−UEP C2 ε = 0.37 dB

−2 10

−3 10 BER −4 10

−5 10

−6 10

−7 10 1 1.5 2 2.5 3 3.5 E /N (dB) b 0

Figure 4: Bit-error rate performance of protection class C1 and C2 for the HOC-UEP scheme.

(expected) 0.7 dB for class C2. These gains are expected to be even higher for lower BERs.

6 Conclusions

In this paper, we present a ﬂexible design method for UEP-LDPC codes with higher order constellations which is applicable to arbitrary signal constellations and arbitrary number and proportions of the protection classes. For an example with 8-PSK, it is shown that the overall BER is reduced by the proposed method and there is a gain of 0.7dBatBER 10−5. The results for the individual protection classes show only slightly reduced UEP capability for the new design method, but lower bit-error rates for all protection classes corresponding to information bits.

References

[1] G. Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Transactions on Information Theory, vol. 28, pp. 55–67, Jan. 1982. Paper C 135

[2] H. Imai and S. Hirakawa, “A new multilevel coding method using error correcting codes,” IEEE Transactions on Information Theory, vol. 23, pp. 371–377, May 1977.

[3] G. Ungerboeck, “Trellis-coded modulation with redundant signal sets Part I: Intro- duction,” IEEE Communications Magazine, vol. 25, pp. 5–11, Feb. 1987.

[4] G. Ungerboeck, “Trellis-coded modulation with redundant signal sets Part II: State of the art,” IEEE Communications Magazine, vol. 25, pp. 12–21, Feb. 1987.

[5] U. Wachsmann, R. Fischer, and J. Huber, “Multilevel codes: Theoretical concepts and practical design rules,” IEEE Transactions on Information Theory, vol. 45, pp. 1361 – 1391, July 1999.

[6] G. Caire, G. Taricco, and E. Biglieri, “Bit-interleaved coded modulation,” IEEE Transactions on Information Theory, vol. 44, pp. 927–946, May 1998.

[7] H. Sankar, N. Sindhushayana, and K. Narayanan, “Design of low-density parity- check (LDPC) codes for high order constellations,” in Proc. Globecom 2004, pp. 3113–3117, Nov. 2004.

[8] R. Gallager, “Low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 8, pp. 21 – 28, Jan. 1962.

[9] M. Tanner, “A recursive approach to low complexity codes,” IEEE Transactions on Information Theory, pp. 533–547, Sep. 1981.

[10] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum- product algorithm,” IEEE Transactions on Information Theory, pp. 498–519, Feb. 2001.

[11] J. Hou, P. Siegel, L. Milstein, and H. Pﬁster, “Capacity-approaching bandwidth- eﬃcient coded modulation schemes based on low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 49, pp. 2141 – 2155, Sept. 2003.

[12] K. Kasai, T. Shibuya, and K. Sakaniwa, “Detailed representation of irregular LDPC code ensembles and density evolution,” in Proc. ISIT 2003, p. 121, June 2003.

[13] C. Poulliat, D. Declercq, and I. Fijalkow, “Optimization of LDPC codes for UEP channels,” in Proc. ISIT 2004, p. 451, June 2004. 136

[14] C. Poulliat, I. Fijalkow, and D. Declercq, “Scalable image transmission using UEP optimized LDPC codes,” in Proc. ISIVC 2004, July 2004. http://publi- etis.ensea.fr/2004/PFD04a.

[15] L. Sassatelli, W. Henkel, and D. Declercq, “Check-irregular LDPC codes for unequal error protection under iterative decoding,” in 4th International Symposium on Turbo-codes and related topics (D. D. L. Sassatelli, W. Henkel, ed.), 2006.

[16] Y. Li and W. Ryan, “Bit-reliability mapping in LDPC-coded modulation systems,” IEEE Commununications Letters, vol. 9, pp. 1–3, Jan. 2005.

[17] K. Fazel and M. Ruf, “Combined multilevel coding and multiresolution modulation,” in Proc. ICC 1993, pp. 1081–1085, May 1993.

[18] J. G. Proakis, Digital Communications. New York: McGraw-Hill, 4th ed., 2001.

[19] T. Richardson, M. Shokrollahi, and R. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 47, pp. 619–637, Feb. 2001.

[20] S.-Y. Chung, T. Richardson, and R. Urbanke, “Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation,” IEEE Transactions on Information Theory, vol. 47, pp. 657–670, Feb. 2001.

[21] T. Tian, C. Jones, D. Villasenor, and R. Wesel, “Selective avoidance of cycles in irregular LDPC code construction,” IEEE Transactions on Communications, vol. 52, pp. 1242–1247, Aug. 2004. Paper D Design of Bandwidth-Eﬃcient Unequal Error Protection LDPC Codes

Authors: Sara Sandberg and Neele von Deetzen

Reformatted version of paper submitted to: IEEE Transactions on Communications, accepted for publication Jan. 29, 2009.

c 2008, IEEE. Reprinted with permission.

137 138 Paper D DESIGN OF BANDWIDTH-EFFICIENT UNEQUAL ERROR PROTECTION LDPC CODES

Sara Sandberg and Neele von Deetzen

Abstract

This paper presents a strategy for the design of bandwidth-efficient LDPC codes with unequal error protection. Bandwidth efficiency is obtained by appropriately designing the codes for higher order constellations, assuming an AWGN channel. The irregularities of the LDPC code are designed, using the Gaussian approximation of the density evolution, to enhance the unequal error protection property of the code as well as account for the different bit error probabilities given by the higher order constellation. The proposed code design algorithm is flexible in terms of the number and proportions of protection classes. It also allows arbitrary modulation schemes. Our method combines the design of unequal error protection LDPC codes for the binary input AWGN channel with the code design for higher order constellations by dividing the variable node degree distribution into sub-degree distributions for each protection class and each level of protection from the modulation. The results show that appropriate code design for higher order constellations reduces the overall bit-error rate significantly. Furthermore, the unequal error protection capability of the code is increased, especially for high SNR.

1 Introduction

Multimedia applications require large amounts of data to be transmitted through networks and wireless transmission systems with reasonable delay and error performance. Therefore, resources like power, time, or bandwidth have to be used economically. Mul- timedia data usually have heterogeneous sensitivity against transmission errors. They often contain a header for data management in higher layers, some essential payload as well as additional payload used for enhanced quality. Hence, the transmission system should provide unequal error protection (UEP) in order to account for the different 140 Paper D properties. Suitable UEP may increase the perceived performance for applications where different bits have different sensitivity to errors, such as transmission of progressively encoded images or a frame with header information. For packet-based transmissions with no possibility of retransmission, an error in the header is critical and may lead to rejection of the packet or even a crash of the source decoder, while errors in the payload are often tolerable. In order to allow for bandwidth-efficient transmission, it is desirable to use higher order modulation techniques, such as M-QAM, M-PSK, M>2, or more advanced constellations. Modulation with higher order constellations (HOCs) may imply that different bits in the symbol have different error probabilities, i.e., the modulation already provides some UEP. Coded modulation is a well-known strategy to optimize the coding scheme given the modulation to improve the performance of transmission systems in terms of overall bit-error rate (BER), [1–4]. In multilevel coding [5], the modulation alphabet is successively partitioned into smaller subsets, where each partitioning level is assigned a label. These labels are protected by separate channel codes with certain protection capabilities. In such a strategy, the codes have to be designed carefully depending on the modulation scheme and its partitioning or labeling strategy. However, applying a shorter code on each level may have drawbacks (e.g. higher BER) compared to designing one long code whose output bits are appropriately assigned to the levels. Therefore, we employ the latter, taking advantage of the longer code. This paper focuses on low-density parity-check (LDPC) codes, originally presented by Gallager in [6]. They exhibit a performance very close to the capacity for the binary- input additive white Gaussian noise (BI-AWGN) channel, [7]. The close-to-optimal performance of LDPC codes for the BI-AWGN channel suggests the use of LDPC codes also for other channels. The aim of this paper is to design LDPC codes for transmissions using HOCs in applications where the source bits have different sensitivities to errors and UEP is desired. As far as we know, such a code design has not been proposed before. However, a code design using turbo codes for coded modulation with UEP has recently been suggested, [8]. LDPC codes are block codes with a sparse parity-check matrix H of dimension (n − k) × n,whereR = k/n denotes the code rate and k and n are the lengths of the information word and the codeword. The codes can be represented by a bipartite graph, called a Tanner graph [9], which facilitates a decoding algorithm known as the message-passing algorithm [10]. The graph consists of two types of nodes, variable nodes and check nodes, which correspond to the bits of the codeword and to the parity-check Paper D 141 constraints, respectively. A variable node is connected to a check node if the bit is included in the parity-check constraint. For regular LDPC codes, all variable nodes have the same degree and all check nodes have another common degree. However, irregular LDPC codes are known to approach capacity more closely than regular LDPC codes. The irregular variable node and check node degree distributions may be defined by the dvmax i−1 dcmax i−1 polynomials [7] λ(x)= i=2 λix and ρ(x)= i=2 ρix respectively, where dvmax is the maximum variable node degree and dcmax is the maximum check node degree of the code. The coefficients of the degree distributions describe the proportion of edges connected to nodes with a certain degree. In order to optimize the degree distribution of an irregular LDPC code, the decoding behavior has to be investigated. Using a message- passing algorithm, the messages along the edges of the graph are updated iteratively. The messages at the input of a variable node and a check node at each iteration represent mutual information and can be computed by means of density evolution using a Gaussian approximation [11] and thereby, the decoding behavior of a code ensemble with a specific degree distribution may be predicted. Density evolution is an asymptotic tool, but is commonly used to design codes of finite length. LDPC codes have been designed for larger constellation sizes in previous works. In [12], separate codes were designed for each level in a multilevel coding scheme. On the other hand, bit-interleaved coded modulation (BICM), [13,14] employs only one code for all levels of the modulation. A design method for discrete multitone modulation (DMT) is proposed in [15]. This method may also be directly applied to HOCs and is very similar to the design approach suggested in [14]. In [14] and [15], the codes are designed to have local properties that match the HOCs and bit positions of the modulation scheme are assigned to the codeword bits. Reliability mappings of bits from the modulation to codeword bits have also been suggested, [16,17]. In many applications, the desired UEP properties are low BER within one or several classes of bits, while the performance of the remaining classes should be comparable to non-UEP codes. In the following we address such codes as codes with good UEP capability. UEP is commonly provided by multilevel coding or adapted code rates, for example by puncturing. These methods have in common that different codes are used for each level of protection. However, for most applications the more important bits are fewer than the less important bits. This implies that even if the coderate used for the more important bits is low, this codeword will usually be short. To avoid the use of short codewords (or a very long delay), LDPC codes that provide UEP within one codeword may be designed instead. Such codes can for example be constructed 142 Paper D by an algebraic method based on the Plotkin construction, [18]. However, since it is widely observed that the connection degree of the variable nodes affects the bit-error rate for a limited number of decoder iterations, it is more typical to design the variable and/or check node degree distribution of the code in an irregular way using density evolution, [19–23]. In this case the codeword bits are divided into several protection classes with different protection depending on the connection degrees of their bits. In [21], the check node degree distribution is adapted, keeping the variable node degree distribution fixed, whereas the authors of [22] optimize the irregular variable node degree distribution while keeping the check node degree distribution fixed. The basic idea in [21–23] is to achieve UEP by dividing the degree distributions into sub-distributions. Such sub-distributions have also been employed for systems without UEP capability to account for HOCs [14, 15, 24]. In [14], the different amount of protection for each modulation level is taken into account in the initialization of the density evolution algorithm that is employed to optimize the sub-degree distributions of the code. Both [15] and [24] designed LDPC codes for a set of parallel subchannels.

In this paper, we propose a UEP-LDPC code design for HOCs that is based on optimizing the variable node degree distribution λ(x). Apart from designing the code to account for the UEP provided by the modulation itself and thereby reducing the overall BER, the aim is to provide a flexible design method that can use the UEP from the modulation to create a code with other UEP properties which are usually specified by the source coding unit. We design UEP-LDPC codes using sub-degree distributions both for the protection classes and for the classes of bits with different protection resulting from the modulation. This allows for a very flexible code design where any conventional modulation scheme, like M-QAM or M-PSK as well as more complex schemes like hierarchical constellations [25], may be used. The separation of the variable node degree distribution into sub-degree distributions significantly increases the number of design parameters. Therefore it is important to note that the code design is solved by iterative linear programming (LP), which enables an efficient optimization of the sub-degree distributions. Our code design is based on the design method for UEP-LDPC codes suggested in [22] that employs iterative LP.

The paper is organized as follows. Section 2 presents the overall system and modulator models. Section 3 contains the main part of this paper which introduces the code optimization of UEP capable codes used with HOCs and provides an algorithm for the code design. In Section 4, some simulation results for 8-PSK and 64-QAM are presented and compared to BPSK-optimized codes. Section 5 concludes the paper. Paper D 143

ui ∈ Ck UEP− s

− REMUX k =1,...,Nc 1 LDPC CNc−1 MNs

CNc

Figure 1: Schematic description of the proposed scheme. The source bits are encoded by a UEP-LDPC code and the coded bits are assigned to modulation classes before modulation.

2SystemModel

2.1 Model Description

Each codeword bit is assigned to a protection class and a modulation class. We apply only one UEP-LDPC code, providing Nc protection classes at its output (see Figure 1).

The independent and identically distributed (i.i.d.) information bits ui are divided into − Nc 1 protection classes C1 ...CNc−1. The least protected class CNc contains the parity bits. Each bit is protected by the UEP-LDPC code according to the protection class it belongs to, where C1 has the best protection. The sizes of the protection classes (they do not have to be of equal size) as well as the allocation of information bits to protection classes are usually deﬁned by the source coding unit. The bits of the protection classes are remultiplexed and assigned to certain bit positions of the modulator, corresponding to modulation classes M1, ..., MNs . A bit that belongs to modulation class Mj will be mapped to the bit position in the symbol that corresponds to that modulation class, i.e., it will be naturally protected according to modulation class Mj. Bits belonging to one modulation class are aﬀected by the same (channel) bit error probability. The bit assignment will be described in Section 3.1. In the following, we assume an AWGN channel with noise variance σ2.

2.2 Modulation

Let us assume a modulation scheme with M =2lm symbols, labeled by binary vectors d =(dlm , ...d2,d1). In order to design codes for HOCs, we investigate the error probabilities of the individual bit positions in the symbol. Depending on the signal constellation and the labeling strategy, the error probabilities of the individual bit positions 144 Paper D may be determined.

One may use traditional constellations like 8-PSK or 64-QAM as well as non-uniform constellations, so-called hierarchical modulation [26] or multiresolution modulation [25]. When UEP is desired, non-uniform constellations have the advantage that they can provide greater diﬀerentiation in error probabilities of the bit positions in the symbol than traditional constellations. Furthermore, the design methods in the above references allow for a controllable amount of UEP through variations of the distances between the signal points.

Generally, one can approximate the symbol-error probability Ps as well as the bit- error probabilities Pb,d1 ...Pb,dlm of a constellation using the union bound. For Gray labeling, the average bit-error probability is often assumed to be equal for all bit positions, ˜ ≈ 1 · i.e., Pb,di Ps. However, it is important to be aware of the different bit-error log2(M) probabilities when designing codes for HOCs. In this work we only consider Gray labeling, since the bit-errors have low statistical dependencies and can thus be assumed to be independent [27]. This is very important for the message-passing decoder. However, for labelings other than Gray, the differences in BER may be more significant. If such labelings are of interest, we suggest the use of the improved decoding algorithm proposed in [28] that combines decoding and demodulation and takes into account the statistical dependencies among bit errors originating in the same symbol.

In [5], it was stated that a symmetric channel with an input alphabet of size 2lm can be represented by lm equivalent binary-input channels. Depending on the labeling, the equivalent channels may not be symmetric. Since the symmetry of the channel is an important property for the density evolution of LDPC code ensembles, we use a method called i.i.d. channel adapters [29]. In order to force symmetric component channels, i.i.d. binary vectors t =[t1 ...tn] are added modulo-2 to the codeword. At the receiver, the log-likelihood ratio Lj is multiplied by −1iftj = 1. Since the equivalent component codes are now symmetric, one can assume their behavior being equal to BPSK signaling. From the known bit-error probabilities of the binary component channels, it is easy to 2 determine their equivalent noise variances σi which are required for density evolution σ2 2 2 later on. We deﬁne the noise vector =[σ1 ... σNs ] to be a vector that contains the equivalent noise variances for each separate bit-error rate, i.e., for each modulation class, ordered with the lowest variance ﬁrst. We assume that there are Ns distinct equivalent noise variances, where Ns ≤ lm. Paper D 145

011

010 001

000

110

100 111 d d 101 3 d2 1

Figure 2: 8-PSK modulation with Gray labeling.

Example: 8-PSK 8-PSK modulation with Gray labeling is shown in Figure 2. Considering only neighboring signal points, the approximate symbol-error rate expression for this modulation is given as [30], $ % π Ps, −PSK =2Q 6Eb/N sin , (1) 8 0 8 where Q(·) is the Gaussian probability Q function. The average bit-error rate is P˜b ≈

Ps/ log2(M). The expressions for the bit-error probabilities of the individual bits in the symbol are $ % π Pb,d ≈ Q 6Eb/N sin , (2) 1 0 8 $ % 1 π Pb,d = Pb,d ≈ Q 6Eb/N sin . (3) 2 3 2 0 8 Note that the above expressions are obtained by applying approximations. The approximations are assumed to be appropriate for our purposes, but can be replaced by more exact formulas.

The equivalent noise variances of each modulation class may be determined from the diﬀerent bit-error probabilities by means of an equivalent BPSK channel,

2 1 σj = & ' . (4) −1 2 Q (Pb,dj ) Usually, LDPC codes are designed for one noise variance and it is assumed that all bits are transmitted over the corresponding channel. In this paper we design codes using the 146 Paper D

d1 ˜ ∼ 2 Pb σBPSK d2

d 3 a) Standard code design

∼ 2 Pb,d1 σ1 d1 ∼ 2 Pb,d2 σ2 d2 ∼ 2 Pb,d3 σ3 d3

b) Design for HOCs

Figure 3: Channel assumptions for a) standard code design b) design for HOCs. equivalent noise variances from (4). To compare the standard code design with our new design in a fair way, we calculate an equivalent average BPSK noise variance, denoted 2 by σBPSK. This is the noise variance of a BPSK channel that would give a bit-error probability equal to the average bit-error probability of the HOC. Thus, we compare our 2 new design for HOCs with a standard code design with σBPSK and use both with HOCs. Figure 3 shows a schematic explanation of the channel assumptions made by the receiver.

In the following, we claim that approximating the HOC channel by Ns equivalent BPSK channels together with i.i.d. channel adapters meets our requirements.

2.3 Notations

We consider a UEP-LDPC code with Nc protection classes. The proportions of each α information class, =[α1, ..., αNc−1], are given by the normalized lengths of each class corresponding to the information bits. αi equals the number of bits belonging to protection class Ci divided by the total number of information bits k. The proportion distribution of the bits in the codeword belonging to the protection classes is thus given − by p =[α1R, ..., αNc−1R, (1 R)]. Ns is the number of diﬀerent bit-error rates for the bits in a symbol. We will describe the bits with a distinct bit-error rate as belonging β to one modulation class Mj, j =1, ..., Ns. =[β1, ...,βNs ] deﬁnes the proportion of bits in the codeword that belongs to each modulation class. Paper D 147

The vector λ contains the overall variable node degree distribution, both for different Ck protection classes and different modulation classes. Let λMj ,i be the proportion of edges connected to variable nodes of degree i that belong to modulation class Mj and protection class Ck. Define λCk =[λCk ,...,λCk ]T Mj Mj ,2 Mj,dvmax and ! "T T C T T C T λ = λC1 ,...,λ Nc , ..., λC1 ,...,λ Nc , M1 M1 MNs MNs

· T λCk − λ where ( ) denotes the transpose. Mj is a column vector of length dvmax 1and is − ρ T a column vector of length (dvmax 1)NcNs. The vector =[ρ2,...,ρdcmax ] describes the check node degree distribution. We deﬁne 1 to be an all-ones vector of appropriate length.

3 UEP-LDPC Codes for Higher Order Constella- tions

3.1 Optimization of the Degree Distribution for HOCs

The mutual information messages from a check node to a variable node (xcv)andfroma variable node to a check node (xvc) at iteration l, computed by means of density evolution using the Gaussian approximation [11], are given by

dcmax (l−1) −1 (l−1) xcv =1− ρjJ((j − 1)J (1 − xvc )) , (5) j=2

N N dv s c max 2 (l) Ck − −1 (l−1) xvc = λMj ,iJ( 2 +(i 1)J (xcv )) , (6) σj j=1 k=1 i=2 with J(·) computing the mutual information x = J(m)by

− E{ −z } J(m)=1 log2(1 + e ) (7) 2 1 −z − (z−m) − √ · 4m =1 log2(1 + e ) e dz 4πm R for a consistent Gaussian random variable z ∼N(m, 2m)withmeanm and variance 2m. In the special case with only one protection class and one modulation class (corresponding 148 Paper D

to standard LDPC code design), i.e., Nc = Ns = 1, the update rule for the messages from variable nodes to check nodes is given by

dvmax (l) 2 −1 (l−1) x = λiJ( +(i − 1)J (x )) . (8) vc σ2 cv i=2 The update rule for the messages from check nodes to variable nodes is not affected by the divisionofthevariablenodedegreedistribution into sub-distributions since the check node degree distribution is constant for all protection classes and modulation classes. Equations (5) and (6) can be combined to yield the mutual information evolution of the LDPC code (l) 2 (l−1) xvc = F (λ, ρ, σ ,xvc ) . (9) (l) (l−1) (l−1) If xvc >xvc for any xvc ,thenλ and ρ describe a code ensemble for which density evolution converges for the noise variance vector σ2. Since the variable node degree distributions give proportions of edges connected to variable nodes of certain degrees, the constraint Ns Nc dvmax Ck λMj ,i = 1 (10) j=1 k=1 i=2 must be fulfilled. When optimizing a degree distribution, there are different strategies. One could for example maximize the code rate given a certain threshold, where the threshold is defined as the lowest Eb/N0 for which density evolution converges. In this paper, we choose to minimize the threshold given a fixed code rate. In order to do so, we rerun a linear programming routine while increasing the SNR, until the SNR is high enough so that it is possible to find a degree distribution for which density evolution converges. The aim of the code design is to reduce the BERs of the protection classes by taking the different error probabilities of the modulation levels into account. The design algorithm should also give the possibility to trade overall BER for UEP capability. The natural way of assigning bits from modulation classes to protection classes to achieve UEP, is to use the best protected bits from the modulation, that is, modulation class

M1, for protection class C1 and continue like that until all bits have been assigned to a protection class. However, this assignment is not guaranteed to give a degree distribution with the lowest possible threshold. As discussed later, there is always a trade-off between a low threshold and good UEP capability. By distinguishing not only between degree distributions of different protection classes but also of different modulation classes, linear programming may be used to assign bits from the modulation classes to the protection classes. Paper D 149

It is well-known that a higher connectivity of a variable node leads to better protection. Thus, the optimization target is to find a variable node degree distribution for the whole code that maximizes the average variable node degree of the class being optimized. UEP capability may be obtained by running the optimization algorithm sequentially, one protection class at a time, for an Eb/N0 slightly higher than the threshold. When the proportion of edges in the Tanner graph that are associated with a specific class is high, many messages will be passed and the local convergence of the graph is fast. However, in the limit when the number of decoder iterations tends to infinity, all messages have transited the whole graph and there should be no UEP capability left, [22]. This implies that the UEP capability theoretically should be decreasing with increasing number of iterations. However, our results show that for a reasonable number of iterations the UEP capability is not affected much by the number of decoder iterations. Ongoing research shows that the choice of code construction algorithm is critical for remaining UEP capability after a reasonable number of iterations. However, these findings are too extensive to be included here1. Figure 8 in Section 4 shows the BER as a function of the number of decoder iterations for the code design and code construction algorithm suggested in this paper.

The target function for protection class Ck can be formulated as

Ns dvmax Ck max λM ,i . (11) λ j j=1 i=2

This target function only concerns the bits of the current protection class. The optimization constraints, such as the convergence constraint (9) or the proportion distribution constraint (10), will involve the whole set of code bits. In every step, the degree distributions of lower protection class bits are ﬁxed and only degree distributions of higher protection classes may be changed, [22]. The target function will ensure a maximized average variable node degree, since a high value of the above sum implies that a large proportion of the edges is connected to variable nodes belonging to class Ck. In [22], the authors show that it is not only the average connection degree to be maximized but also the minimum variable node degree. Therefore, the second optimization target of our code design is the maximization of the minimum variable node degree of each protection class. Therefore, the optimization is ﬁrst performed for a high minimum variable node degree. If no degree distribution for which density evolution converges can be found, the optimization is repeated for a lower minimum variable node degree as in [22].

1A paper on the UEP capabilities of several LDPC construction algorithms has been submitted to TCOM 150 Paper D

Repeated optimization with the target function (11) results in a degree distribution with UEP capability. However, this target function does not account for the fact that variable nodes connected to the ﬁrst modulation class have lower error probability after demodulation than those from worse modulation classes. Therefore, we introduce a scaling factor kj for each modulation class which decreases with increasing modulation class index, i.e., k1 >k2 >...>kNs > 0.

Ns dvmax Ck max kj λM ,i (12) λ j j=1 i=2

The choice of this scaling factor aﬀects how valuable a better modulation class is compared to a modulation class with higher equivalent noise variance. We choose kj =

Ns − j + 1. This choice of kj has the effect that the linear programming algorithm, if possible while fulfilling all constraints, will use modulation classes with low noise variance for the protection class being optimized. Besides the optimization constraints given in (9) and (10), there are some more constraints which have to be fulfilled by the optimized λ. The variable node degree distribution is connected to the check node degree distribution and the code rate by dcmax j ρj/j R =1− =2 . (13) dvmax i=2 λi/i Furthermore, the proportion vectors α and β impose the following two constraints on λ, where nCk and nMj denote the total number of variable nodes belonging to protection class Ck and to modulation class Mj, respectively,

· · − nCk = αk R nk=1,...,Nc 1 , and (14)

· nMj = βj nj=1,...,Ns . (15)

λ Moreover, nCk and nMj are connected to by Ns dvmax Ck dcmax λMj ,i ρi nC = · n · (1 − R)/ (16) k i i j=1 i=2 i=2 and Nc dvmax Ck dcmax λMj ,i ρi nM = · n · (1 − R)/ , (17) j i i k=1 i=2 i=2 respectively. Paper D 151

When designing good LDPC code ensembles, the stability condition which ensures convergence of the density evolution for mutual information close to one should be fulfilled [7]. The stability condition gives an upper bound on the number of degree-2 variable nodes. For a BPSK scheme, where all bits are affected by the same noise variance σ2,we have [7] 1 −r − x − 1 2 2σ2 >e = P0(x)e dx = e (18) λ (0)ρ (1) R with P0(x) being the message density corresponding to the received values and λ (x)and ρ(x) being the derivatives of the degree polynomials. It is straightforward to see that, Ns Nc Ck dcmax · − for our scheme, λ (0) = j=1 k=1 λMj ,2 and ρ (1) = m=2 ρm (m 1). In our case, 2 the bits are affected by channel noise with different equivalent noise variances σj (see (4)) and thus different densities P ,j(x). The density of the whole set of bits is equivalent 0 to the (weighted) sum of the individual densities, i.e., j βj · P0,j(x). This gives the stability condition

N N s s − 1 −r − x 2σ2 2 j e = βj · P0,j(x)e dx = βj · e . (19) R j=1 j=1

All the above given constraints are rewritten to contain only λ, ρ, α, β, σ2,andR, i.e., without using n and k. This makes the algorithm independent of the code length.

3.2 Optimization Algorithm

The optimization algorithm proposed here is a modification of the hierarchical optimization algorithm presented in [22] for HOCs. The optimization is performed at Eb/N0 = δ + , which will be the threshold of the optimized code, where δ is the lowest possible threshold in dB for the given ρ and dvmax ,and is an offset from the lowest threshold that provides more flexibility in the choice of λ. An irregular LDPC code always provides some inherent UEP capability. This is achieved by mapping higher degree variable nodes to the more protected classes. For = 0 this is the only UEP capability available. On the other hand, by introducing >0, we allow for more freedom in the code design. The LP algorithm will assign higher degrees or larger fractions of edges to the high degrees of the first protection class and accept worse properties for the less important classes. Hence, >0 leads to increased UEP capability compared to the inherent UEP capability of the ’minimum threshold code’.

The algorithm can be divided into an inner and an outer loop. For a given Eb/N0, the outer loop runs over the protection classes, starting with the first. At this point, the degree distribution of the code is designed while optimizing the corresponding protection 152 Paper D class. As mentioned before, there are two target functions to be maximized, i.e., the average connection degree and the minimum connection degree of the class’ variable nodes. Since we are using LP with only a single-objective function, we choose it to be the maximization of the average connection degree. The maximization of the minimum variable node degree is performed by the inner loop which runs over different values for the minimum degree, starting from some maximum value and successively reducing it during the procedure. Once a valid solution is found for the current protection class, the next protection class Ck is optimized. At this point, the classes C1, ..., Ck−1 have C λC1 λ k−1 ∀ already been optimized and Mj , ..., Mj , j, are fixed. The optimization algorithm can be stated as follows.

2 I) Fix Eb/N0 = δ + and calculate σ .

II) Find λ by performing the inner optimization loop for each protection class.

For the optimization of class Ck, a linear-programming routine is executed. It requires the deﬁnition of the check node degree distribution ρ, Eb/N0 = δ + ,themaximumvari- able node degree dvmax , the code rate R, and the proportion vectors α and β.

(k) 1) Initialization dvmin = dvmax

2) While optimization failure

a) Optimize Ns dvmax Ck max kj λ (20) λ Mj ,i j=1 i=2

under the constraints [C1] − [C6].

[C1] Rate constraint, see (13) C Ns Nc dvmax λ k dcmax M ,i 1 ρi j = (21) i 1 − R i j=1 k=1 i=2 i=2

[C2] Proportion distribution constraints i) See (10) Ns Nc T λCk 1 Mj = 1 (22) j=1 k=1

ii) ∀k ∈{1,...,Nc − 1}, see (14) and (16) d C d Ns vmax λ k cmax Mj ,i R ρi = αk (23) i 1 − R i j=1 i=2 i=2 Paper D 153

iii) ∀j ∈{1,...,Ns − 1}, see (15) and (17)

C Nc dvmax λ k dcmax Mj ,i 1 ρi = βj (24) i 1 − R i k=1 i=2 i=2

[C3] Convergence constraint, see (9)

F (λ, ρ, σ2,x) >x (25)

[C4] Stability condition, see (18) and (19) ⎡ ⎤ −1 Ns Nc Ns dcmax 2 Ck −1/2σ λ < ⎣ βje j · ρm m − ⎦ Mj ,2 ( 1) (26) j=1 k=1 j=1 m=2

[C5] Minimum variable node degree constraint

∀i

[C6] Previous optimization constraints

C ∀k

(k) (k) b) If failure, dvmin = dvmin − 1

End (While)

3.3 Code Construction

When the optimal degree distribution of the variable nodes is found, a parity-check matrix is constructed by a modification of the approximate cycle extrinsic (ACE) message degree algorithm [31]. The ACE algorithm constructs a parity-check matrix following a given variable node degree distribution while selectively avoiding small cycle clusters that are isolated from the rest of the graph. For irregular LDPC codes, the ACE algorithm has good performance in the error-floor region. The original ACE algorithm [31] only ensures a certain variable node degree distribution. However, the design algorithm optimizes the variable node degree distribution given a certain check node degree distribution and a parity-check matrix with degrees given by λ and ρ is desired. The modified ACE construction of the parity-check matrix used here also ensures that the check node degree distribution equals ρ. Whereas the ones in a column are located at random positions in the original ACE algorithm, we allow only those positions where a one does not violate the defined check node degree distribution. 154 Paper D

Generally, the recently proposed ACE constrained progressive edge growth (PEG) code construction algorithm [32] has been shown to perform well compared to the ACE algorithm, especially in the error-ﬂoor region. Despite this good performance, it is unsuit- able for UEP applications, since it entirely looses its UEP capability after a few decoder iterations.

4 Simulation Results

In this section, simulation results for examples with 8-PSK and 64-QAM are presented. We denote our scheme by ”HOC-UEP”, which is a UEP-LDPC code optimized for the 2 different σj from the modulation. We compare the HOC-UEP scheme to two other schemes. The first one is a UEP-LDPC code optimized for BPSK [22] which is denoted 2 by “UEP”, designs the code for a BPSK channel with the comparable σBPSK (see Section 2.2) and assigns the best bits from the modulation to the first protection class and vice versa. The second comparison is a design without UEP similar to the design proposed in [14]. The variable node degree distributions are optimized for R =1/2, Nc =3, 7 8 9 α =[0.3, 0.7], dvmax = 30, and ρ(x)=0.00749x +0.99101x +0.00150x ,whichis found by numerical optimization in [7] to be a good check node degree distribution for dvmax = 30.

4.1 Results for 8-PSK

2 For Gray-labeled 8-PSK, the noise vector σ is calculated according to (4), with Ns =2 and β =[2/3, 1/3]. Table 1 shows the degree distributions of the UEP scheme. We choose =0.1 dB to allow for some increased UEP capability. The resulting degree C distributions λ k are given for each protection class Ck. For comparison, the degree distribution for the minimum threshold, that is = 0 dB, is also shown. This degree distribution corresponds to the code design with inherent UEP only. The degree distri- λCk butions of the HOC-UEP scheme, Mj , are given in Table 2. For comparison, we also optimize a code for HOCs but without UEP capability, which is similar to the approach in [14]. Note that we separate the degree distributions for information bits and parity bits which is not done in [14]. The degree distributions are shown in Table 3. Finite-length codeword simulations with n = 4096 and 50 decoding iterations are performed using the equivalent BPSK channels. A soft demapper provides the message Paper D 155

Table 1: Degree distributions of the UEP scheme.

C1 C2 C3

=0dB λ7 =0.0799 λ3 =0.1790 λ2 =0.2103

λ8 =0.0948 λ6 =0.0737 λ3 =0.0181

λ30 =0.3029 λ7 =0.0414

=0.1dB λ11 =0.1783 λ3 =0.2041 λ2 =0.1841

λ12 =0.1184 λ4 =0.0393 λ3 =0.0575

λ30 =0.2183

Table 2: Degree distributions of the HOC-UEP scheme.

C1 C2 C3

=0dB M1 λ9 =0.1703 λ3 =0.1673 λ2 =0.1240

λ10 =0.0555

λ30 =0.1811

M2 λ30 =0.0854 λ4 =0.0225 λ2 =0.0878

λ5 =0.0738 λ3 =0.0022

λ7 =0.0117 λ4 =0.0183

=0.1dB M1 λ12 =0.3290 λ3 =0.1070 λ2 =0.1585

λ30 =0.1782

M2 λ3 =0.0396 λ2 =0.0547

λ4 =0.1026 λ3 =0.0137

λ5 =0.0165

=0.2dB M1 λ15 =0.4840 λ3 =0.0848 λ2 =0.1733

λ30 =0.0327

M2 λ3 =0.0782 λ2 =0.0414

λ4 =0.0940 λ3 =0.0115

=0.3dB M1 λ16 =0.4993 λ3 =0.0268 λ2 =0.2163

M2 λ16 =0.0345 λ3 =0.1849 λ2 =0.0092

λ4 =0.0291 156 Paper D

Table 3: HOC design without UEP similar to [14]. Information bits Parity bits

=0dB M1 λ3 =0.1474 λ4 =0.0008 λ2 =0.1237

λ5 =0.0021 λ6 =0.0008 λ3 =0.0154

λ7 =0.0023 λ8 =0.0005

λ9 =0.2204 λ10 =0.0040

λ11 =0.0008 λ12 =0.0004

λ13 =0.0002 λ14 =0.0002

λ15−25 =0.0013 λ26 =0.0002

λ27 =0.0003 λ28 =0.0004

λ29 =0.0011 λ30 =0.1751

M2 λ3 =0.0019 λ4 =0.0509 λ2 =0.0881

λ5 =0.0581 λ6 =0.0049 λ3 =0.0005

λ7 =0.0008 λ8 =0.0057

λ9 =0.0017 λ10 =0.0006

λ11 =0.0003 λ12 =0.0002

λ13 =0.0001 λ14 =0.0001

λ15−25 =0.0008 λ26 =0.0001

λ27 =0.0002 λ28 =0.0003

λ29 =0.0005 λ30 =0.0869

passing decoder with the channel log-likelihood ratios (LLRs) which are computed using 2 the appropriate noise variances σj of the modulation classes. Figure 4 shows the overall BER after 50 decoder iterations averaged over all protection classes for both =0dBand =0.1 dB. It is seen that the overall BERs of the HOC- UEP codes are lower than for the UEP codes. For an overall BER of 10−4,thereisa gain of around 0.7 dB by the HOC-UEP scheme. This is due to the consideration of 2 2 the diﬀerent σj in contrast to only assuming the average σBPSK.TheoverallBERfor the design similar to the approach in [14] (HOC without UEP) is shown for comparison, even though this scheme has no UEP capability. It is expected that the overall BERs for the codes with >0 dB are higher than for the corresponding codes with =0dB, because the thresholds of the codes are increased in order to allow an increased average variable node degree of the most protected classes. However, at high Eb/N0 the HOC- UEP scheme shows a slightly lower overall BER for the case with =0.1dBcompared to = 0 dB. Reasons for this are discussed later in this section. Paper D 157

10−1

10−2

10−3

10−4 Overall BER −5 10 UEP ε=0 dB UEP ε=0.1 dB −6 HOC−UEP ε=0 dB 10 HOC−UEP ε=0.1 dB Approach similar to [14] 10−7 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 3.2 E /N (dB) b 0

Figure 4: Overall bit-error rate performance of the UEP scheme and the HOC-UEP scheme for =0dB and =0.1 dB.

TheBERperformanceoftheindividual protection classes C1 and C2 for the UEP scheme are shown in Figure 5. Note that we do not show protection class C3 because it contains parity bits only and is of little interest for possible applications. As expected, the UEP capability is increased with increasing .

Figure 6 shows the BER performance of protection classes C1 and C2 for the HOC- UEP scheme. The results show that the HOC-UEP =0.1dBcodehaslowerBERfor both information classes than the HOC-UEP = 0 dB code. Note that the diﬀerences in

BER are partly balanced by protection class C3 which is not shown here. By comparing Figure 5 and Figure 6, we see that the BERs of the HOC-UEP scheme are much lower than those of the UEP scheme at high Eb/N0. The gain of the HOC-UEP scheme compared −6 to the UEP scheme is 0.8 dB for protection classes C1 and C2,ataBERof10 and 10−4, respectively. We also see that the diﬀerence in performance between the protection classes is higher for the HOC-UEP scheme than for the UEP scheme, especially at high

Eb/N0. Because of the large diﬀerence in BER of the protection classes and also the knowledge of the proportion of bits in the classes, we know that it is mainly C2 that governs the performance in terms of the overall BER. By comparing also with Figure 4, it is seen that the overall BER of the approach similar to [14] without UEP is almost equal to the overall BER of the HOC-UEP scheme with = 0 dB and worse than the 158 Paper D

10−1

10−2

10−3

10−4 BER

−5 UEP C ε=0 dB 10 1 UEP C ε=0 dB 2 −6 UEP C ε=0.1 dB 10 1 UEP C ε=0.1 dB 2

10−7 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 3.2 E /N (dB) b 0

Figure 5: Bit-error rate performance of protection class C1 and C2 for the UEP scheme and 8-PSK.

HOC-UEP scheme with =0.1 dB. When comparing these schemes, remember also that the HOC-UEP scheme provides a BER signiﬁcantly lower than the overall BER for C1.

Figure 7 shows the effect of different threshold offsets in the HOC-UEP scheme, with

BER as a function of for Eb/N0 =2.4dB.Itshowsthat =0.1 dB is a good choice for our example, since the BERs for both classes are lower than for code design without increased UEP capability ( = 0 dB). For higher values of , the ﬁrst class is assigned even more edges and the second and third classes have concentrated low degrees. With increasing , which implies increasing the code threshold, the global convergence of the code becomes worse and this aﬀects the performance of all classes. Please note that for short-length codes, the degree distribution with the lowest threshold may not yield the best performance [7].

Figure 8 shows the BER of the HOC-UEP scheme as a function of the number of decoder iterations for Eb/N0 =2.4 dB. The UEP capability is not aﬀected much with increasing number of decoder iterations, at least up to a maximum of 200 iterations. Paper D 159

10−1 HOC−UEP C ε=0 dB 1 HOC−UEP C ε=0 dB 2 10−2 HOC−UEP C ε=0.1 dB 1 HOC−UEP C ε=0.1 dB 2 10−3

10−4 BER

10−5

10−6

10−7 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 3.2 E /N (dB) b 0

Figure 6: Bit-error rate performance of protection class C1 and C2 for the HOC-UEP scheme and 8-PSK.

10−1 HOC−UEP C 1 HOC−UEP C 2 10−2

10−3

10−4 BER

10−5

10−6

10−7 0 0.05 0.1 0.15 0.2 0.25 0.3 ε (dB)

Figure 7: Performance of the HOC-UEP scheme for Eb/N0 =2.4 dB for various values of and 8-PSK. 160 Paper D

10−1 UEP C ε=0.1 dB 1 UEP C ε=0.1 dB 2 10−2 HOC−UEP C ε=0.1 dB 1 HOC−UEP C ε=0.1 dB 2 10−3

10−4 BER

10−5

10−6

10−7 0 50 100 150 200 Number of iterations

Figure 8: Bit-error rate performance as a function of the number of decoder iterations for

Eb/N0 =2.4 dB and 8-PSK.

4.2 Results for 64-QAM

In this section, we give an example of a UEP-LDPC code designed for and used with

64-QAM. We design a code with the parameters n, k, R, Nc, α, ρ,anddvmax as in the previous section. 64-QAM yields three modulation classes of equal size, and thus,

Ns =3andβ =[1/3, 1/3, 1/3]. Figure 9 shows the BER performance of protection classes C1 and C2 of the 64-QAM HOC-UEP code compared to a UEP code designed for BPSK, both with =0.1 dB and transmitted over the 64-QAM modulated channel. The plot shows that for very low SNR, the BERs of the HOC-UEP scheme are slightly worse but outperform the UEP scheme for an SNR higher than approximately 1.7 dB. −5 For a BER of 10 ,HOC-UEPC1 has an Eb/N0 gain of almost 1.5 dB compared to UEP

C1. This result conﬁrms the assumption that a higher modulation index yields greater performance gain of the HOC-UEP scheme compared to the UEP scheme.

5 Conclusions

We present a ﬂexible design method for UEP-LDPC codes with higher order constellations. The design algorithm is applicable to arbitrary signal constellations, an arbitrary Paper D 161

10−1

10−2

10−3

10−4 BER

−5 UEP C ε=0.1 dB 10 1 UEP C ε=0.1 dB 2 −6 HOC−UEP C ε=0.1 dB 10 1 HOC−UEP C ε=0.1 dB 2 10−7 1.5 2 2.5 3 3.5 4 E /N (dB) b 0

Figure 9: Bit-error rate performance of the UEP and HOC-UEP schemes for 64-QAM modulation. number of protection classes and arbitrary protection class sizes. For an example with 8-PSK, it is shown that the overall BER is reduced by the proposed method and there is a gain of 0.7dBatBER10−4 compared to using the UEP-LDPC codes designed for BPSK. An example with 64-QAM shows an even higher gain. The results also show that the UEP capability is increased if codes are properly designed for the modulation scheme. The proposed code design is compared to a design method for higher order constellations without UEP and we show that the two methods have similar overall BER even though our scheme provides a protection class with BER signiﬁcantly lower than the overall BER. The achieved UEP capability may be of major interest for a number of applications, e.g., when header information must be transmitted.

References

[1] G. Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Transactions on Information Theory, vol. 28, pp. 55–67, Jan. 1982.

[2] H. Imai and S. Hirakawa, “A new multilevel coding method using error correcting codes,” IEEE Transactions on Information Theory, vol. 23, pp. 371–377, May 1977. 162 Paper D

[3] G. Ungerboeck, “Trellis-coded modulation with redundant signal sets Part I: Intro- duction,” IEEE Communications Magazine, vol. 25, pp. 5–11, Feb. 1987.

[4] G. Ungerboeck, “Trellis-coded modulation with redundant signal sets Part II: State of the art,” IEEE Communications Magazine, vol. 25, pp. 12–21, Feb. 1987.

[5] U. Wachsmann, R. Fischer, and J. Huber, “Multilevel codes: Theoretical concepts and practical design rules,” IEEE Transactions on Information Theory, vol. 45, pp. 1361 – 1391, July 1999.

[6] R. Gallager, “Low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 8, pp. 21 – 28, Jan. 1962.

[7] T. Richardson, M. Shokrollahi, and R. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 47, pp. 619–637, Feb. 2001.

[8] M. Aydinlik and M. Salehi, “Turbo coded modulation for unequal error protection,” IEEE Transactions on Communications, vol. 56, pp. 555–564, Apr. 2008.

[9] M. Tanner, “A recursive approach to low complexity codes,” IEEE Transactions on Information Theory, pp. 533–547, Sep. 1981.

[10] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum- product algorithm,” IEEE Transactions on Information Theory, pp. 498–519, Feb. 2001.

[11] S.-Y. Chung, T. Richardson, and R. Urbanke, “Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation,” IEEE Transactions on Information Theory, vol. 47, pp. 657–670, Feb. 2001.

[12] J. Hou, P. Siegel, L. Milstein, and H. Pﬁster, “Capacity-approaching bandwidth- eﬃcient coded modulation schemes based on low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 49, pp. 2141 – 2155, Sept. 2003.

[13] G. Caire, G. Taricco, and E. Biglieri, “Bit-interleaved coded modulation,” IEEE Transactions on Information Theory, vol. 44, pp. 927–946, May 1998.

[14] H. Sankar, N. Sindhushayana, and K. Narayanan, “Design of low-density parity- check (LDPC) codes for high order constellations,” in Proc. Globecom 2004, pp. 3113–3117, Nov. 2004. Paper D 163

[15] A. Sanaei and M. Ardakani, “LDPC code design for nonuniform power-line channels,” EURASIP Journal on Advances in Signal Processing, vol. 2007, pp. Article ID 76146, 9 pages, 2007. doi:10.1155/2007/76146.

[16] Y. Li and W. Ryan, “Bit-reliability mapping in LDPC-coded modulation systems,” IEEE Commununications Letters, vol. 9, pp. 1–3, Jan. 2005.

[17] R. Maddock and A. Banihashemi, “Reliability-based coded modulation with low- density parity-check codes,” IEEE Transactions on Communications, vol. 54, pp. 403–406, Mar. 2006.

[18] V. Kumar and O. Milenkovic, “On unequal error protection LDPC codes based on Plotkin-type constructions,” IEEE Transactions on Communications, vol. 54, pp. 994–1005, June 2006.

[19] K. Kasai, T. Shibuya, and K. Sakaniwa, “Detailed representation of irregular LDPC code ensembles and density evolution,” in Proc. ISIT 2003, p. 121, June 2003.

[20] C. Poulliat, D. Declercq, and I. Fijalkow, “Optimization of LDPC codes for UEP channels,” in Proc. ISIT 2004, p. 451, June 2004.

[21] L. Sassatelli, W. Henkel, and D. Declercq, “Check-irregular LDPC codes for unequal error protection under iterative decoding,” in 4th International Symposium on Turbo-codes and related topics (D. D. L. Sassatelli, W. Henkel, ed.), 2006.

[22] C. Poulliat, D. Declercq, and I. Fijalkow, “Enhancement of unequal error protection properties of LDPC codes,” EURASIP Journal on Wireless Communications and Networking, vol. 2007, pp. Article ID 92659, 9 pages, 2007. doi:10.1155/2007/92659.

[23] N. Rahnavard, H. Pishro-Nik, and F. Fekri, “Unequal error protection using partially regular LDPC codes,” IEEE Transactions on Communications, vol. 55, pp. 387–391, Mar. 2007.

[24] H. Pishro-Nik, N. Rahnavard, and F. Fekri, “Nonuniform error correction using low- density parity-check codes,” IEEE Transactions on Information Theory, vol. 51, pp. 2702–2714, July 2005.

[25] K. Fazel and M. Ruf, “Combined multilevel coding and multiresolution modulation,” in Proc. ICC 1993, pp. 1081–1085, May 1993. 164

[26] Y. Liu and C. Heneghan, “Optimizing scalable media content delivery using hierarchical modulation techniques,” in European Symp. on Mobile Media Delivery (EuMob), Sept. 2006.

[27] A. Sezgin, D. Wübben, R. Bönke, and V. Kün, “On EXIT-charts for space-time block codes,” in Proc. ISIT 2003, p. 64, June 2003.

[28] Y. Nana, E. Sharon, and S. Lytsin, “Improved decoding of LDPC coded modulation,” IEEE Communications Letters, vol. 10, pp. 375–377, May 2006.

[29] J. Hou, P. Siegel, L. Milstein, and D. Pﬁster, “Multilevel coding with low-density parity-check component codes,” in Proc. Globecom ’01, vol. 2, pp. 1016–1020, Nov. 2001.

[30] J. G. Proakis, Digital Communications. New York: McGraw-Hill, 4th ed., 2001.

[31] T. Tian, C. Jones, D. Villasenor, and R. Wesel, “Selective avoidance of cycles in irregular LDPC code construction,” IEEE Transactions on Communications, vol. 52, pp. 1242–1247, Aug. 2004.

[32] D. Vukobratovic and V. Senk, “Generalized ACE constrained progressive edge- growth LDPC code design,” IEEE Communications Letters, vol. 12, pp. 32–34, Jan. 2008. Paper E On the UEP Capabilities of Several LDPC Construction Algorithms

Authors: Neele von Deetzen and Sara Sandberg

Reformatted version of paper submitted to: IEEE Transactions on Communications, Nov. 18, 2008.

c 2008, IEEE. Reprinted with permission.

165 166 Paper E ON THE UEP CAPABILITIES OF SEVERAL LDPC CONSTRUCTION ALGORITHMS

Neele von Deetzen and Sara Sandberg

Abstract

Design of irregular low-density parity-check (LDPC) codes for enhanced unequal error protection (UEP) properties has received much interest lately. Several authors show that the connection degree of a variable node affects its bit-error probability. However, some authors argue that this is valid only for a very small number of decoding iterations while others show good UEP properties also after a large number of iterations. This paper analyzes construction algorithms for the parity-check matrices of LDPC codes with respect to their capabilities of providing UEP. We show that the choice of code construction algorithm is critical and highly affects the performance and the UEP properties of the code. Our results explain disagreements in earlier research. Furthermore, the differences in UEP capability are explained by the introduction of detailed check node degree distributions. We further demonstrate that the UEP capability of a code may be enhanced by modifying its detailed check node degree distribution.

1 Introduction

Unequal error protection (UEP) low-density parity-check (LDPC) codes that provide more protection for certain bits within the codeword are important for applications where the source bits have different sensitivities to errors. The desired UEP properties are a low bit-error rate (BER) or frame-error rate (FER) within one or several classes of bits, while the performance of the remaining classes should be comparable to non-UEP codes. Such codes can, for example, be constructed by an algebraic method based on Plotkin-type constructions [1]. However, since it is widely observed that the connection degree of a variable node affects its BER, at least for a limited number of decoding iterations, it is typical to design the variable and/or check node degree distribution of the code in an irregular way using density evolution [2–4]. It should be noted though, that the results of papers on irregular UEP-LDPC codes disagree. For example, [4] shows significant UEP capabilities after 200 message-passing iterations, while [1] argues that no UEP gradation 168 Paper E can be detected for irregular UEP-LDPC codes after 50 iterations. In this paper we explain the reasons behind the disagreeing results by analyzing different construction algorithms with respect to how the graph properties of the corresponding codes affect the UEP capabilities. This paper focuses on low-density parity-check codes, originally presented by Gallager in [5]. They exhibit a performance very close to the capacity for the binary-input additive white Gaussian noise channel [6]. LDPC codes are block codes with a sparse parity-check matrix H of dimension (n−k)×n,wherek and n are the lengths of the information word and the codeword, respectively, and R = k/n denotes the code rate. An LDPC code can be represented by a bipartite graph, called Tanner graph [7], which facilitates a decoding algorithm known as the message-passing algorithm [8]. The graph consists of two types of nodes, variable nodes and check nodes, which correspond to the bits of the codeword and to the parity-check constraints, respectively. A variable node is connected to a check node if the bit is included in the parity-check constraint. For regular LDPC codes, all variable nodes have degree dv and all check nodes have degree dc. However, irregular LDPC codes are known to approach capacity more closely than regular LDPC codes. We consider irregular LDPC codes with variable node and check node degree distributions dvmax i−1 dcmax i−1 defined by the polynomials [6] λ(x)= i=2 λix and ρ(x)= i=2 ρix ,where dvmax and dcmax are the maximum variable and check node degree of the code. The coefficients of the degree distributions describe the proportion of edges connected to nodes with a certain degree. UEP is usually obtained by assigning important bits to high-degree variable nodes and less important bits to the lower-degrees. Good degree distributions are commonly computed by means of density evolution using a Gaussian approximation [9]. Once a degree distribution is obtained, a parity-check matrix H has to be constructed according to the degree profile. Many construction algorithms, such as the approximate cycle extrinsic message degree (ACE) algorithm [10], the progressive edge-growth (PEG) algorithm [11], etc., have been developed. Random construction, following the approach of [6], was typically used a few years ago. However, several authors have suggested construction algorithms with better BER performance, especially in the error-floor region, mainly by avoiding small cycles in the Tanner graph. We consider five different algorithms for the construction of the parity-check matrix. All of the obtained codes belong to the same code ensemble, i.e., they have the same variable and check node degree distributions. In this paper we confirm by simulation that the design of an irregular variable node degree distribution provides UEP capability for a low number of message-passing iterations regardless of the construction algorithm used. However, the results also show that Paper E 169 the choice of the construction algorithm is critical when good UEP properties are desired after a moderate or high number of iterations. UEP capability after many iterations is important since this enables considerably lower error rates than a low number of iterations. To ensure UEP capability of the code regardless of the choice of construction algorithm, the decoder must typically be interrupted after only 10 iterations, resulting in performance losses. The paper is organized as follows. Several construction algorithms are presented in Section 2. In Section 3, we shortly describe the ensemble design and discuss performance results of the algorithms and their differences. Section 4 discusses properties of parity- check matrices H which are relevant for the UEP behavior. In Section 5, we modify one of the construction algorithms to enhance the UEP capability of the resulting code. Finally, conclusions are drawn in Section 6.

2 Construction Algorithms

In this section, we will describe the basic concepts of several well-known construction algorithms, such as random construction, the progressive edge-growth (PEG) algorithm, zigzag construction, the approximate cycle extrinsic message degree (ACE) algorithm, and an ACE-constrained PEG algorithm. For the sake of brevity, we only introduce the algorithms. For details, the reader is referred to the references given in the respective sections.

2.1 Random Construction

The random construction algorithm places the edges randomly in the graph, according to the given variable node and check node degree distributions. We consider a random construction where only length-4 cycles between degree-2 variable nodes are avoided. The resulting Tanner graph has girth g =4.

2.2 Progressive Edge-Growth (PEG) Construction

The PEG algorithm is an eﬃcient algorithm for the construction of parity-check matrices with large girth by progressively connecting variable nodes and check nodes [11]. Starting with the lowest-degree variable nodes, edges are chosen according to an edge selection procedure. The selection procedure aims at minimizing the impact of each new edge on the girth. This is done by expanding a tree from the current variable node down to a certain depth. If possible, the algorithm connects the variable node to a check node that 170 Paper E is not yet reached at this depth of the tree. If all check nodes are reached, the check node that will create the largest cycle through the current variable node is selected. If several possible choices exist, a check node with the lowest check node degree under the current graph setting is chosen. Finally, if there is more than one possible lowest-degree check node left, one of them is chosen randomly. Thereby, the resulting Tanner graph automatically has a concentrated check node degree distribution. This is shown in [6] to be advantageous for good convergence.

2.3 Zigzag Construction

The zigzag construction algorithm connects the edges of degree-two variable nodes in azigzagmanner,accordingto[12].Inprinciple, the part of the parity-check matrix corresponding to the degree-two variable nodes will be bi-diagonal. This algorithm avoids all cycles involving only variable nodes of degree two, assuming that the number n2 of degree-two variable nodes is smaller than the number of check nodes, i.e., n2

2.4 Approximate Cycle Extrinsic Message Degree (ACE) Con- struction

For finite block lengths, Tian et al. [10] proposed an efficient graph conditioning algorithm called the approximate cycle extrinsic message degree (ACE) algorithm. The aim of the ACE construction is to lower the error floor by emphasizing both the connectivity as well as the length of cycles. The ACE algorithm avoids small cycle clusters that are isolated from the rest of the graph and is shown to lower the error-floors of irregular LDPC codes significantly, while only slightly degrading the performance in the waterfall region. The approximate cycle extrinsic message degree of a length-2d cycle is i(di − 2), where di is the degree of the ith variable node in the cycle. An LDPC code has parameters

(dACE,η) if all the cycles whose lengths are 2dACE or less have an ACE of at least η.The ACE algorithm is an eﬃcient Viterbi-like algorithm with linear complexity proposed to detect and avoid harmful short cycles during code construction. Cycles with low ACE are considered to be harmful, since these cycles include variable nodes with low degrees Paper E 171 and the cycles have low connectivity to nodes outside the cycle. Given the variable node degree distribution λ(x), columns of the parity-check matrix are generated one at a time starting from low-weight columns. The edges of every new column are generated randomly and the ACE algorithm checks whether the (dACE,η) requirement is met. If not, this column is generated again. This procedure is repeated until the whole parity check matrix is generated. In our implementation, we add extra constraints to construct a graph according to the given check node degree distribution ρ(x). The performance of highly irregular LDPC codes is of great interest when considering irregular UEP-LDPC codes, since the UEP properties are obtained by assigning variable nodes with high degree to the best protected class. However, with highly irregular codes it is diﬃcult to ensure a high girth and the ACE algorithm is of importance.

2.5 PEG-ACE Construction

The PEG-ACE algorithm is a generalization of the popular PEG algorithm, that is shown to generate good LDPC codes with short and moderate block lengths having large girth. In the PEG-ACE algorithm [13], the sequential methodology of the PEG algorithm is employed with a modified check node selection procedure. If the creation of cycles cannot be avoided while adding an edge, the PEG-ACE construction algorithm chooses an edge that creates the longest possible cycle with the largest possible ACE value. The algorithm constructs a code with the largest possible ACE constraints, which is assumed to reduce the number of trapping sets significantly. As the PEG algorithm, the PEG-ACE construction algorithm usually picks a check node among its candidates which has the lowest check node degree. It has been shown that a PEG-ACE code performs better than an ACE code in the error-floor region [13].

3 Simulation Results

3.1 Ensemble Design

We consider the UEP-LDPC ensemble design proposed in [3], which is based on a hierarchical optimization of the variable node degree distribution for each protection class. The algorithm maximizes the average variable node degree within one class at a time while guaranteeing a minimum variable node degree as high as possible. The optimization can be stated as a linear programming problem and can, thus, be easily solved. To keep the overall performance of the UEP-LDPC code reasonably good, the search for UEP codes 172 Paper E is limited to degree distributions whose convergence thresholds lie within a certain range of the minimum threshold of a code with the same parameters. We fix to 0.1dB, which is shown in [3] to give a good trade-off between the performances of the protection classes. The UEP-LDPC ensemble design algorithm is initialized with a maximum variable node degree dvmax , the code rate R, and a check node degree distribution. The bits of the codeword are divided into protection classes Cj according to their protection requirements. We design a UEP-LDPC code with Nc = 3 protection classes with rate 1/2, 7 8 9 dvmax =30andρ(x)=0.00749x +0.99101x +0.00150x , which is found by numerical optimization in [6] to be a good check node degree distribution for dvmax = 30. The proportions of the classes are chosen such that C1 contains 20% of the information bits and C2 contains 80%. The third class (C3) contains all parity bits. Therefore, we are mainly interested in the performances of classes C1 and C2. The resulting variable node (Cj ) degree distribution is defined by the coefficients λi which denote the fractions of edges incident to degree-i variable nodes of protection class Cj. The overall degree distribution j Nc dvmax (C ) i−1 is therewith given by λ(x)= j=1 i=2 λi x . Table 1 summarizes the optimized variable node degree distribution for the resulting UEP-LDPC code.

Table 1: Variable node degree distribution of the UEP-LDPC ensemble.

C1 C2 C3 (C1) (C2) (C3) λ18 =0.2521 λ3 =0.0786 λ2 =0.2130 (C1) (C2) (C3) λ19 =0.0965 λ4 =0.2511 λ3 =0.0141 (C1) λ30 =0.0946

3.2 Performance Comparison

UEP-LDPC codes with length n = 4096 are constructed using the construction algorithms described in Section 2. All codes belong to the ensemble described above. From each construction algorithm, we consider one code realization in the following. It should be noted that the diﬀerences between several code realizations constructed with the same construction algorithm are small. We present simulation results for BPSK transmission over the AWGN channel. Figure 1 shows the FER and the BER as a function of Eb/N0 for the random and ACE code after 100 decoder iterations. They both show good UEP properties, but the ACE code outperforms the random code for C2 and C3.Wealsosee Paper E 173

100

10−1

10−2

10−3

10−4 BER / FER Random C1 2 10−5 Random C Random C3 ACE C1 10−6 ACE C2 ACE C3 10−7 0 0.5 1 1.5 2 E /N (dB) b 0

Figure 1: FER and BER of the random code and the ACE code as a function of Eb/N0,after 100 iterations. The bold curves show FER and the thin curves show BER. Both codes show good UEP capabilities, but the ACE code outperforms the random code for C2 and C3. that the ACE code has a lower error-floor than the random code. Figure 2 shows the FER and the BER for the zigzag-random and PEG-ACE code after 100 decoder iterations. The zigzag-random code shows moderate UEP capabilities, while the PEG-ACE code does not show any UEP at all in FER and very little in BER. Simulation results for the PEG code and the zigzag-PEG code are omitted here since they show almost exactly the same performance and UEP capability as the PEG-ACE code. For simplicity, we will summarize the construction algorithms into the following two groups: non-UEP algorithms and UEP-capable algorithms. The non-UEP construction algorithms are the PEG, the zigzag-PEG, and the PEG-ACE construction. The UEP-capable construction algorithms are the random, the ACE, and the zigzag-random construction. For standard code design, i.e. without UEP, the PEG-ACE construction has been shown to lower the error-floor while the loss in the waterfall-region is minimal [13]. The results in Figure 2 show the same behavior. Remarkably, the PEG-ACE code shows almost no difference in performance between the classes. The PEG-ACE construction does not lower the error-floors of all classes compared to the random construction as may 174 Paper E

100

10−1

10−2

10−3

10−4 BER / FER Zigzag−random C1 2 10−5 Zigzag−random C Zigzag−random C3 PEG−ACE C1 10−6 PEG−ACE C2 PEG−ACE C3 10−7 0 0.5 1 1.5 2 E /N (dB) b 0

Figure 2: FER and BER of the zigzag-random code and the PEG-ACE code as a function of

Eb/N0, after 100 iterations. The bold curves show FER and the thin curves show BER. The zigzag-random code shows moderate UEP capabilities, while the PEG-ACE code does not show any UEP at all in FER and very little in BER. be expected, but it removes the UEP capability by improving C2 and C3 while degrading C1. Also, the loss in the waterfall-region is slightly higher than shown for the standard code design, while the gain in the error-ﬂoor region is substantial since all classes have low error ﬂoors.

Figure 3 shows the BER as a function of decoder iterations at Eb/N0 =1.25 dB for one UEP-capable code and one non-UEP code, that is, the ACE and the PEG-ACE code. The other algorithms in both groups show similar results, corresponding to the respective group. Both groups have similar performance for the first 10 iterations where there is a significant difference in BER between C1 and C2. However, limiting the number of iterations to less than 10 would cause a huge performance loss. For a high number of iterations, it is seen that the PEG-ACE code has almost no UEP. For low Eb/N0,all classes of the PEG-ACE code perform fairly bad compared to the other codes, see Figure 2. However, in the error-floor region, C2 and C3 of the PEG-ACE code have much lower error rates than the other codes. Paper E 175

100 ACE C1 ACE C2 3 10−1 ACE C PEG−ACE C1 PEG−ACE C2 3 10−2 PEG−ACE C BER 10−3

10−4

10−5 0 50 100 150 200 Iteration

Figure 3: BER as a function of the number of decoder iterations for the ACE and the PEG-

ACE code at Eb/N0 =1.25 dB. Both codes show good UEP properties after a small number of iterations (< 10), but after a large number of iterations all classes of the PEG-ACE code perform almost the same.

The results presented in this section suggest the use of the PEG-ACE code for high

Eb/N0.AtEb/N0 =1.6 dB, all classes of PEG-ACE have the same performance as the best class of ACE. Good performance of all classes is of course even better than UEP capability with only good performance of the most protected class. However, for low

Eb/N0, the PEG-ACE code performs badly and the ACE code with UEP capability is a better choice.

4 Relevant Graph Properties

In this section, we present properties of the Tanner graph which are relevant for the UEP behavior of the code. These properties concern the connectivities between variable nodes of different protection classes. It should also be mentioned that the UEP capability is mainly independent of finite-length issues. We investigated cycles and trapping sets and found differences between the algorithms. However, later results will show that there is 176 Paper E

Table 2: The number of cycles of length 4 and 6 involving at least one variable node from the respective class. Length 4 Length 6 C1 C2 C3 C1 C2 C3 Random 4 909 37 10 196 594 35 143 17 463 ACE 3 582 236 48 183 279 31 065 11 516 Zigzag-random 2 020 0 0 195 297 19 683 12 535 PEG 0 0 0 119 226 18 874 5 203 PEG-ACE 0 0 0 115 892 10 698 943

no direct connection between cycles and the UEP capability. To compare the algorithms in terms of cycles we count the cycles encountered from each variable node and sum over all variable nodes in the class. This means that each cycle will be counted several times (2 times for cycles of length 4, etc.). Table 2 gives the number of cycles of lengths 4 and 6 that involve at least one variable node from the respective class. The table shows that the UEP-capable codes (random, ACE and zigzag-random) contain many cycles of length 4 in C1 and very few in the other classes. The non-UEP codes all have girth 6 and thus contain no cycles of length 4. Interestingly, the cycles in C1 do not seem to affect the performance of C1 much, since the BERs of C1 for the UEP-capable codes are still lower than for the non-UEP codes. This is in agreement with the observations in [10] that only isolated short cycles are harmful. Since the degrees of variable nodes in C1 are relatively high, the cycles in C1 are not isolated but have high ACE values. On the other hand, short cycles facilitate the existence of trapping sets in a graph. Trapping sets have been shown to govern the error-floor performance of LDPC codes [14]. The number of cycles is therefore of importance for the error-floor performance.

4.1 Connectivity Between Protection Classes

Since the degree distributions λ(x)andρ(x) are equal for all codes, we investigate how the variable nodes of diﬀerent protection classes are connected through the check nodes. It is especially interesting to see how the incident variable nodes of a check node are spread between the classes. Let us consider a certain check node and how the edges of this check node are connected to the protection classes. We investigate if the edges are Paper E 177 uniformly distributed to the protection classes or if a majority of edges is connected to a certain protection class. Generally, a check node degree distribution may be deﬁned from the node’s perspective as dcmax i−1 ρ˜(x)= ρ˜ix . i=2

The coefficientsρ ˜i correspond to the fraction of degree-i check nodes. In order to account for connections to different protection classes, we define detailed check node degree distributions for the protection classes Cj,

dcmax j (Cj ) (C ) i−1 ρ˜ (x)= ρ˜i x ,i=1...Nc . (1) i=0

(Cj ) The coefficientsρ ˜i correspond to the fraction of check nodes with i edges connected to class-Cj variables nodes. Note that i is not the overall degree of the check nodes but only the number of edges which are connected to class-Cj variable nodes. For example, 1 (C ) 1 ρ˜4 is the number of all check nodes with exactly 4 edges connected to C , divided by n − k, where the remaining edges of these check nodes may be arbitrarily connected to j dcmax (C ) the other classes. By definition we have i=0 ρ˜i =1,j=1,...,Nc . This detailed check node degree distribution is similar to the detailed representation described in [15], but we consider the degree distribution from the node’s perspective while [15] considers the edge’s perspective. The representation in [15] is also more detailed than necessary for our purpose since it defines connections to nodes of certain degrees instead of certain protection classes. Table 3 presents the coefficients of the detailed check node degree distributions for the ACE, the zigzag-random, and the PEG-ACE code. Note that the maximum check node degree of the whole code is dcmax = 10. Most of the coefficients of the ACE code are non-zero for all three protection classes, while the PEG-ACE code has only a few non-zero coefficients. That is, the PEG-ACE code only has a few different types of check nodes, and the numbers of connections to the (C1) (C2) protection classes are very similar for all nodes: The PEG-ACE coefficientsρ ˜4 ,˜ρ3 3 (C ) 1 andρ ˜2 are all large, which shows that most check nodes have 4 edges connected to C , 3edgestoC2, and 2 edges to C3. This means that the variable nodes of a protection class are generally well connected to other protection classes through the check nodes. In contrast, the ACE code exhibits many different kinds of check nodes. Some of the check nodes are mainly connected to one protection class, having only one or two edges going to other protection classes. There even exist check nodes having 10 edges to a protection class, which means that they are solely connected to this class. This property seems 178 Paper E

Table 3: Detailed check node degree distributions. ACE Zigzag-random PEG-ACE C1 C2 C3 C1 C2 C3 C1 C2 C3 (Cj ) ρ˜0 0.04102 0.02588 0 0.04248 0.06152 0 0 0 0 (Cj ) ρ˜1 0.05273 0.12158 0.48193 0.05664 0.14258 0.04199 0 0.00537 0 (Cj ) ρ˜2 0.10742 0.23584 0.24268 0.10889 0.23193 0.87842 0.00049 0.12109 0.95752 (Cj ) ρ˜3 0.14795 0.27686 0.14648 0.14746 0.22363 0.07568 0.10498 0.77832 0.04248 (Cj ) ρ˜4 0.23926 0.21289 0.06445 0.21240 0.14258 0.00342 0.79980 0.09473 0 (Cj ) ρ˜5 0.21094 0.08594 0.03516 0.23145 0.10693 0.00049 0.09424 0.00049 0 (Cj ) ρ˜6 0.14453 0.03027 0.01611 0.14258 0.05664 0 0.00049 0 0 (Cj ) ρ˜7 0.05029 0.00879 0.00537 0.05566 0.03174 0 0 0 0 (Cj ) ρ˜8 0.00586 0.00195 0.00391 0.00244 0.00244 0 0 0 0 (Cj ) ρ˜9 0 0 0.00293 0 0 0 0 0 0 (Cj ) ρ˜10 0 0 0.00098 0 0 0 0 0 0 to be important for the capability of providing UEP. With more non-zero coefficients the classes are more isolated and the propagation of messages between the classes will be slower. If most check nodes have several edges to all protection classes, reliable and unreliable messages from different classes may proceed to other classes more easily and affect their performance. The detailed check node degree distributions of the zigzag-random code have few non-zero coefficients for C3, while the distributions for C1 and C2 are similar to the ACE code. Many check nodes have two edges connected to C3, while the connectivities to the other classes vary. The connectivity between the classes is therefore higher than for the ACE code, but lower than for the PEG-ACE code. This agrees with the UEP capabilities, since the zigzag-random code shows less UEP than the ACE code and more than the PEG-ACE code. The detailed distributions of the other codes are omitted here since the distribution of the random code is very similar to that of the ACE code and all non-UEP codes have almost the same detailed check node degree distributions. In order to visualize the observations, Figure 4 shows the detailed check node degree distributions of the ACE, the zigzag-random, and the PEG-ACE code.

4.2 Detailed Mutual Information Evolution

The effect of the connectivity between the classes on the performance may be analyzed by calculating the mutual information (MI) functions of the different codes. By calculating the theoretical MI functions, the effect of connectivity is isolated from effects due Paper E 179

ACE Zigzag−random PEG−ACE 1 1 1

(C1) ρ˜i 0.5 0.5 0.5

0 0 0 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 2 3 4 5 6 i i i 1 1 1

(C2) ρ˜i 0.5 0.5 0.5

0 0 0 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 1 2 3 4 5 i i i 1 1 1

(C3) ρ˜i 0.5 0.5 0.5

0 0 0 1 2 3 4 5 6 7 8 9 1 2 3 4 5 2 3 i i i

Figure 4: Detailed check node degree distributions of the ACE, the zigzag-random, and the PEG-ACE code. The UEP-capable ACE code has many non-zero coefficients, while the non- UEP PEG-ACE code has few non-zero coefficients and thereby better connectivity between the classes. to cycles, trapping sets, and codeword length, since the calculation is based on the corresponding cycle-free graph. Typically, the MI functions are calculated from the degree distributions λ(x)andρ(x) of a code. However, in our case all codes have the same overall degree distributions λ(x)andρ(x). To observe the differences between the algorithms, a detailed computation of MI may be performed by considering the edge-based MI messages traversing the graph instead of node-based averages. This has been done for protographs in [16]. We follow the same approach, but use the parity-check matrix instead of the protograph base matrix. See [17,18] for more details on MI analysis.

Let IAv be the aprioriMI between one input message and the codeword bit associated to the variable node. IEv is the extrinsic MI between one output message and the codeword bit. Similarly on the check node side, we deﬁne IAc (IEc)tobetheapriori (extrinsic) MI between one check node input (output) message and the codeword bit corresponding to the variable node providing (receiving) the message. The evolution is 180 Paper E initialized by the MI between one received message and the corresponding codeword bit, denoted by Ich, which corresponds to the channel capacity. For the AWGN channel, it is given by Ich = J(σch), where 2 Eb σch =8R (2) N0 and Eb/N0 is the signal-to-noise ratio at which the analysis is performed. The function J(·) is deﬁned by ∞ 2 2 1 − (y−σ /2) −y − √ 2σ2 J(σ)=1 e log2(1 + e )dy (3) −∞ 2πσ2 and computes the MI based on the noise variance. For a variable node with degree dv, the extrinsic MI between the s-th output message and the corresponding codeword bit is [16] ⎛ ⎞ dv ⎝ − − ⎠ IEv|s = J [J 1(IAv|l)]2 +[J 1(Ich)]2 , (4) l=1,l= s where IAv|l is the aprioriMI of the message received by the variable node on its l-th edge. The extrinsic MI for a check node with degree dc may be written as ⎛ ⎞ dc ⎝ − ⎠ IEc|s =1− J [J 1(1 − IAc|l)]2 , (5) l=1,l= s where IAc|l is the aprioriMI of the message received by the check node on its l-th edge. Note that the MI functions are subject to the Gaussian approximation (see [9]) and are not exact. The following algorithm describes the MI analysis of a given parity-check matrix. We denote element (i, j) of the parity-check matrix by hi,j.

1) Initialization

Ich = J(σch)

2) Variable to check update

a) For i =1,...,n− k and j =1,...,n,ifhi,j =1,calculate ⎛ ⎞ ⎝ − − ⎠ IEv(i, j)=J [J 1(IAv(s, j))]2 +(J 1(Ich))2 , (6)

s∈Ci,s= i

where Ci is the set of check nodes incident to variable node i. Paper E 181

b) If hi,j =0,IEv(i, j)=0.

c) For i =1,...,n− k and j =1,...,n,setIAc(i, j)=IEv(i, j).

3) Check to variable update

a) For i =1,...,n− k and j =1,...,n,ifhi,j =1,calculate ⎛ ⎞ ⎝ − ⎠ IEc(i, j)=1− J [J 1(1 − IAc(i, s))]2 , (7)

s∈Vj ,s= j

where Vj is the set of variable nodes incident to check node j.

b) If hi,j =0,IEc(i, j)=0.

c) For i =1,...,n− k and j =1,...,n,setIAv(i, j)=IEc(i, j).

4) Aposterioricheck node MI For i =1,...,n− k,calculate ⎛ ⎞ ⎝ − ⎠ IAPPc(i)=1− J [J 1(1 − IAc(i, s))]2 . (8)

s∈Vj

5) Aposteriorivariable node MI For j =1,...,n,calculate ⎛ ⎞ ⎝ − − ⎠ IAPPv(j)=J [J 1(IAv(s, j))]2 +[J 1(Ich)]2 . (9)

s∈Cj

6) Repeat 2)-5) until IAPPv =1forj =1,...,n.

The a posteriori MI of the check nodes IAPPc at Eb/N0 =0.7dB is shown in Figure 5 for the ACE and PEG-ACE code. An average IAPPc of each class is calculated as an average IAPPc of all edges incident to variable nodes of the corresponding class. The

figure shows that the average IAPPc of all classes are almost equal for the PEG-ACE code, while they differ for the ACE code. This behavior may be explained by (8) and the connections of the check nodes to the different protection classes described by the detailed check node degree distributions in Table 3. For the PEG-ACE code, almost all check nodes are connected to 4 variable nodes from C1, 3 variable nodes from C2 and 2 variable nodes from C3. Even if the extrinsic information from the variable nodes (IEv = IAc) differs (due to the irregular variable 182 Paper E node degree distribution), almost all check nodes combine the same number of nodes from each class. This gives almost equal IEc for all check nodes according to (8), which means that the performance of different classes will be averaged over the whole codeword in this step. For the ACE code on the other hand, the number of variable nodes from different classes that are connected to a check node differs much more. This allows for having some check nodes with higher MI than others and the difference in MI of the check nodes will increase the UEP capability of the code.

1 ACE C1 0.9 ACE C2 ACE C3 0.8 PEG−ACE C1 0.7 PEG−ACE C2 PEG−ACE C3 0.6

0.5 APPc I 0.4

0.3

0.2

0.1

0 0 10 20 30 40 50 60 Iteration

Figure 5: Check node a posteriori MI as a function of the number of decoder iterations at

Eb/N0 =0.7 dB. The average IAPPc of all classes are almost equal for the PEG-ACE code, while they diﬀer for the ACE code.

Remember that for all codes we consider here, there will be some UEP capability strictly depending on the irregularity of the LDPC code. In the ﬁrst iteration, all IEc will be almost equal (due to the concentrated check node degree distribution) and the only diﬀerence in IAPPv between the classes will depend on the variable node degrees.

Figure 6 shows the variable node a posteriori MI IAPPv over the number of iterations. It is shown in [17] that small differences in MI may lead to significant differences in BER for MI values near to 1. Therefore, the figure shows the distance of the MI to its maximum value, i.e. 1 − IAPPv, on a logarithmic scale. Paper E 183

100

10−2

10−4 APPv 1−I ACE C1 10−6 ACE C2 ACE C3 PEG−ACE C1 10−8 PEG−ACE C2 PEG−ACE C3 0 10 20 30 40 50 60 Iteration

Figure 6: Distance of variable node a posteriori MI to the maximum MI, as a function of the 1 number of decoder iterations, at Eb/N0 =0.7 dB. Protection class C of the ACE code converges faster than that of the PEG-ACE code, while the other classes take more iterations to converge for the ACE code than for the PEG-ACE code.

The figure shows that the convergence speeds of the protection classes are farther apart for the ACE algorithm. Protection class C1 of the ACE code converges faster than that of the PEG-ACE code, while the other classes take more iterations to converge for the ACE code than for the PEG-ACE code. Theoretically, the MI may approach 1 arbitrarily close and there will still be UEP. The actual amount of UEP depends on the convergence speeds of the classes, [3]. In practice, the accuracy is limited by the numerical precision of the computations and, for example, approximations of the J-function. However, using the approximation of the error probability based on mutual information from [17] ⎛ ⎞ Eb − − 8R + J 1(IAv)2 + J 1(IEv)2 1 ⎝ N0 ⎠ Pb ≈ erfc √ , (10) 2 2 2 the error probabilities of the protection classes may be estimated. Note that the results are not exact for finite-length codes and become more inexact with lower error rates. Nevertheless, we observe significant differences in BER between the classes if their MI values are close to 1 and only differ in the fifth decimal position. The above discussion shows that the connectivity of the protection classes, defined by 184 Paper E the detailed check node degree distribution in Table 3 plays an important role to the UEP capability. However, in comparing the theoretical MI from above with the true values obtained by measuring the actual decoding LLRs, we observe deviations. The measured MI values are lower than the theoretical ones which is due to finite-length issues such as cycles and trapping sets. Nevertheless, the UEP properties are valid for both theoretical and measured observations.

5 Modiﬁed PEG-ACE Construction with Increased UEP Capability

The above sections discussed the connectivity of check nodes to the protection classes and differences between the UEP codes and the non-UEP codes were found. In order to verify that the presented argument indeed is the reason for the differences in UEP capability, we modify the (non-UEP) PEG-ACE construction algorithm to yield codes with a similar detailed check node degree distribution as the UEP-capable ACE code. The algorithm is modified in such a way that it only allows check nodes for the can- j didate list which do not violate certain detailed check node degree distributionsρ ˜(C )(x). Thereby, the parity-check matrix is forced to have detailed check node profiles similar to the ACE code instead of its natural distribution. By doing so, the detailed check node degree distribution given in Table 4 is obtained. Notice that this is very similar to the detailed distributions of the ACE code given in Table 3. Figure 7 shows the BER and FER of the modified PEG-ACE code in comparison to the original PEG-ACE code. It shows that by changing the detailed check node degree distribution of the original PEG-ACE algorithm, it is possible to enhance its UEP capability significantly. Instead of equal error rates for all classes, the modification has improved the BER of C1, while slightly degrading C2 and C3. Compared to the ACE algorithm, the modified PEG-ACE even shows more UEP capability, since the performance of C1 is better for the PEG-ACE code while C2 and C3 perform worse. Furthermore, it should be mentioned that the modified PEG-ACE code is even capable of increasing the differences in BER between C2 and C3 compared to the UEP-capable codes presented in Section 2. These results are meant to further support the above discussion that the differences in UEP capability between different LDPC construction algorithms are due to the connectivity between the classes, that is, the detailed check node degree distribution. The modified PEG-ACE code has girth 6. Counting the cycles as in Section 4 yields Paper E 185

Table 4: Detailed check node degree distribution of the modiﬁed PEG-ACE code. Modiﬁed PEG-ACE C1 C2 C3 (Cj ) ρ˜0 0 0.03711 0 (Cj ) ρ˜1 0.03662 0.15137 0.48291 (Cj ) ρ˜2 0.17529 0.23340 0.22510 (Cj ) ρ˜3 0.16211 0.25098 0.14648 (Cj ) ρ˜4 0.24707 0.16650 0.05908 (Cj ) ρ˜5 0.20020 0.07666 0.08496 (Cj ) ρ˜6 0.12939 0.05078 0.00146 (Cj ) ρ˜7 0.04443 0.02686 0 (Cj ) ρ˜8 0.00488 0.00635 0 (Cj ) ρ˜9 0 0 0 (Cj ) ρ˜10 0 0 0

177 122 cycles involving at least one variable node from C1, 8 931 cycles in C2 and 838 cycles in C3. The numbers of cycles in the modified PEG-ACE graph are similar to the number of cycles in the PEG-ACE graph. Still the two algorithms have totally different UEP behavior. These results suggest that cycles do not affect the UEP capability. With the modified PEG-ACE algorithm, it is possible to enhance the UEP capability of a code by modifying its detailed check node degree distribution. Optimization of the detailed check node degree distribution is outside the scope of this paper. However, the detailed check node degree distribution may be used as a tool to move between codes with good UEP capability but lower overall performance and codes with good overall performance but less UEP capability.

6 Conclusions

We have compared the UEP capabilities of several construction algorithms. Results show that the considered algorithms may be divided into two groups: UEP-capable codes are those constructed by the random, the ACE, and the zigzag-random construction algorithm, while the non-UEP codes are constructed by the PEG, the PEG-ACE and the zigzag-PEG construction algorithm. These ﬁndings explain why earlier literature on irregular UEP-LDPC codes show disagreeing results. Analysis of the parity-check matrices of the diﬀerent codes shows that the connectivity 186 Paper E

100

10−1

10−2

10−3

10−4 BER / FER Mod. PEG−ACE C1 2 10−5 Mod. PEG−ACE C Mod. PEG−ACE C3 PEG−ACE C1 10−6 PEG−ACE C2 PEG−ACE C3 10−7 0 0.5 1 1.5 2 E /N (dB) b 0

Figure 7: BER and FER of the modified PEG-ACE code in comparison to the original PEG- ACE code. The modification of the PEG-ACE code has increased its UEP capability significantly. between the different classes is better for the non-UEP codes than for the UEP-capable codes. By introducing the concept of a detailed check node degree distribution, we have shown that the number of edges from a check node to variable nodes of different classes affects the UEP capability. Furthermore, the PEG-ACE construction algorithm has successfully been modified in order to construct good UEP-capable codes.

References

[1] V. Kumar and O. Milenkovic, “On unequal error protection LDPC codes based on Plotkin-type constructions,” IEEE Transactions on Communications, vol. 54, pp. 994–1005, June 2006.

[2] L. Sassatelli, W. Henkel, and D. Declercq, “Check-irregular LDPC codes for unequal error protection under iterative decoding,” in 4th International Symposium on Turbo-codes and related topics (D. D. L. Sassatelli, W. Henkel, ed.), 2006. Paper E 187

[3] C. Poulliat, D. Declercq, and I. Fijalkow, “Enhancement of unequal error protection properties of LDPC codes,” EURASIP Journal on Wireless Communications and Networking, vol. 2007, pp. Article ID 92659, 9 pages, 2007. doi:10.1155/2007/92659.

[4] N. Rahnavard, H. Pishro-Nik, and F. Fekri, “Unequal error protection using partially regular LDPC codes,” IEEE Transactions on Communications, vol. 55, pp. 387–391, Mar. 2007.

[5] R. Gallager, “Low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 8, pp. 21 – 28, Jan. 1962.

[6] T. Richardson, M. Shokrollahi, and R. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 47, pp. 619–637, Feb. 2001.

[7] M. Tanner, “A recursive approach to low complexity codes,” IEEE Transactions on Information Theory, pp. 533–547, Sep. 1981.

[8] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum- product algorithm,” IEEE Transactions on Information Theory, pp. 498–519, Feb. 2001.

[9] S.-Y. Chung, T. Richardson, and R. Urbanke, “Analysis of sum-product decoding of low-density parity-check codes using a Gaussian approximation,” IEEE Transactions on Information Theory, vol. 47, pp. 657–670, Feb. 2001.

[10] T. Tian, C. Jones, D. Villasenor, and R. Wesel, “Selective avoidance of cycles in irregular LDPC code construction,” IEEE Transactions on Communications, vol. 52, pp. 1242–1247, Aug. 2004.

[11] X.-Y. Hu, E. Eleftheriou, and D. M. Arnold, “Regular and irregular progressive edge-growth Tanner graphs,” IEEE Transactions on Information Theory, vol. 51, pp. 386–398, Jan. 2005.

[12] X.-Y. Hu, E. Eleftheriou, and D. M. Arnold, “Progressive edge-growth Tanner graphs,” in Proc. IEEE Global Telecommunications Conference 2001, vol. 2, pp. 995– 1001, Nov. 2001.

[13] D. Vukobratovic and V. Senk, “Generalized ACE constrained progressive edge- growth LDPC code design,” IEEE Communications Letters, vol. 12, pp. 32–34, Jan. 2008. 188 Paper E

[14] T. Richardson, “Error ﬂoors of LDPC codes,” in Proc. 41st Annual Allerton Conf. on Commun., Control, and Computing, vol. 41, pp. 1426–1435, Oct. 2003.

[15] K. Kasai, T. Shibuya, and K. Sakaniwa, “Detailedly represented irregular low- density parity-check codes,” IEICE Trans. on Fundamentals, vol. E86-A, pp. 2435– 2444, Oct. 2003.

[16] G. Liva and M. Chiani, “Protograph LDPC codes design based on EXIT analysis,” in Proc. IEEE GLOBECOM 2007, pp. 3250–3254, Nov. 2007.

[17] S. ten Brink, “Convergence behavior of iteratively decoded parallel concatenated codes,” IEEE Transactions on Communications, vol. 49, pp. 1727–1737, Oct. 2001.

[18] S. ten Brink, G. Kramer, and A. Ashikhmin, “Design of low-density parity-check codes for modulation and detection,” IEEE Transactions on Communications, vol. 52, pp. 670–678, April 2004.