Deriving Good LDPC Convolutional Codes from LDPC Block Codes Ali E
Total Page:16
File Type:pdf, Size:1020Kb
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 2, FEBRUARY 2011 835 Deriving Good LDPC Convolutional Codes from LDPC Block Codes Ali E. Pusane, Member, IEEE, Roxana Smarandache, Member, IEEE, Pascal O. Vontobel, Member, IEEE, and Daniel J. Costello, Jr., Life Fellow, IEEE Dedicated to the memory of our dear friend and colleague Ralf Koetter (1963–2009) Abstract—Low-density parity-check (LDPC) convolutional turbo codes and their associated decoding algorithms in 1993 codes are capable of achieving excellent performance with low en- [1]. A few years after the invention of the turbo coding coding and decoding complexity. In this paper, we discuss several schemes, researchers became aware that Gallager’s low-den- graph-cover-based methods for deriving families of time-invariant and time-varying LDPC convolutional codes from LDPC block sity parity-check (LDPC) block codes and message-passing codes and show how earlier proposed LDPC convolutional code iterative decoding, first introduced in [2], were also capable constructions can be presented within this framework. Some of of capacity-approaching performance. The analysis and design the constructed convolutional codes significantly outperform the of these coding schemes quickly attracted considerable atten- underlying LDPC block codes. We investigate some possible rea- sons for this “convolutional gain,” and we also discuss the—mostly tion in the literature, beginning with the work of Wiberg [3], moderate—decoder cost increase that is incurred by going from MacKay and Neal [4], and many others. An irregular version LDPC block to LDPC convolutional codes. of LDPC codes was first introduced by Luby et al. in [5], [6], Index Terms—Block codes, convolutional codes, low-density and analytical tools were presented in [7] and [8] to obtain parity-check (LDPC) codes, message-passing iterative decoding, performance limits for graph-based message-passing iterative pseudocodewords, pseudoweights, quasi-cyclic codes, unwrap- decoding algorithms, such as those suggested by Tanner [9]. ping, wrapping. For many classes of channels, these tools have been success- fully employed to design families of irregular LDPC codes that perform very well near capacity [10], [11]. Moreover, for I. INTRODUCTION the binary erasure channel these tools have enabled the de- N the last 15 years, the area of channel coding has been sign of families of irregular LDPC codes that are not only I revolutionized by the practical realization of capacity- capacity-approaching but in fact capacity-achieving (see [12] approaching coding schemes, initiated by the invention of and references therein). The convolutional counterparts of LDPC block codes are LDPC convolutional codes. Analogous to LDPC block codes, Manuscript received April 25, 2010; revised August 18, 2010; accepted September 21, 2010. Date of current version January 19, 2011. The work of LDPC convolutional codes are defined by sparse parity-check A. E. Pusane and D. J. Costello, Jr. was supported in part by the National matrices, which allow them to be decoded using iterative mes- Science Foundation (NSF) under Grants CCR-0205310 and CCF-0830650, and by NASA under Grant NNX09AI66G. The work of A. E. Pusane was also sage-passing algorithms. Recent studies have shown that LDPC supported by a Graduate Fellowship from the Center for Applied Mathematics, convolutional codes are suitable for practical implementation University of Notre Dame, Notre Dame, IN. The work of R. Smarandache was in a number of different communication scenarios, including supported by the NSF under Grants DMS-0708033 and CCF-0830608, and in part by the NSF under Grant CCR-0205310. The material in this paper was continuous transmission and block transmission in frames of presented in part at the IEEE International Symposium on Information Theory, arbitrary size [13]–[15]. Nice, France, June 2007. Two major methods have been proposed in the literature for This paper is part of the special issue on “Facets of Coding Theory: From Algorithms to Networks,” dedicated to the scientific legacy of Ralf Koetter. the construction of LDPC convolutional codes, two methods A. E. Pusane was with the Department of Electrical Engineering, University that in fact started the field of LDPC convolutional codes. The of Notre Dame, Notre Dame, IN 46556 USA. He is now with the Department first method was proposed by Tanner [16] (see also [17] and of Electrical and Electronics Engineering, Bogazici University, Bebek, Istanbul 34342, Turkey (e-mail: [email protected]). [18]) and exploits similarities between quasi-cyclic block codes R. Smarandache is with the Department of Mathematics and Statistics, San and time-invariant convolutional codes. The second method was Diego State University, San Diego, CA 92182 USA (e-mail: rsmarand@sci- ences.sdsu.edu). presented by Jiménez-Feltström and Zigangirov [19] and relies P. O. Vontobel is with Hewlett-Packard Laboratories, Palo Alto, CA 94304 on a matrix-based unwrapping procedure to obtain the parity- USA (e-mail: [email protected]). check matrix of a periodically time-varying convolutional code D. J. Costello, Jr. is with the Department of Electrical Engineering, University of Notre Dame, Notre Dame, IN 46556 USA (e-mail: [email protected]). from the parity-check matrix of a block code. Communicated by G. D. Forney, Jr., Associate Editor for the special issue on The aims of this paper are threefold. First, we show that "Facets of Coding Theory: From Algorithms to Networks." these two LDPC convolutional code construction methods, Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. once suitably generalized, are in fact tightly connected. We Digital Object Identifier 10.1109/TIT.2010.2095211 establish this connection with the help of so-called graph 0018-9448/$26.00 © 2011 IEEE 836 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 2, FEBRUARY 2011 . .. (1) . .. .. .. .. .. .. covers.1 A second aim is to discuss a variety of LDPC convolu- By and , we mean, respectively, a row vector tional code constructions. Although the underlying principles over of length and a row vector over of length are mathematically quite straightforward, it is important to . In the following, if is some matrix, then denotes understand how they can be applied to obtain convolutional the entry in the th row and th column of . Note that we use codes with good performance and attractive encoder and de- the convention that indices of vector entries start at (and not coder architectures. A third aim is to make progress towards at ), with a similar convention for row and column indices of a better understanding of where the observed “convolutional matrix entries. (This comment applies also to semi-infinite ma- gain” comes from, and what its costs are in terms of decoder trices, which are defined such that the row and column index sets complexity. equal .) The only exception to this convention are bi-infinite The paper is structured as follows. After some notational re- matrices, where the row and column index sets equal . Finally, marks in Section I-A, we discuss the basics of LDPC convolu- will denote the Kronecker product of the matrices tional codes in Section II. In particular, in that section, we give and . a first exposition of the LDPC convolutional code construction methods due to Tanner and due to Jiménez-Feltström and Zigan- girov. In Section III, we discuss two types of graph-cover code II. LDPC CONVOLUTIONAL CODES constructions and show how they can be used to connect the This section defines LDPC convolutional codes and discusses code construction methods due to Tanner and due to Jiménez- why they are interesting from an implementation perspective. Feltström and Zigangirov. Based on these insights, Section IV Afterwards, we review two popular methods of obtaining presents a variety of LDPC convolutional code constructions LDPC convolutional codes by unwrapping block codes. Later (along with simulation results), and in Section V, we mention in this paper, namely in Section III, we will use graph covers to some similarities and differences of these constructions com- show how these two methods are connected, and in Section IV pared to other recent code constructions in the literature. Af- we will see how these two methods can be implemented and terwards, in Section VI, we analyze some aspects of the con- combined to obtain LDPC convolutional codes with very good structed LDPC convolutional codes and discuss some possible performance. reasons for the “convolutional gain,” before we conclude the paper in Section VII. A. Definition of LDPC Convolutional Codes A. Notation A semi-infinite binary parity-check matrix as in (1), shown at the top of the page, defines a convolutional code as We use the following sets, rings, and fields: is the ring follows. Namely, it is the set of semi-infinite sequences given of integers; is the set of nonnegative integers; is the by field of size two; is the ring of polynomials with coef- ficients in and indeterminate ; and is the ring of polynomials in modulo , where is a pos- itive integer. We also use the notational shorthand for where denotes the transpose of a vector or of a matrix. We comment on several important aspects and properties of 1Note that graph covers have been used in two different ways in the LDPC the code and its parity-check matrix . code literature. On the one hand, starting with the work of Tanner [20], they • If the submatrices ,have have been used to construct Tanner graphs [9] of LDPC codes, and therefore size with , then is said to have parity-check matrices of LDPC codes. Codes constructed in this way are nowa- days often called proto-graph-based codes, following the influential work of (design) rate . Thorpe [21], who formalized this code construction approach. On the other • The parameter that appears in (1) is called the syndrome hand, starting with the work of Koetter and Vontobel [22], [23], finite graph former memory.