CONVOLUTIONAL CODES IN VSS . .. By: Dr. Kurt R. Matis Director of Systems Research SIMULATION OF CONVOLUTIONAL CODES IN VSS This document describes the modeling and simulation of short constraint- length convolutional codes used in conjunction with Viterbi decoding in the Visual System Simulator (VSS). After a brief review of the history and application of convolutional codes, a detailed description of VSS models for encoding/decoding of these codes is presented. Step-by-step examples illustrate how to construct simulations and analyze results. Convolutional Code Basics This note begins with some background information on the use of convolutional codes. The development of convolutional codes is discussed along with a history of important applications. This information is meant to provide a perspective on the selection of the convolutional code models that are provided in VSS. Transmission efficiency and reliability can be improved by encoding information digits in a way that creates an interdependence between symbols which are transmitted over a channel. At the receiving end, the interdependence can be exploited to detect or even correct transmission errors, provided erroneous symbols are not received too frequently. Such coding is called error-control coding and is shown in the configuration of Figure 1. Received Source Encoded Symbols Decoded Symbols Symbols ˆ Symbols {bk } {ai} Channel {bk} Transmitter s(t) Channel r(t) Receiver Channel {âi} Encoder Decoder {rk} Figure 1. System Employing Error-Control Coding Visual System Simulator 1 CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS Encoders for error control are usually called channel encoders to differentiate them from various encoders used for other purposes within digital communication systems. The process of error-control coding is also referred to variously as “Error Detection And Correction” (EDAC) and “Cyclic Redundancy Check” (CRC), although CRC actually refers to a specialized type of code. Coding used for error detection is often called, as one might expect, error detection coding. It is often used within Automatic Request for Query (ARQ) systems to detect a block or packet error and request the transmitter to retransmit a message. Again, intuitively, when error-control coding is used for the purpose of actually correcting certain received symbols at the decoder it is usually called error correcting coding or error correction coding. In some cases it is called “Forward Error Correction” (FEC) coding to distinguish it from the process used in conjunction with the aforementioned ARQ error-correction scheme. Finally, error control codes are sometimes used for both correction of a limited number of errors and detection of a certain number of errors when there are too many errors to correct. As shown in Figure 1, the encoder inserts redundant symbols into the original symbol sequence, {}ai , to create the output sequence, {}bk , at a rate1 ⁄ R times the rate of the original transmitted sequence {}ai . The quantity R is referred to as the code rate. Since the symbols, {}bk , are no longer independent, the decoder deals with vectors of received symbols,r , which may potentially be very long, perhaps encompassing the entire transmission. From a practical point of view, error control codes impart a dependence to the symbols within the transmitted data stream that is localized from within a few symbols to a few hundred symbols. This means that the decoder need not examine the entire transmission before making decisions about the transmitted symbols, but can decode information symbols by examining a subsequence of symbols, b, which is "near" the original information symbol sequence a. We will quantify these notions when discussing the specific classes of codes. Whatever the length of the sequence, a, the decoder must produce an estimate, â, based on the received sequence, r. Equivalently, since there is a one-to-one correspondence between the information sequence a and the code word, b, the decoder can produce an estimate of the code word, b, and then map this sequence back into the original information sequence, a. Clearly, â = a if and only if bˆ = b . 2 Visual System Simulator CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS Convolutional Codes Compared with Block Codes The two classes of error control codes are block codes and convolutional codes. An (N, K) block encoder maps blocks of K source bits into blocks of N encoded bits. There is complete independence between blocks. N is called the block length of the code and R = K / N is the code rate. In contrast to block codes, which impose a block structure on the input information sequence, convolutional codes impose a sliding dependency over a span of input symbols. The encoder can be implemented as a shift register into which information bits are shifted 'b' at a time. After each shift, n modulo-2 sums are computed from the contents of selected shift register stages. These n modulo-2 sums then constitute the encoder output subsequence associated with a particular subsequence of b-input information bits. The normalized code rate in this case is R = b / n. This indicates the number of bits, on average, that are carried by each symbol output from the encoder.1 The quantity k is called the constraint length of the code and is equal to the length of the encoding shift register. A shift register of length k has k-1 delay or storage elements, each storing one bit. A typical k = 3, R = 1/2 encoder is shown in Figure 2. 1. Unfortunately, the classical literature describing convolutional codes has used some of the same nomencla- ture historically used to represent quite distinct attributes of block codes. For example, the capital letter “N” represents the length of a block code word, whereas the small letter “n” represents the number of adders in a convolutional encoder. Application Notes 3 CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS Figure 2. Convolutional Encoder for k = 7, R = 1/2 Convolutional Code Note that the terminology concerning convolutional codes is not uniform in the field. We generally try to follow the usage in Viterbi and Omura [1], which is consistent with Figure 2. In this note, we restrict attention to linear, feedforward convolutional codes which encode a binary input stream into a binary output stream. This is the encoding model depicted in Figure 2. Other VSS models address the functionality of feedback codes, nonlinear codes and codes designed for use with more general alphabets. The number of storage elements in this case is two - always one less than the constraint length, k. Depending on the connections to the mod-2 adders (described by the so-called “code connection vectors”), the code will have different properties as discussed below. The convolutional encoder can be thought of as a finite-state machine with a number of states corresponding to all possibilities of bit values contained within the storage elements of the shift register. With any given state and any specified input (a single “0” or “1” in this case), both the next state and the output of the encoder can be predicted. This model of the encoder as a finite-state machine is embodied in the FSM (finite-state machine) diagram at the right of the figure. The FSM provides a succinct description of all possible evolutions of the sequence of states as successive encodings are performed. Tabulations of good convolutional code constructions are available for various values of k and R [2], [3], [4]. For any given value of k, n and b, the "code construction" procedure consists of finding connections of the modulo-2 adders shown in Figure 2 to the appropriate shift-register stages 4 Visual System Simulator CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS to generate a sequence with the maximum error-correcting capability. VSS contains extensive tabulations of optimum convolutional codes. The evolution in time of the possible sequence of encoder states can be described in terms of a trellis diagram.Figure 3 illustrates a trellis diagram for the code of Figure 2. Generated bits are determined from a combination of the current input bit sequence of b bits plus the "state" of the encoder, which consists of the most recent k - b bits in the shift register. b For the k = 3, R = 1/2 code, there are four possible states with 2 = 2 branches into and out of each state. Each branch has n output symbols associated with it, in this case, 2. 00 0000 a 00 a 00 a = 00 11 11 11 11 11 bb b = 01 11 11 11 10 10 10 10 01 0100 01 00 01 00 cc c = 10 d = 11 01dd 01 01 10 10 10 Figure 3. Trellis-Code Representation for Encoder of Figure 2 Convolutional codes were first introduced by Elias in 1955 [5]. In 1961, Wozencraft proposed a probabilistic method for decoding convolutional codes called sequential decoding [6]. In 1963, Massey proposed a less efficient but simple decoding algorithm called "threshold decoding" [7]. Finally, in 1967, Viterbi [8] proposed a maximum-likelihood decoding algorithm that still bears his name. The Viterbi decoding algorithm is relatively easy to implement for codes with short constraint lengths and allows effective use of soft-decision Application Notes 5 CONVOLUTIONAL CODES IN VSS Simulation of Convolutional Codes in VSS information in the form of real-valued or quantized receiver outputs. Although other forms of decoding are still used widely in certain applications, Viterbi decoding has become by far the most widely-used decoding method for convolutional codes. The Viterbi algorithm is an iterative algorithm for correlating received sequences with all possible transmitted sequences in the trellis to arrive at the one with the best fit.