LINEAR SEQUENTIAL NETWORKS Concepts and Applications By: Tom Truong and Shaun Evans

Linear Sequential Networks are the enabling device for a multitude of different communication technologies. With LSN’s, communication can be made more reliable by using forward error correction techniques via block codes and convolutional codes. The realization of these codes can result in a large implementation. However, if a cyclic code is employed, then a polynomial form of the code can be used and a larger implementation can be replaced with a relatively small shift-register implementation. This is where LSN’s prove to be a very practical device in error correction and detection. When a channel needs more data integrity or an increase in data rates causes burst errors, LSN’s can be used to overcome the problem. This can be seen in wire, wireless and optical channels.

Forward Error Correction (FEC) and error detection is one area in communication design that utilizes the concept of LSN’s. FEC uses redundancy to improve the reliability of a communication channel. This increase in redundancy results in lower data transmission rates but can result in a channel that has fewer errors or a channel that can use less power because it can tolerate more errors [1]. Conversely, if the data rate is to be kept the same, the bandwidth would have to be increased to accommodate the additional redundancy bits. Forward Error Correction schemes can operate on the current data without the need to look at previously received symbols, hence always moving in a forward fashion. FEC codes can be classified into two main categories: block codes and convolutional codes. A block code can be characterized as a code that can be generated using only information from its current block of information. If the code needs information both from current and previous blocks, then the code is considered to be a convolutional code [2]. In either case, the code will generate a number of parity bits that will accompany the data. The number of parity bits in relation to the number of data bits determines the rate of the code. The number of total bits that the channel can support (n) and number of data bits (k) can be used to determine the rate (R) of the code: R = k/n. For example, if a code had just one parity bit on 4 bits of data, then it would be a 4/5 rate code. If there are no parity bits, then the code has a rate of 1.

Block codes can be constructed by applying a generator matrix to the data. This translates to using a ROM implementation to handle the table contents. But, if the code has cyclic properties, a generator matrix can be represented as a polynomial. Being able to describe a function in terms of a polynomials means that LSN’s can be used for implementation. This greatly reduces the complexity of the implementation.

There are a number of different block codes in use today. Some of the more popular ones are Hamming codes, BCH (Bose, Chaudhuri and Hocquenghem) codes, and RS (Reed Solomon) codes. These codes are popular because they are a block code with a specific property that enables them to correct more than one error [2]. They belong to a subclass of polynomial codes called cyclic codes. A cyclic code has the characteristic such that every cyclic shift of a codeword is also a codeword. A cyclic code is also divisible by xn- 1. For RS coding, the input data is divided into blocks such that groups of errors can be corrected. For RS decoding, the amount of errors that can be corrected is: tb = (kb –nb)/2 where nb is the length of the code word. Since RS codes are divided into blocks, they work especially well in correcting burst errors.

Hamming codes are another type of block codes and a simple example of them can be visualized via a Venn diagram. It can also be proven that Hamming codes are cyclic and can thus be treated as a polynomial function as well. A typical Hamming code is the [7, 4, 3] code. The first number indicates the length of the code, the second indicates the number of data bits and the third is the number of parity bits. Hamming codes employ bounded distance decoding and the [7, 4, 3] code can only correct one error or detect two. They are however, easier to implement than other codes. In Figure 1, the parity bits are p0, p1 and p2 and the message bits are m0, m1, m2 and m3. For a given message, the parity bits must be chosen such that each circle encloses an even number of 1’s . An error in any of the four message bits can be corrected using this method [1].

p 0 m0 p1

m3 m 1 m2

p2

Figure 1: Hamming Code visualization

The BCH code is a cyclic block code and RS codes are a particularly important subclass of BCH codes. Both of these codes are based on the concept of finite fields. Finite fields are sets of numbers over which all calculations are performed. The input to the calculations and their results must be numbers contained within the field [3]. BCH code can correct up to t errors: t = (d-1)/2 , where d is the minimum distance of the matrix. Minimum distance can be defined as the smallest number of bit differences between any two code words in the generator matrix. BCH codes are well understood for lengths up to several thousand [1].

Convolutional codes are frequently used in digital communications. A convolutional code is generated by passing information to be transmitted through a linear state shift register. They can be best visualized as a particular sequence moving through a trellis diagram. The previous state of the machine is in memory and coupled with the current input determines the next state. The decoder will examine this path and find the most likely encoding path through the trellis. There are two different methods to reproduce the path through the trellis. One way is called hard decision decoding and the other is soft decision decoding. Soft decision decoding takes more effort to implement but can yield a better bit error rate. There exists different methods of decoding the trellis path but one of the most effective is the Viterbi algorithm. The Viterbi algorithm chooses the vector through the trellis that is closest to the received vector in terms of minimum Hamming distance. This is achieved by performing a recursive calculation.

Another version of a convolutional code is a turbo code. Two encoders work in parallel and their outputs are interleaved together. This encoding style produces very good properties in the resulting code. The Hamming distance between code words is large and the interleaving introduces a time diversity into the system. This is a good property when encountering fading channels. Errors can be spread out instead of happening in bursts.

Lets take a look at one of these codes as they are used in practice. In particular, lets discuss how block codes are used to generate and decode parity bits referred to as Cyclic Redundancy Check (CRC) bits. CRC is often used in communication links to overcome errors generated by noise in the channel. The encoder for an (n,k) block code is shown below in figure 2.. Assuming at time zero, all of the registers are set to zero and data is allowed to shift both into the circuit and also directly to the output. As the data is shifted into the circuit it is divided by g(x), the generator polynomial. At this time, the switch is allowing the input data to feed directly to the output. After all of the data has completely shifted into the registers (k clock cycles), the circuit stops accepting new data, flips the output switch and start shifting out the parity data, p(x). The resulting output has both the original data and the parity bits. Together these are called a codeword [2].

+ D Q D Q D Q D Q + D Q D Q + D Q D Q

Figure 2: Block encoder for g(x) = x8 + x7 + x6 + x4 + 1

The error detecting circuit works in a similar fashion as shown in figure 3. The received data is shifted in and again divided by the same g(x). After all of the data is shifted in the registers contain the syndrome coefficients. If the data decoded is uncorrupted (i.e. a codeword) then the syndrome will be all zeros. If there is a non zero number in the syndrome, then errors are present and the OR gate will flag the error. Note that both the data and the parity bits are used in the error detection circuit. This algorithm will be able to correctly detect n-k errors.

+ D Q D Q D Q D Q + D Q D Q + D Q D Q 8

Figure 3: Error detection circuit using g(x) = x8 + x7 + x6 + x4 + 1 Automatic repeat request (ARQ) schemes are another way to increase the reliability of a communication channel. If the data received contains errors, the CRC will generate a non-zero syndrome detecting the error and then the circuit will send a repeat request back to the transmitter via a feedback channel. This can provide high quality data but the rate of it unknown due to the unknown number of retries necessary. This method introduces delay into the system because the data needs to be buffered while a retry is sent, received and then put in the buffer in its proper place. Figure 3 shows the transmitter sequence on top and the receiver sequence on bottom. The acknowledge (ACK) and not acknowledge (NAK) are signals on the feedback channel and indicate whether or not the data received needs to be resent. Notice how transmission number three is flagged with a NAK and then resent in between messages 7 and 8. The receiver will need to buffer incoming messages and insert resent messages into their proper location [4].

1 2 3 4 5 6 7 3 8 9 10 7 11 12 13

K K K K K K K K K K C C A C C C C A C C A A N A A A A N A A

1 2 3 4 5 6 7 3 8 9 10 7 11

Figure 4: ARC Technique

Another coding technique that we will examine closer is the Reed-Solomon code. Reed- Solomon is a powerful technique that is pervasively used to correct corrupted data in a noisy channel. The technique is very effective in the case where the error is of burst type. Reed-Solomon code is unique because it corrects errors on a symbol basis where a symbol is defined in Galois Finite Field. Galois Finite Field is typically chosen as GF(2q) where q is the resulting symbol length of q binary bits. The finite field is implemented using Linear Shift Feedback Network (LSFN) based on a primitive and irreducible polynomial. In the industry, the high order of the polynomial is chosen as eight to obtain octal format.

Another property that makes Reed-Solomon code a preferable choice for many applications to correct burst errors in noisy channel is its high code rate where the rate is the ratio of information message length and code length. This essentially means that the redundancy overhead is minimum with respect to other linear block codes. Commonly, Reed-Solomon code is defined as RS(n, k, t) where n is the code length, k is the information message, and t is error correcting capability. Depending on application, these parameters can vary to accommodate any required error-correcting specification.

Reed-Solomon code fits into communication model perfectly as shown in Figure 5. A typical communication model involves transmitter, channel, and receiver [5]. The channel can be any medium such as wire, wireless, as well as optical. In most of today channels, there are interferences; these interferences degrade signal quality and cause the receiver to interpret user information incorrectly. Fortunately, the channel interference can be modeled and eliminated by Reed-Solomon code. On the transmitter side, before the user information is sent through the channel, it is encoded by an RS-encoder; on the receiver side, the encoded message or codeword passes through an RS-decoder and the original message will be recovered correctly. The RS-encoder and RS-decoder are based on the famous Reed-Solomon generating polynomial, defined below [6].

2t 1  i  G(x)    x  i  0 

Clearly, there are 2t consecutive roots, and alpha symbol defines in Galois Field.

x_t(n) Signal/Data RS Modulation Processing Encoder Transmitter Interference Channel

x_r(n) Signal/Data RS De- Processing Decoder Modulation Receiver

Figure 5: Typical Communication Model The RS-encoder function takes k input message symbols and produces n output codeword symbols. Let’s define the input message as m(x) where its length is k, the output codeword as t(x) where its length is n, and the parity redundancy as p(x) where its length is 2t. P(x) is obtained from the remainder of m(x) divided by G(x) which is the RS- generating polynomial. Once p(x) is found, t(x) is simply the concatenation of m(x) and p(x) to form systematic code. The systematic codeword can then be modulated and sent to the channel. The RS-encoding process is shown in figure 6. It’s important to note that the implementation and concept for the polynomial division to get the p(x) remainder is similar to a Linear Shift Network except that it’s operated over Galois Finite Field.

m(x) m(x)/G(x) p(x) {m(x),p(x)} t(x)

Figure 6: RS-encoding In comparison to the RS-encoder, the RS-decoder is a much more interesting problem because it involves various functions that lead to error-free data recovery. The first function is error detection that involves the syndrome calculation. The second function involves the derivation to key equations specifically the error locator polynomial as well as the error magnitude polynomial. The third and forth functions use the key equations to obtain exact values for error location and magnitude. Once the error location and magnitude are found, the correct decoded data d(x) is simply an XOR function of the received codeword and error polynomial e(x) which also has length n. The implementations of these functions are not only in pipeline architecture, but they also require efficient algorithms. Specifically, there are three main algorithms in RS-decoding; they are Berlekamp-Massey, Chien, and Forney algorithms. Each of these algorithms has a specific role in correcting the error from the received codeword. Berlekamp-Massey algorithm takes the syndrome values from the syndrome calculator and generates key equations; the Chien algorithm takes Berlekamp-Massey key equations and produces an error location for the error polynomial; finally, Forney algorithm takes Berlekamp- Massey equations and produces the error magnitude for the error polynomial [7]. The RS- decoding process is shown in figure 7.

Berlekamp- Chien & Forney r(x) Syndrome Massey Key Error Locator & e(x) Calculator Equation Magnitude

d(x) Pipelines xor

Figure 7: RS-decoding

The pipeline needs to correct data on the fly. The pipeline delay is the latency of the decoding algorithm.

One of many applications of Reed-Solomon forward error correction is optical communication. Optical communication is the backbone of today’s data and voice networks; most data centers, as shown in figure 8, that provide broadband services, one way or another, are connected to an optical network to deliver data and voice packets across the globe. As the price of broadband services drops, many people will have high- speed connections to their homes. This, of course, will increase the demand for bandwidth in the optical communication backbone and also creates many challenges for network design engineers to come up with a cost effective solution to meet the demand. Many networking companies utilize Dense Wave Division Multiplexing (DWDM) technology to overcome these challenges. Traditional implementation prior to DWDM is that each of fiber optic lines can only carry one laser beam; the break-through in DWDM allows multiple laser beams at different wavelength to share the same fiber optic lines, and networking companies can avoid putting in new fiber optic lines that can be very expensive [8]. In addition, the data capacity for each laser beam, today, can carry up to 40 Gbps. Though DWDM solves many bandwidth problems and saves significant amount of money by reusing current available optical lines, it creates a new set of problems specifically relating to data integrity.

User 1…n User 1…n User 1…n User 1…n

Data Data Data Data Centers Centers Centers Centers

Node-A Node-B Node-C Node-D

Figure 8. Data center connect to Optical backbone

Optical data transport is affected mainly by attenuation and dispersion. Optical attenuation is caused by the degradation of optical energy as the optical signal travels through optical fiber. Dispersion, specifically in DWDM, is caused by peak energy of multiple wavelengths on the same optical fiber. These problems, result in Inter-Symbol- Interference (ISI), can be thought of as optical noise, increase the Bit Error Rate (BER) of optical transmission, and decrease data integrity. To maintain the quality of optical transmission within an acceptable BER, Reed-Solomon FEC is introduced. The FEC improves the data integrity by detecting and correcting errors in the optical channel that are caused by optical noise.

Solving data integrity problems for optical communication by implementing Reed- Solomon FEC is an ideal solution. The FEC resolves the optical attenuation and dispersion in one shot. Typically, an optical signal, that carries user information arrives to a node, will be converted to an electrical signal, get processed, be converted back to an optical signal, and send to another node until it reaches its destination; the idea is demonstrated in figure 9 below. Therefore, regarding attenuation, networking companies try to save cost by reducing number of optical network nodes; this, in turn, means the distance between any two nodes increases. Increasing the distance between any two nodes causes optical signal power to degrade when it reaches its destination node. Using FEC in each node, the signal will have power gain. In other words, the FEC increases signal to noise ratio (SNR) for the optical signal, and this extra signal gain by using FEC code can be traded for longer distances between network nodes. FEC, in this case, resolves the optical attenuation problem nicely.

Optical Signal O/E RS O/E Optical Signal In FEC Out

Figure 9. Optical Node with FEC The second problem is optical dispersion. This only occurs with optical networks that have DWDM because a fiber optic line has limits on its peak energy. When multiple laser beams are transmitted over a fiber optic line simultaneously, there’s a probability that the aggregate energy of each beam sometimes exceeds the fiber optic limit. For example, let assume that a fiber optic line had 100 db peak energy; suppose 100 beams are shared in the same fiber optic line; if the encoded optical signal of a high bit is 2 db, then whenever there are more than 50 beams out of 100 that have consecutive high bit switches simultaneously, the fiber line will have a dispersion problem. In other words, the user information encoded in the optical signal will have a burst error behavior. One solution is to reduce the number of beams to keep them a below specified fiber peak energy; this essentially increases the wavelength spacing between different laser beams. However, to maximize bandwidth usage, network engineers want to transmit as many beams across the fiber as possible; this means as long as the dispersion errors are within the error correcting capability of the FEC, the number of beams are optimal. As expected, at the destination node, the user data is corrupted. Since, the behavior of the error is of burst type, the FEC is an excellent candidate to recover the user information from the received corrupted data.

Clearly, Reed-Solomon FEC is extremely useful in optical communication applications. In this case, it’s not only used to main data integrity for the network, but it also indirectly is used to trade a increase in demand for bandwidth.

Linear Sequential Networks prove to be useful in wide variety of communications applications. We have seen here how they are used to produce Cyclic Redundancy Checks in mobile communications and how they are used with Reed-Solomon codes for Forward Error Correction in optical DWDM networks. These are only some examples of the many practical applications involving error correction and LSN’s. These examples hopefully demonstrate the value of these circuits in communication systems.

References:

[1] P. Kumar, Error-Correcting Codes, Unpublished, University of Southern California, 2002. [2] H. Wesolowski, Mobile Communication Systems, Wiley & Sons, Chichester, 2002. [3] A.D Houghton, The Engineer’s Error Correcting Handbook, Chapman & Hall, London, 1997. [4] F. Halsall, Data Communications, Computer Networks and Open Systems, Addison- Wesley, Harlow, 1996. [5] M.H Khan, Y.R. Shayan, T. Le-Ngoc and V.K. Bhargava, Forward Error Control Coding for Mobile Communication Systems, IEEE Micro, 1988. [6] E.M. Popovici and P. Fitzpatrick, Reed-Solomon Codecs for Optical Communications, IEEE Micro, vol. 2, May, 2002. [7] T.K. Truong, J.H. Jeng, and I.S. Reed, Fast Algorithm for Computing the Roots of Error Locator Polynomials up to Degree 11 in Reed-Solomon Decoders, IEEE [8] Draft ITU-T G.709, 02/2001. Transactions on Communications, vol. 49, May 2001. [9] ITU-T G.975, Forward Error Correction for Submarine Systems, 11/96 [10]Hideki Sawaguchi, Seiichi Mita, and Jack K. Wolf, A Concatenated Coding Technique for Partial Response Channels, IEEE Transaction on Magnetics, vol. 37, NO. 2, March 2001.