Lecture 8: Local Codes Sivakanth Gopi October 21, 2019 In this part of the course, we will talk about the notion of “local- ity”. We can construct codes with good rate and distance which are encodable and decodable in near-linear time (e.g., Expander codes). And over large alphabets, we can achieve optimal rate-distance trade- off with near-linear time encoding and decoding (e.g., Reed-Solomon codes). We will also see later in the course that we can now con- struct optimal binary codes for many random noise channels with near-linear time encoding and decoding (e.g., Polar codes). This is amazing progress. Can we do even better? In many scenarios, both theoretical and practical, we will see that we only need to decode a small part of the message or correct a small part of the corrupted encoding. For example in distributed storage, to recover a single crashed server, we shouldn’t be reading all the rest of the servers. In such cases, linear time is not good enough, we need to do things in sublinear time or perhaps even in constant time! Codes which support sublinear algorithms for tasks like decoding, correction or testing are called “local codes”. Codes with locality Let C : Sk ! Sn be some code. Given some string z which is close to some codeword C(x) in Hamming distance, there are three natural algorithmic tasks we would like to do: • Correction: Correct the errors in z to get C(x) • Decoding: Decode z to get x • Testing: Test whether z is actually close to some codeword. Local codes allow us to do these tasks in sublinear time. We will define locally decodable codes formally and give informal definitions for locally correctable codes and locally testable codes. A q-query locally decodable code (LDC) allows us to decode any message bit xi with high probability by reading at most q locations of z. We will define LDCs formally now. Definition 1 (Locally decodable code (LDC) [KT00]). A code C : Sk ! Sn is a (q, d, h)-LDC if, for every i 2 [k], there exists a randomized decoder (a probabilistic algorithm) Ai such that: lecture 8: local codes 2 • For every message x 2 Sk and z 2 Sn such that D(C(x), z) ≤ dn, 1 Pr[A (z) = x ] ≥ + h.(1) i i 2 • The decoder Ai queries non-adaptively at most q coordinates of z. Locally Correctable Code: C is called a (q, d, h)-LCC if there is a randomized local correction algorithm which given some i 2 [n], reads at most q locations of z, which is d-close to C(x), and outputs C(x)i with good probability at least 1/2 + h. Any LCC can be converted into an LDC Locally Testable Code: C is called a q-query LTC if there is a ran- while preserving relevant parameters. domized local testing algorithm which reads at most q locations of z In fact any linear code can be made systematic i.e. the message is part of the and accepts if there are no corruptions, and rejects with good proba- encoding. Therefore any linear LCC is bility if there are too many corruptions. also an LDC trivially. Hadamard Code To get familiar with the definitions, let us look at the example of Hadamard code which is simultaneously a 2-query LDC, 2-query LCC and 3-query LTC! The local correctability and testability of The Hadamard code is a exponential length linear code, H : Fk ! Hadamard code plays an important role 2 in the proof of the PCP theorem. n k k F2 where n = 2 . The codeword coordinates are indexed by y 2 F2 k and for a message x 2 F2, the encoding is given by H(x)y = hx, yi k i.e. the codewords are just evaluations of linear functions on F2. It is not hard to see that it is a linear code and the minimum distance of the code is n/2, in fact every non-zero codeword has weight exactly n/2. We will now prove that the Hadamard is a 2-query LCC. We can achieve local correction way up to half-minimum-distance. Lemma 2. For every d 2 (0, 1/4), Hadamard code is a (2, d, 1/2 − 2d)- LCC. Proof. Suppose we are given a corrupted version of a codeword H(x), say H](x) s.t. D(H(x), H](x)) ≤ dn. To correct the symbol at k ] y 2 F2, the local corrector queries H(x) at z, z + y for a uniformly k random z 2 F2 and computes the parity of the two bits. With proba- bility at least 1 − 2d, both the queries land in the uncorrupted part of H](x) , and if that’s the case, H](x)z + H](x)z+y = hx, zi + hx, z + yi = hx, yi which is the correct symbol. Since the message symbols are part of the codeword (H(x)ei = hx, eii = xi), it is also a 2-query LDC. To test if some given word is close to some codeword is equivalent k to testing if a function f : F2 ! F2 is close to being linear. To test lecture 8: local codes 3 k this, a local tester can sample z, y 2 F2 and query f at z, y, z + y and accept if f (z + y) = f (z) + f (y). This is the famous linearity test of Blum, Luby and Rubinfeld. It clearly accepts a linear function and it will reject a function which is far from linear with good probability (which requires a proof, but we omit it here). Thus H is a 3-query LTC. Reed-Muller Codes We have seen that the Hadamard code, which is obtained by eval- k uating linear functions at all points of Fq has good local correction, decoding and testing properties. But it is exponentially long. It is nat- k ural to try evaluating all low-degree polynomials over Fq. Since there are more low-degree polynomials, the rate of our codes will improve. These are precisely the Reed-Muller Codes. Let a = (a1, a2,..., am) be an m-tuple of non-negative integers and m let jaj = ∑i=1 ai denote the sum of its entries. Let x = (x1, x2,..., xm) a1 a2 am be a tuple of variables. We will denote the monomial x1 x2 ... xm by a a x . Note that the degree of x is jaj. An m-variate polynomial over Fq is given by a f (x1, x2,..., xm) = ∑ cax a where ca 2 Fq are some coefficients. The set of all such polynomials is denoted by Fq[x1,..., xm]. The (total) degree of the polynomial f is defined as deg( f ) = maxfjaj : ca 6= 0g. The set of all degree such ≤d polynomials of degree at most d is denoted by Fq [x1,..., xm]. We will also use deg ( f ) to denote the maximum degree of x variable xi i in f , i.e., deg ( f ) = maxfa : c 6= 0g. Let P (m, d) denote the set xi i a q ≤d of all polynomials in Fq [x1,..., xm] with individual degrees of each variable at most q − 1, i.e., P (m, d) = f f 2 F≤d[x ,..., x ] : 8i deg ( f ) ≤ q − 1g. q q 1 m xi ≤d When q > d, Pq(m, d) = Fq [x1,..., xm]. For example, P2(m, d) is the set of all m-variate “multilinear” polynomials of degree at most d. The restriction that deg ( f ) ≤ q − 1 xi q is because of the fact that x = x ( ) i i Fact 3. There is a bijection between Pq m, d and evaluations of polyno- for all xi 2 Fq. Therefore, if we only ≤d m care about evaluations over F , we can mials in Fq [x1,..., xm] over all points of Fq . The bijection is given by q reduce the degree of each variable to at f 2 (m d) ! h f (a)i m In particular, two distinct polynomials in Pq , a2Fq . most q − 1. m Pq(m, d) cannot have the same evaluation over all points of Fq . Definition 4. A degree-d Reed-Muller code, denoted by RMq(m, d), is the m set of evaluations of polynomials Pq(m, d) over all points in Fq . Formally, n o ( ) = h ( )i m 2 ( ) RMq m, d f a a2Fq : f Pq m, d . lecture 8: local codes 4 Note that Reed-Muller codes are linear codes, because an Fq- linear combination of polynomials in Pq(m, d) is also in Pq(m, d). Is RM2(m, 1) the same the Hadamard We will now try to understand the rate and distance of Reed-Muller code? Almost, think about it! n m codes. RMq(m, d) is a subspace of Fq for n = q . The dimension of RMq(m, d) is: dim(RMq(m, d)) = dim(Pq(m, d)) = jfa : jaj ≤ d, 0 ≤ ai ≤ q − 1gj . There is no simple closed form expression for dim(RMq(m, d)). For d m m q = 2, dim(RM2(m, d)) = ∑r=0 ( r ) = (≤d). To simplify presentation, from now onwards, we will assume that q > d unless otherwise specified. In this case, dim(RMq(m, d)) = m+d d ( d ) &d m . We will now show that RMq(m, d) also have good dis- tance. ≤d Lemma 5 (Schwartz-Zippel). Suppose q > d and let f 2 Fq [x1,..., xm] be a non-zero polynomial.