Property testing on boolean functions Thesis proposal

Eric Blais Computer Science Department Carnegie Mellon University Pittsburgh, PA [email protected]

May 20, 2010

Abstract

Property testing deals with the following general question: given query access to some combinatorial object, what properties of the ob- ject can one test with only a small number of queries? In this thesis, we will study property testing on boolean functions. Specifically, we will focus on two basic questions in the field: Can we characterize the set of properties on boolean functions that can be tested with a constant number of queries? And can we determine the exact number of queries needed to test some fundamental properties of boolean functions?

1 Contents

1 Introduction 3

2 Definitions and notation 4 2.1 Boolean functions ...... 4 2.2 Property testing ...... 5

3 Characterization of testable properties 5 3.1 Related work ...... 6 3.2 Testing function isomorphism ...... 7 3.3 Completed work on testing function isomorphism ...... 7 3.4 Proposed research ...... 8

4 Exact query complexity bounds 10 4.1 Related work ...... 10 4.2 Completed work on testing juntas ...... 11 4.3 Proposed research ...... 11

5 Other proposed research 12 5.1 Alternative models of property testing ...... 13 5.2 Application of property testing ideas to other domains . . . . 14

6 Suggested timeline 14

2 1 Introduction

This thesis is primarily concerned with the study of boolean functions. Boolean functions play a central role in many areas of computer science: complexity theory [44], machine learning [19], coding theory [43], cryptog- raphy [18], circuit design [34], data structures [30], and combinatorics [16]. Boolean functions have also become an important subject in the field of property testing; it is this last topic that I propose to study in this thesis. Property testing deals with the following general question: for which properties of combinatorial objects (such as boolean functions) can we dis- tinguish between objects that have the property and objects that are “far” from having the property while making only a small number of local queries to the object? The field of property testing is very young, but it has already generated a large body of research (see, e.g., the recent surveys [24, 36, 37]) and has led to notable insights into sub-linear time algorithms [38] and learning theory [36]. Much of the research in property testing has focused on characterizing the set of necessary and sufficient conditions for a property to be testable with only a constant number of queries.1 This line of research has been very successful in characterizing testable graph properties and algebraic prop- erties, but there has been comparatively little progress in characterizing testable properties of boolean functions. The first goal of this thesis is to make some progress in characterizing testable properties of boolean functions. I aim to do so through the study of function isomorphism properties. These properties, their relation to the characterization project, and my plan for this part of the thesis are presented in Section 3. The second goal of this thesis is to go beyond the characterization project and obtain precise bounds on the number of queries required to test certain fundamental properties of boolean functions. Some of this research has already been carried out, and has resulted in nearly tight bounds on the query complexity required to test juntas. These results and other properties that I plan to study as part of this goal are presented in Section 4. The above two goals form the principal line of inquiry that I plan to pursue in the completion of this thesis. There are, however, many other

1I.e., a number of queries that is independent of the size of the object. Precise defini- tions are for all the concepts discussed in this introduction are presented in Section 2.

3 interesting questions related to property testing on boolean functions. Sec- tion 5 contains a brief description of some of the questions that I may also pursue in the course of completing this thesis.

2 Definitions and notation

This section introduces the definitions and notation that we use in the rest of the document. All our definitions and notation choices are standard; we include them here for completeness.

2.1 Boolean functions

A boolean function f : {0, 1}n → {0, 1} is a function that takes in n boolean variables or, equivalently, an n-dimensional boolean vector, and outputs a boolean value. We write Bn to represent the set of boolean functions n S n {0, 1} → {0, 1}, and let B = n≥1 B . A coordinate i ∈ [n] is relevant to the function f : {0, 1}n → {0, 1} if there are is an input x ∈ {0, 1}n such that f(x) 6= f(x(i)), where x(i) ∈ {0, 1}n is the input obtained by flipping the value of the ith coordinate of x. The influence of i in f is the probability that f(x) 6= f(x(i)) when x is chosen uniformly at random from {0, 1}n; the coordinate i is relevant iff it has nonzero influence. A function that has at most k relevant coordinates is called a k-junta. For any function f : {0, 1}n → {0, 1} and permutation π on [n], we π n π let f : {0, 1} → {0, 1} be the function defined by f (x1, . . . , xn) = n f(xπ(1), . . . , xπ(n)) for every input x = (x1, . . . , xn) ∈ {0, 1} . When a func- tion g : {0, 1}n → {0, 1} satisfies g = f π for some permutation π, we say that f and g are isomorphic. The distance between two boolean functions f, g : {0, 1}n → {0, 1} is the probability that they disagree on an input chosen uniformly at random from {0, 1}n: ∆(f, g) := Pr [f(x) 6= g(x)]. x∈{0,1}n

4 2.2 Property testing

A property of boolean functions is a subset P ⊆ B of all boolean functions that is closed under permutation of the coordinates. That is, if P is a property of boolean functions and f : {0, 1}n → {0, 1} is in P, then for every permutation π on [n] the function f π is also in P. The distance between a function f : {0, 1}n → {0, 1} and a property P is the minimum distance between f and any function in P ∩ Bn:

∆(f, P) := min ∆(f, g). g∈P∩Bn

For any  > 0, when ∆(f, P) ≥  we say that f is -far from P; when ∆(f, P) ≤  we say that f is -close to P. The central notion in property testing is, not surprisingly, that of a tester:

Definition 1 (Tester). Fix q : N → N and  > 0. A randomized algorithm A with oracle access to an unknown function f : {0, 1}n → {0, 1} is a (q, )- tester for the property P if it queries f on at most q(n) inputs and 2 • Accepts f with probability at least 3 when f ∈ P; and 2 • Rejects f with probability at least 3 when f is -far from P.

A tester that accepts functions in P with probability 1 has one-sided er- ror. A tester that fixes all of its queries in advance is non-adaptive; otherwise it is adaptive. The query complexity of the task of -testing the property P is the func- tion QP, : N → N where QP,(n) is the minimal value of q(n) for which there is a (q, )-tester for P. A property P is efficiently testable, or simply testable, when for every  > 0, there exists a constant c = c() such that QP, is bounded above by c. When this is not the case, P is considered not (efficiently) testable.

3 Characterization of testable properties

In this section, we discuss the first goal mentioned in the introduction: to make some progress on the project of characterizing the testable properties of boolean functions.

5 We begin the section with a survey of related work. We then introduce the function isomorphism testing problem and review the results that I have recently obtained on this problem. Finally, we discuss the ways in which I plan to apply the results on testing function isomorphism to obtain new characterizations of testable properties of boolean functions.

3.1 Related work

The problem of characterizing the set of testable properties was first raised by Goldreich, Goldwasser, and Ron [27] in the context of testing graph properties. The problem has been largely solved in this context [2, 3, 4, 6], and much progress has also been made on the characterization of testable algebraic properties [12, 29, 40]. (See also [41] and the references therein.) The progress on characterizing testable properties on boolean functions is more limited. Many properties of boolean functions have been studied: monotonicity [21, 23, 26], being a dictator function [11, 35] or a junta [17, 22], being representable by a conjunction or a DNF with few terms [35], being a halfspace [32, 33], having a low-dimensional or sparse Fourier representa- tion [28], etc. But these results remain largely isolated by the specificity of the proof techniques that are used. A more general result was obtained by Diakonikolas et al. [20] who showed that many properties related to the concise representation of boolean functions (e.g., being representable by a DNF with few terms, by a small decision list, by a small decision tree, or by a small boolean circuit) are efficiently testable using the method of “testing by implicit learning”. Their result implies the following partial characterization of testable properties on boolean functions.

Theorem 1 (Diakonikolas et al. [20]). Let P be a property on boolean func- tions for which there exists a constant k = k() such that for every f ∈ P, f is -close to a k-junta. Then P is efficiently testable.

As far as I know, this result is the strongest known characterization on the testability of properties on boolean functions. In the next sections, we explore a new approach for obtaining other characterization results.

6 3.2 Testing function isomorphism

For a function g : {0, 1}n → {0, 1}, the g-isomorphism property is the set of functions that are identical to g up to permutation of the coordinates. The family of function isomorphism properties is notable for a couple important reasons. First, these properties are the smallest properties on boolean functions (recall that properties on boolean functions are closed under permutation of the coordinates, so each property contains the iso- morphism class of at least one function). The second, and most important, reason for which the family of func- tion isomorphism properties is notable is that the complexity of testing g- isomorphism varies significantly depending on the choice of g. For instance, when g is any fully symmetric function (such as the Majority function), then the class of functions isomorphic to g contains a single function, and so testing g-isomorphism reduces to the problem of testing identity, which trivially requires O(1/) queries. By contrast, we will see below that when we choose g uniformly at random from all functions {0, 1}n → {0, 1}, then with high probability testing g-isomorphism non-adaptively requires Ω(n) queries. The variety of query complexities in the function isomorphism testing problems suggests the following open problem.

Open Problem 1. Characterize the set of functions g : {0, 1}n → {0, 1} for which testing g-isomorphism can be done with a query complexity that is independent of n.2

In the next section, I review some of the results that have already been obtained in the study of this open problem. Section 3.4 discusses the ways in which I plan to take those results to obtain new characterizations of testable properties on boolean functions.

3.3 Completed work on testing function isomorphism

The problem of testing function isomorphism was first introduced by Fischer et al. [22], who showed that testing isomorphism to juntas can be done with few queries:

2Formally, the statement should apply to families of boolean functions ˘gn : {0, 1}n → {0, 1}¯ , but we will not deal with such technicalities in this thesis proposal. n≥1

7 Theorem 2 (Fischer et al. [22]). Let g : {0, 1}n → {0, 1} be a k-junta and fix  > 0. Then testing g-isomorphism requires at most poly(k, ) queries.

Fischer et al. [22] also gave the first lower bound for a function iso- morphism testing problem: they showed that non-adaptive algorithms for √ √ testing isomorphism to the k-parity function, for k ≤ o( n), requires Ω(˜ k) queries. Ryan O’Donnell and I significantly generalized this lower bound to show that for every “strong” k-junta g, the number of queries required to test g-isomorphism must be a function of k:

Theorem 3 (B. and O’Donnell [15]). Let g : {0, 1}n → {0, 1} be a k-junta n for some k ≤ 2 in which all k relevant coordinates have influence at least . Then testing g-isomorphism non-adaptively requires Ω(log k) queries.3

Theorems 2 and 3 complement each other quite well: the former says that if g is a O(1)-junta, then testing g-isomorphism can be done with a number of queries independent of n, while the latter asserts that if g is a “strong” ω(1)-junta, then any algorithm for testing g-isomorphism will need a number of queries that depends on n. Very recently, Noga Alon and I have obtained another general lower bound of a different nature:

Theorem 4 (Alon and B. [1]). For a 1 − o(1) fraction of all functions g : {0, 1}n → {0, 1}, testing g-isomorphism non-adaptively requires Ω(n) queries.

In fact, the proof of Theorem 4 gives an even stronger result: for most functions g, non-adaptive algorithms need Ω(n) queries to distinguish be- tween functions isomorphic to g and nearly-random functions.

3.4 Proposed research

The last section has outlined some results on testing function isomorphism, but so far no connection has been established to the original problem of characterizing testable properties on boolean functions. I intend to complete this part of the project for the thesis. There are a number of specific ways in which I plan to do so.

3The theorem stated here is in fact a special case of the main theorem in [15]; see the paper for details.

8 First, the proof of Theorem 3 immediately yields a corollary that is similar in spirit to Theorem 1:

Corollary 1 (follows from [15]). Let P be a property on boolean functions for which there exists a function t : N → N that is not bounded above and such that (1) every f ∈ P ∩ Bn is -far from being a t(n) − 1-junta, and (2) for infinitely many n ≥ 1, there is a function f ∈ P ∩ Bn that is a t(n)-juntas. Then P is not testable.

The statement of the Corollary is somewhat technical, but it does en- compass some natural properties. Most notably, the class of “non-concise” properties – such as the property of not being -approximable by a circuit of size log n – satisfy the Corollary’s conditions. Nonetheless, it would be nice if we could weaken the conditions of the corollary (including perhaps even removing condition (1) entirely); I plan to examine this question for this thesis. The second way in which I plan to connect the results on testing function isomorphism to the characterization of testable properties is by building on Theorem 4. In particular, the strengthening of the theorem mentioned below the theorem statement is quite powerful: it says that for most boolean functions g, any non-trivial property P that contains g will be hard to test, since it is hard to even distinguish between a function isomorphic to g (that is in P) from a random function (that is -far from P with high probability when P is non-trivial). I plan to examine how this technique can be used to get some characterizations of non-testable properties, and to work on extending Theorem 4 to enable more such characterizations. The third and final way in which I hope to build connections between testing function isomorphism and characterizing testable properties is not yet as well defined: simply put, I want to better understand why Theorems 3 and 4 are true. Can we obtain natural characterizations of the functions for which testing function isomorphism requires many queries? (Say, in terms of the influence of the coordinates in the function, perhaps?) My hope is that a better understanding of these results will also lead to valuable insight that may be used to obtain more general characterizations for other properties of boolean functions.

9 4 Exact query complexity bounds

We now turn to the second goal of this thesis: to obtain exact bounds on the query complexity for some fundamental properties of boolean functions.

4.1 Related work

There are two notable properties of boolean functions for which there has been a significant research effort aimed at determining the exact query com- plexity required to test these properties: monotonicity and juntas. The study of monotonicity testing on boolean functions was initiated by Goldreich et al. [25]. They gave the first upper bounds on the query com- plexity of the problem, showing that roughly O(n2/) queries are sufficient to determine whether a function f : {0, 1}n → {0, 1} is monotone or -far from monotone. An improved analysis of this algorithm allowed Dodis et al. [21] and Goldreich et al. [26] to obtain a better bound of O(n/) on the query complexity of the problem. Shortly afterwards, Fischer et al. [23] showed that there exists  > 0 such that the query complexity of testing monotonicity is bounded below √ by Ω(log n). They also established a tighter lower bound of Ω( n) queries for non-adaptive testing algorithms with one-sided error. Since then, many variants of the monotonicity testing problem have been studied but no other improvements on the upper or lower bounds on the query complexity for testing monotonicity of boolean functions. The study of juntas was initiated by the remarkable article of Fischer et al. [22]. The first result presented in the article is a natural algorithm for testing k-juntas with roughly O(k4) queries. This result was sufficient to show that the property of being a k-junta is testable for every constant k. Yet, Fischer et al. did not end their analysis there; instead, they gave a num- ber of improvements on their basic algorithm that reduced the query com- plexity to O(k2). They gave different algorithms that achieve this bound, including one that is non-adaptive. Fischer et al. [22] also introduced the first lower bound for testing juntas, √ showing that non-adaptive√ algorithm for testing k-juntas (for k < o( n)) must make at least Ω( k) queries to the function. By a standard technique, this lower bound translates to a lower bound of Ω(log k) on the general query complexity for testing k-juntas. This bound was strengthened to Ω(k) by Chockler and Gutfreund [17].

10 The gap between the upper and lower bounds on the problem of testing juntas remained unchanged until a couple years ago, when I introduced new algorithms for testing juntas more efficiently. These results are presented in the next section.

4.2 Completed work on testing juntas

My first result on testing juntas combined a sampling technique and some ideas of Fischer et al. [22] to obtain an improved bound on the query com- plexity for testing juntas.

Theorem 5 ([13]). There is a non-adaptive algorithm for testing k-juntas with O˜(k3/2) queries.

This result was strong enough to disprove a conjecture of Fischer et al. [22], but left a polynomial-sized gap between it and the best-known lower bound. This gap was mostly closed by a follow-up result:

Theorem 6 ([14]). There is an adaptive algorithm for testing k-juntas with O(k log k) queries.

Theorems 5 and 6 both led to new insights into the structure of boolean functions that are far from juntas. More specifically, the proofs of both theorems relied on new characterizations of the distribution of influence of coordinates (and sets of coordinates) in functions that are -far from k- juntas. For the details, we refer the readers to [13, 14].

4.3 Proposed research

There are many other properties of boolean functions for which we still have a large gap between the best-known upper and lower bounds on the query complexity for testing these properties. For this thesis, I plan on pursuing research that aims to close the gap for two specific problems: testing concise representations and testing monotonicity. As we discussed in Section 3.1, Diakonikolas et al. [20] obtained non- trivial upper bounds for many properties related to the concise representa- tion of functions – such as the property of being representable by a DNF with few terms, by a small decision list or decision tree, or by a small boolean circuit. For many of these properties, there is still a very large gap between

11 the best upper and lower bounds on the exact query complexity required to test these properties. In fact, for some of these properties (such as being computable by a small boolean circuit), no lower bounds exist at all. I plan to try to improve both the upper bounds and the lower bounds for testing some of the concise representation properties. I plan to improve the upper bounds by extending the implicit learning technique to use the more efficient junta tests from [13, 14]. And I plan to improve the lower bounds by applying some of the techniques that have been developed in [1, 15]. The second problem that I propose to examine is to close the gap between the upper and lower bounds on the query complexity for testing monotonic- ity. The current algorithm for testing monotonicity of boolean functions simply samples edges in the hypercube at random and accepts the func- tions if the edges all satisfy the monotonicity constraint. In the worst case, this algorithm needs O(n) queries to correctly reject all functions that are -far from monotone. However, a closer examination shows that for most functions that are -far from monotone, the algorithm is likely to detect √ a non-monotone edge when it makes only O( n) edge queries. In fact, it appears that the only functions for which this statment is not true must satisfy some strong structural constraints. I intend to look at this problem in more detail in the near future. Lastly, there is one more issue that I wish to mention here: as the results in the previous section show, there is still a gap between the query complexity – O(k log k) – of the best adaptive algorithm for testing k-juntas and the query complexity – O(k3/2) – of the best non-adaptive algorithm for testing k-juntas. While I am not currently looking at this question, it would be interesting to see if the non-adaptive upper bound can be improved of if there is a gap between the adaptive and non-adaptive query complexities for testing juntas.

5 Other proposed research

The problems listed in the previous two sections form the core of the research that I plan to do for my thesis. But there are other problems related to property testing on boolean functions that I also hope to study in the near future. In this section, we cover two such problems that I may study in the course of completing my thesis research.

12 5.1 Alternative models of property testing

One interesting way to look at property testing is as a relaxation of machine learning: In learning tasks, one must find a hypothesis in a concept class that gives a good approximation to some target function. In property testing, one only needs to determine whether there is a hypothesis in the concept class that approximates the target function well. (In this view, we associate the concept class with a property – the tester then distinguishes between the cases where the target function is in the concept class and where all hypotheses in the concept class are -far from the target function.) In many cases, there is an large difference between the query complexity of the learning and testing problems [27]. As a result, in situations where queries are expensive, it could be quite useful to use testing algorithms as a pre-processing step that precedes the learning task: in this way, if no good hypothesis exists, this fact is discovered with a minimal number of queries (and perhaps some other concept classes can be considered). Unfortunately, there is one problem with applying this idea in practice: property testing corresponds to a very strong model of machine learning – the membership query learning model – in which any input of the algorithm’s choosing can be labeled. In many cases, this assumption is unrealistic and weaker learning models are more appropriate. This state of affairs naturally suggests the need for defining and explor- ing alternative models of property testing that correspond to the weaker models of learning. There are two models, in particular, that I would like to explore: property testing with random samples, and property testing on partial boolean functions. The model of property testing with random samples is, as its name suggests, a variation on the property testing model where the tester no longer is able to query the function f : {0, 1}n → {0, 1} on inputs of its choosing, but instead each query is answered by returning a randomly selected input x ∈ {0, 1}n and the value of f(x). This model of testing corresponds to the standard PAC model of learning [42]. I have not yet studied this model, but would be interested doing so and in seeing if there are connections between this model and the area of testing distributions [7, 8, 9, 10, 39]. The model of property testing on partial boolean functions aims to find some middle ground between the standard property testing model (in which the tester has full control over the queries it chooses to make) and the random sample model (in which the tester has no control over the inputs queried).

13 One possible way to do so is to define a model in which the input to the tester is a function f and a subset X ⊆ {0, 1}n chosen randomly and is then free to query for the value of f(x) for any x ∈ X, but cannot query any other input. This model corresponds to active learning [31]. Once again, I have not yet studied this model but intend to do so in the near future. The first step in this research, of course, will be to formalize the model appropriately.

5.2 Application of property testing ideas to other domains

The last problem I want to mention is actually a meta-problem: can we apply the knowledge that we gain on boolean functions from property testing to other fields of study? This is a question that I want to keep in mind as I pursue my thesis research. I have already had some success in applying ideas from property test- ing to problems in other domains: recently, I have used the techniques for proving lower bounds on the query complexity for testing juntas [13, 17] to obtain lower bounds on the minimum depth of a new kind of decision tree, known as k+ decision trees – for computing some functions [5].

6 Suggested timeline

This thesis proposal is being held on May 27, 2010. My plan is to pursue the research proposed in this document and to start writing my dissertation in January 2011. I hope to complete my thesis and its defense in or around June 2011.

References

[1] Noga Alon and Eric Blais. Testing boolean function isomorphism, 2010. Submitted to RANDOM ’10. [2] Noga Alon, Eldar Fischer, Ilan Newman, and Asaf Shapira. A combi- natorial characterization of the testable graph properties: It’s all about regularity. In STOC ’06, pages 251–260, 2006. [3] Noga Alon and Asaf Shapira. A characterization of the (natural) graph properties testable with one-sided error. In FOCS ’05, pages 429–438, 2005.

14 [4] Noga Alon and Asaf Shapira. Every monotone graph property is testable. In FOCS ’05, pages 128–137, 2005. [5] James Aspnes, Eric Blais, Murat Demirbas, Ryan O’Donnell, Atri Rudra, and Steve Uurtamo. k+ decision trees, 2010. Manuscript. [6] Tim Austin and Terence Tao. On the testability and repair of hereditary hypergraph properties, 2008. [7] Tugkan Batu, Sanjoy Dasgupta, Ravi Kumar, and Ronitt Rubinfeld. The complexity of approximating the entropy. SIAM J. Computing, 35(1):132–150, 2005. [8] Tugkan Batu, Eldar Fischer, Lance Fortnow, Ravi Kumar, Ronitt Ru- binfeld, and Patrick White. Testing random variables for independence and identity. In FOCS ’01, pages 442–451, 2001. [9] Tugkan Batu, Lance Fortnow, Ronitt Rubinfeld, Warren D. Smith, and Patrick White. Testing that distributions are close. In FOCS ’00, pages 259–269, 2000. [10] Tugkan Batu, Ravi Kumar, and Ronitt Rubinfeld. Sublinear algorithms for testing monotone and unimodal distributions. In STOC ’04, pages 381–390, 2004. [11] Mihir Bellare, Oded Goldreich, and Madhu Sudan. Free bits, PCPs and non-approximability – towards tight results. SIAM J. Comput., 27(3):804–915, 1998. [12] Arnab Bhattacharyya, Victor Chen, Madhu Sudan, and Ning Xie. Test- ing linear-invariant non-linear properties. In STACS ’09, pages 135–146, 2009. [13] Eric Blais. Improved bounds for testing juntas. In RANDOM ’08, pages 317–330, 2008. [14] Eric Blais. Testing juntas nearly optimally. In STOC ’09, pages 151– 158, 2009. [15] Eric Blais and Ryan O’Donnell. Lower bounds for testing function isomorphism. In CCC ’10, 2010. [16] B´elaBollob´as. Combinatorics: Set Systems, Hypergraphs, Families of Vectors, and Combinatorial Probability. Cambridge University Press, 1986.

15 [17] Hana Chockler and Dan Gutfreund. A lower bound for testing juntas. Information Processing Letters, 90(6):301–305, 2004. [18] Thomas W. Cusick and Pantelimon St˘anic˘a. Cryptographic Boolean Functions and Applications. Academic Press, 2009. [19] Luc Devroye, L´aszl´oGy¨orfi,and G´abor Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1996. [20] Ilias Diakonikolas, Homin K. Lee, Kevin Matulef, Krzysztof Onak, Ronitt Rubinfeld, Rocco A. Servedio, and Andrew Wan. Testing for concise representations. In FOCS ’07, pages 549–558, 2007. [21] Yevgeniy Dodis, Oded Goldreich, Eric Lehman, Sofya Raskhodnikova, Dana Ron, and Alex Samorodnitsky. Improved algorithms for testing monotonicity. In RANDOM ’99, pages 97–108, 1999. [22] Eldar Fischer, Guy Kindler, Dana Ron, Shmuel Safra, and Alex Samorodnitsky. Testing juntas. J. Comput. Syst. Sci., 68(4):753–787, 2004. [23] Eldar Fischer, Eric Lehman, Ilan Newman, Sofya Raskhodnikova, Ronitt Rubinfeld, and Alex Samorodnitsky. Monotonicity testing over general poset domains. In STOC ’02, pages 474–483, 2002. [24] Oded Goldreich. Introduction to testing graph properties, 2010. ECCC Technical report TR10-082. [25] Oded Goldreich, Shafi Goldwasser, Eric Lehman, and Dana Ron. Test- ing monotonicity. In FOCS ’98, pages 426–435, 1998. [26] Oded Goldreich, Shafi Goldwasser, Eric Lehman, Dana Ron, and Alex Samorodnitsky. Testing monotonicity. Combinatorica, 20(3):301–337, 2000. [27] Oded Goldreich, Shari Goldwasser, and Dana Ron. Property testing and its connection to learning and approximation. J. of the ACM, 45(4):653–750, 1998. [28] Parikshit Gopalan, Ryan O’Donnell, Rocco Servedio, Amir Shpilka, and Karl Wimmer. Testing Fourier dimensionality and sparsity. In ICALP ’09, pages 500–512, 2009. [29] Tali Kaufman and Madhu Sudan. Algebraic property testing: the role of invariance. In STOC ’08, pages 403–412, 2008.

16 [30] Donald Knuth. The Art of Computer Programming: Volume 4, Fascicle 1. Addison-Wesley, 1999.

[31] David D. Lewis and William A. Gale. A sequential algorithm for train- ing text classifiers. In SIGIR ’94, pages 3–12, 1994.

[32] Kevin Matulef, Ryan O’Donnell, Ronitt Rubinfeld, and Rocco A. Serve- dio. Testing halfspaces. In SODA ’09, pages 256–264, 2009.

[33] Kevin Matulef, Ryan O’Donnell, Ronitt Rubinfeld, and Rocco A. Serve- dio. Testing ±1-weight halfspace. In RANDOM ’09, pages 646–657, 2009.

[34] Christoph Meinel and Thorsten Theobald. Algorithms and Data Struc- tures in VLSI Design. Springer, 1998.

[35] Michal Parnas, Dana Ron, and Alex Samorodnitsky. Testing basic boolean formulae. SIAM J. Discrete Math., 16(1):20–46, 2002.

[36] Dana Ron. Property testing: a learning theory perspective. Founda- tions and Trends in Machine Learning, 1(3):307–402, 2008.

[37] Dana Ron. Algorithmic and analysis techniques in property testing. Foundations and Trends in Theoretical Computer Science, 5(2):73–205, 2009.

[38] Ronitt Rubinfeld. Sublinear time algorithms. In ICM ’06, 2006.

[39] Ronitt Rubinfeld and Rocco Servedio. Testing monotone high- dimensional distributions. Random Structures and Algorithms, 34(1):24–44, 2009.

[40] Asaf Shapira. Green’s conjecture and testing linear-invariant properties. pages 159–166, 2009.

[41] Madhu Sudan. Invariance in property testing, 2010. ECCC Technical report TR10-051.

[42] Leslie Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134–1142, 1984.

[43] Jacobius H. van Lint. Introduction to Coding Theory. Springer, 1999.

[44] Ingo Wegener. The Complexity of Boolean Functions. Wiley, 1987.

17