The Random Oracle Hypothesis Is False
Total Page:16
File Type:pdf, Size:1020Kb
The Random Oracle Hyp othesis is False 1 ; 2 3 ; 4 3 ; 5 Richard Chang Benny Chor Oded Goldreich 1 6 1 ; 7 Juris Hartmanis Johan Hastad Desh Ranjan 1 Pankaj Rohatgi September 25, 1997 Abstract The Random Oracle Hyp othesis, attributed to Bennett and Gill, essentially states that the relationships b etween complexity classes whichholdforalmost all rela- tivized worlds must also hold in the unrelativized case. Although this pap er is not the rst to provideacounterexample to the Random Oracle Hyp othesis, it do es provide a most comp elling counterexample by showing that for almost all oracles A A A,IP 6=PSPACE . If the Random Oracle Hyp othesis were true, it would con- tradict Shamir's result that IP = PSPACE . In fact, it is shown that for almost all A A oracles A, co-NP 6 IP . These results extend to the multi-prover pro of systems of Ben-Or, Goldwasser, Kilian and Wigderson. In addition, this pap er shows that the Random Oracle Hyp othesis is sensitive to small changes in the de nition. A class IPP, similar to IP, is de ned. Surprisingly, the IPP = PSPACE result holds for all oracle worlds. Warning: Essentially this pap er has b een published in Information and Com- putation and is hence sub ject to copyright restrictions. It is for p ersonal use only. 1 Department of Computer Science, Cornell University, Ithaca, NY 14853, U.S.A. Supp orted in part by NSF ResearchGrant CCR-88-23053. 2 Supp orted in part by an IBM Graduate Fellowship. 3 Department of Computer Science, Technion, Haifa 32000, Israel. 4 Supp orted by grantNo. 88-00282 from the United States { Israel Binational Science Foundation (BSF), Jerusalem, Israel. 5 Supp orted by grantNo. 88-00301 from the United States { Israel Binational Science Foundation (BSF), Jerusalem, Israel. 6 Department of Computer Science, Royal Institute of Technology, S-100 44 Sto ckholm, Sweden. 7 Supp orted in part by a Mathematical Sciences Institute (MSI) Fellowship. 1 Intro duction Computational complexity theory studies the quantitativelaws whichgovern computing. It seeks a comprehensive classi cation of problems by their intrinsic diculty and an under- standing of what makes these problems hard to compute. The key concept in classifying the computational complexity of problems is the complexity class which consists of all the problems solvable on agiven computational mo del and within a given resource b ound. Structural complexity theory is primarily concerned with the relations among various complexity classes and the internal structure of these classes. Figure 1 shows some ma jor complexity classes. Although much is known ab out the structure of these classes, there have not b een any results which separate any of the classes b etween P and PSPACE . We b elieve that all these classes are di erent and regard the problem of proving the exact relationships among these classes as the Grand Challenge of complexity theory. The awareness of the imp ortance of P, NP, PSPACE , etc, has led to a broad investigation of these classes and to the use of relativization. Almost all of the ma jor results in recursive function theory also hold in relativized worlds. Quite the contrary happ ens in complexity theory. It was shown in 1975 [3] that there exist oracles A and B such that A B A B P =NP and P 6=NP : This was followed by an extensive investigation of the structure of complexity classes under relativization. An impressive set of techniques was develop ed for oracle constructions and some very subtle and interesting relativization results were obtained. For example, for a long time it was not known if the Polynomial-time Hierarchy (PH) can be separated by oracles from PSPACE . In 1985, A. Yao [32] nally resolved this problem by constructing an oracle A, such that A A PH 6=PSPACE : Hastad [21] simpli ed this pro of and constructed an oracle B , such that P;B B 8k; PH 6= : k These metho ds were re ned by Ko [25] to show that for every k 0 there is an oracle which th collapses PH to exactly the k level and keeps the rst k 1 levels of PH distinct. That is, for all k , there exists an A such that P;A P;A P;A P;A P;A 6= 6= 6= and = ; i 0: 0 1 k k k +i Another asp ect of relativized computations was studied by Bennett and Gill who decided to measure the set of oracles which separate certain complexity classes. They showed that A A P 6=NP for almost all oracles. More precisely, they showed that for almost all oracles A the following relationships hold [5]: A A A P 6=NP 6= co-NP A A SPACE [log n] 6=P A A PSPACE 6=EXP A A A P =RP =BPP : 1 Figure omitted. Figure 1: Some standard complexity classes. 2 Many other interesting random oracle results followed. For almost all oracles A [8, 9, 27]: A BH is in nite A A PH PSPACE The Berman-Hartmanis Conjecture fails relativetoA. A The last result asserts that there exist non-isomorphic many-one complete sets for NP for random oracle A. It was conjectured that all NP many-one complete sets are p olynomial-time isomorphic [6]. Surveying the rich set of relativization results, we can make several observations. First, almost all questions ab out the relationship b etween the ma jor complexity classes have con- tradictory relativizations. That is, there exist oracles which separate the classes and oracles which collapse them. Furthermore, many of our pro of techniques relativize and cannot re- solve problems with contradictory relativizations. Finally,we have unsuccessfully struggled for over twenty years to resolve whether P=?NP=?PSPACE . These observations seemed to supp ort the conviction that problems with contradictory relativizations are extremely dicult and may not be solvable by current techniques. This opinion was succinctly expressed by John Hop croft [22]: This p erplexing state of a airs is obviously unsatisfactory as it stands. No prob- lem that has b een relativized in two con icting ways has yet been solved, and this fact is generally taken as evidence that the solutions of such problems are beyond the current state of mathematics. How should complexity theorists remedy \this p erplexing state of a airs"? In one ap- proach, we assume as a working hyp othesis that PH has in nitely many levels. Thus, any assumption which would imply that PH is nite is deemed incorrect. For example, Karp, P Lipton and Sipser [24] showed that if NP P/p oly, then PH collapses to . So, we 2 b elieve that SAT do es not have p olynomial sized circuits. Similarly, we b elieve that the Turing-complete and many-one complete sets for NP are not sparse, b ecause Mahaney [29] showed that these conditions would collapse PH. One can even show that for any k 0, SAT[k ] SAT[k +1] SAT[k ] SAT[k +1] P =P implies that PH is nite [23]. Hence, we b elieve that P 6=P for all k 0. Thus, if the Polynomial Hierarchy is indeed in nite, we can describ e many asp ects of the computational complexityofNP. A second approach used random oracles. Since most of the random oracle relativization results agreed with what complexity theorists b elieved to b e true in the base case and since random oracles have no particular structure of their own, it seemed that the b ehavior of complexity classes relative to a random oracle should b e the same as the base case b ehavior. This led Bennett and Gill to p ostulate the Random Oracle Hyp othesis [5] which essentially states that structural relationships which hold in almost all oracle worlds also hold in the unrelativized case | i.e., the real world. In the following, we will discuss a set of results ab out interactive pro ofs which provide dramatic counterexamples to the b elief that problems with contradictory relativizations can- not be resolved with known techniques. So on after these results were publicized, several researchers indep endently noticed that these results also add a striking new counterexam- ple against the Random Oracle Hyp othesis [10, 20]. There have previously b een several 3 counterexamples in the literature and in unpublished rep orts [19, 26, 13]. Some of these counterexamples use double relativization and classes which are not closed under p olyno- mial time reductions. While the results in this pap er are not the rst, the authors b elieve that they are the most natural and comp elling. Thus, contradictory relativizations should no longer be viewed as strong evidence that a problem is beyond our grasp. We hop e that these results will encourage complexity theorists to renew the attack on problems with con- tradictory relativizations. 2 A Review of IP The class IP is the set of languages that have interactive pro ofs or proto cols. IP was rst de ned as way to generalize NP [1, 17]. NP can be characterized as b eing precisely those languages for which one can present a p olynomially long pro of to certify that the input string is in the language. Moreover, the pro of can b e checked in p olynomial time. It is this idea of presenting and checking the pro of that the de nition of IP generalizes. Is there a way of giving convincing evidence that the input string is in a language without showing the whole pro of to a veri er? Clearly,ifwe do not give a complete pro of to a veri er which do es not have the power or the time to generate and check a pro of, then we cannot exp ect the veri er to b e completely convinced.