On the Complexity of Numerical Analysis Eric Allender Peter B¨urgisser Rutgers, the State University of NJ Paderborn University Department of Computer Science Department of Mathematics Piscataway, NJ 08854-8019, USA DE-33095 Paderborn, Germany [email protected] [email protected] Johan Kjeldgaard-Pedersen Peter Bro Miltersen PA Consulting Group University of Aarhus Decision Sciences Practice Department of Computer Science Tuborg Blvd. 5, DK 2900 Hellerup, Denmark IT-parken, DK 8200 Aarhus N, Denmark [email protected] [email protected] Abstract In Section 1.3 we discuss our main technical contribu- tions: proving upper and lower bounds on the complexity We study two quite different approaches to understand- of PosSLP. In Section 1.4 we present applications of our ing the complexity of fundamental problems in numerical main result with respect to the Euclidean Traveling Sales- analysis. We show that both hinge on the question of under- man Problem and the Sum-of-Square-Roots problem. standing the complexity of the following problem, which we call PosSLP: Given a division-free straight-line program 1.1 Polynomial Time Over the Reals producing an integer N, decide whether N>0. We show that PosSLP lies in the counting hierarchy, and combining The Blum-Shub-Smale model of computation over the our results with work of Tiwari, we show that the Euclidean reals provides a very well-studied complexity-theoretic set- Traveling Salesman Problem lies in the counting hierarchy ting in which to study the computational problems of nu- – the previous best upper bound for this important problem merical analysis. We refer the reader to Blum, Cucker, Shub (in terms of classical complexity classes) being PSPACE. and Smale [9] for detailed definitions and background ma- terial related to this model; here, we will recall only a few salient facts. In the Blum-Shub-Smale model, each machine 1 Introduction computing over the reals has associated with it a finite set of real machine constants. The inputs to a machine are el- Rn = R ements of Sn ∞, and thus each polynomial-time The original motivation for this paper comes from a de- machine over R accepts a “decision problem” L R∞. sire to understand the complexity of computation over the The set of decision problems accepted by polynomial-time⊆ reals in the Blum-Shub-Smale model. In Section 1.1 we machines over R is denoted PR. give a brief introduction to this model and we introduce the There has been considerable interest in relating computa- problem PosSLP and explain its importance in understand- tion over R to the classical Boolean complexity classes such ing the Blum-Shub-Smale model. as P, NP, PSPACE, etc. This is accomplished by consider- In Section 1.2 we present yet another reason to be in- ing the Boolean part of decision problems over the reals. terested in PosSLP. We isolate a computational problem That is, given a problem L R∞, the Boolean part of L ⊆ that lies at the root of the task of designing numerically sta- is defined as BP(L):=L 0; 1 ∞. (Here, we follow the ∩{ } n ble algorithms. We show that this task is computationally notation of [9]; 0; 1 ∞ = 0; 1 , which is identical { } Sn{ } equivalent to PosSLP. The material in Sections 1.1 and 1.2 to 0; 1 ∗.) The Boolean part of PR, denoted BP(PR),is { } provides motivation for studying PosSLP and for attempt- defined as BP(L) L PR . ing to place it within the framework of traditional complex- By encoding{ the| advice∈ function} in a single real constant ity classes. as in Koiran [27], one can show that P=poly BP(PR). ⊆ Dagstuhl Seminar Proceedings 06111 Complexity of Boolean Functions http://drops.dagstuhl.de/opus/volltexte/2006/613 The best upper bound on the complexity of problems in BP(PR). The proof in fact shows even PPosSLP=poly ⊆ BP(PR) that is currently known was obtained by Cucker BP(PR). The real constant encoding the advice function, and Grigoriev [16]: will, of course, in general be transcendental. Thus, there is a strong relationship between non-uniformity in the clas- BP(PR) PSPACE=poly: (1) ⊆ sical model of computation and the use of transcendental There has been no work pointing to lower bounds on the constants in the Blum-Shub-Smale model. We conjecture that this relationship can be further strengthened: complexity of BP(PR); nobody has presented any com- pelling evidence that BP(PR) is not equal to P=poly. PosSLP Conjecture 1.2 P =poly = BP(PR) There has also been some suggestion that perhaps BP(PR) is equal to PSPACE=poly. For instance, cer- 1.2 The Task of a Numerical Analyst tain variants of the RAM model that provide for unit- cost arithmetic can simulate all of PSPACE in polyno- The Blum-Shub-Smale model is a very elegant one, but mial time [6, 23]. Since the Blum-Shub-Smale model also it does not take into account the fact that actual numerical provides for unit-time multiplication on “large” numbers, computations have to deal with finitely represented values. Cucker and Grigoriev [16] mention that researchers have We next observe that even if we take this into account, the raised the possibility that similar arguments might show that PosSLP problem still captures the complexity of numerical polynomial-time computation over R might be able to sim- computation. ulate PSPACE. Cucker and Grigoriev also observe that cer- Let u =0be a dyadic rational number. The floating tain na¨ıve approaches to provide such a simulation must fail. point representation6 of u is obtained by writing u = v2m One of our goals is to provide evidence that BP(PR) lies 1 where m is an integer and 2 v < 1. The floating point properly between P=poly and PSPACE=poly. Towards this representation is then given by≤| the| sign of v,andtheusual goal, it is crucial to understand a certain decision problem binary representations of the numbers v and m.Thefloat- PosSLP: The problem of deciding, for a given straight-line ing point representation of 0 is the string| | 0 itself. We shall program, whether it represents a positive integer.(Forpre- abuse notation and identify the floating point representation cise definitions, see the next section.) of a number with the number itself, using the term “floating The immediate relationship between the Blum-Shub- point number” for the number as well as its representation. Smale model and the problem PosSLP is given by the Let u =0be a real number. We may write u as u = proposition below. Following B¨urgisser and Cucker [13], m 6 1 u02 where u0 < 1 and m is an integer. Then, we 0 2 ≤| | define PR to be the class of decision problems over the reals define a floating point approximation of u with k significant decided by polynomial time Blum-Shub-Smale machines m bits to be a floating point number v2 so that v u0 using only the constants 0; 1. (k+1) | − |≤ 2− . We will focus on one part of the job that is done by PosSLP 0 Proposition 1.1 P =BP(PR). numerical analysts: the design of numerically-stable algo- rithms. In our scenario, the numerical analyst starts out with Proof. (Sketch) It is clear that PosSLP is in BP(P0),since R a known function f, and the task is to design a “good” al- we can implement a standard SLP interpreter in the Real gorithm for it. When we say that the function f is “known”, Turing Machine framework and evaluate the result in linear we mean that the analyst starts out with some method of time using unit cost instructions. To show the other direc- computing (or at least approximating) f; we restrict atten- tion, assume we have a polynomial time machine over R tion to the “easy” case where the method for computing f using only the constants 0; 1. Given a bit string as input, we uses only the arithmetic operations +; ; ; , and thus the simulate the computation by storing the straight-line pro- description of f that the analyst is given− ∗ can÷ be presented gram representation of the intermediate results instead of as an arithmetic circuit with operations +; ; ; .Usu- their values. Branch instructions can be simulated by us- ally, the analyst also has to worry about the− problems∗ ÷ that ing the oracle to determine if the contents of a given regis- are caused by the fact that the inputs to f are not known ter (represented by a straight-line program) is greater than precisely, but are only given as floating point numbers that zero. are approximations to the “true” inputs – but again we will It was shown by Chapuis and Koiran [14] that algebraic focus on the “easy” case where the analyst will merely try 0 constants do not help. More specifically, BP(PR) equals to compute a good approximation for f(x1;::: ;xn) on the the Boolean part of the class of decision problems over the exact floating point numbers x1;::: ;xn that are presented reals decided by polynomial time Blum-Shub-Smale ma- as input: chines using real algebraic numbers as constants. The generic task of numerical computation: Given an As already mentioned, by encoding the advice function integer k in unary and a straight-line program (with )tak- in a single real constant, one can show that P=poly ing as inputs floating point numbers, with a promise÷ that it ⊆ 2 r neither evaluates to zero nor does division by zero, compute The desired output is a floating point number u = u02 , (k+1) a floating point approximation of the value of the output where v u0 2− .