Networks of Relations

Networks of Relations Thesis by Matthew Cook In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California 2005 (Defended May 27, 2005) ii c 2005 Matthew Cook All Rights Reserved iii Acknowledgements I would like to thank the ARCS foundation, the National Institute of Health, the Alpha Project, and my advisor Shuki Bruck for supporting me during my studies. I would also like to thank Shuki for being a good advisor and collaborator. I am grateful not only to Shuki but to all the people I have worked with, including Erik Winfree and David Soloveichik, in collaboration with whom the material in section 3.4.2 was produced. My family has supported my adventure of being a student, especially my wife Eva,´ my children András, Adam, and Emma, my mother Sarah, and my grandfather Howard, and to them I am very grateful. iv Abstract Relations are everywhere. In particular, we think and reason in terms of mathematical and English sentences that state relations. However, we teach our students much more about how to manipulate functions than about how to manipulate relations. Consider functions. We know how to combine functions to make new functions, how to evaluate functions efficiently, and how to think about compositions of functions. Especially in the area of boolean functions, we have become experts in the theory and art of designing combinations of functions to yield what we want, and this expertise has led to techniques that enable us to implement mind-bogglingly large yet efficient networks of such functions in hardware to help us with calculations. If we are to make progress in getting machines to be able to reason as well as they can calculate, we need to similarly develop our understanding of relations, especially their composition, so we can develop techniques to help us bridge between the large and small scales. There has been some important work in this area, ranging from practical applications such as relational databases to extremely theoretical work in universal algebra, and sometimes theory and practice manage to meet, such as in the programming language Prolog, or in the probabilistic reasoning methods of artificial intelligence. However, the real adventure is yet to come, as we learn to develop a better understanding of how relations can efficiently and reliably be composed to get from a low level representation to a high level representation, as this understanding will then allow the development of automated techniques to do this on a grand scale, finally enabling us to build machines that can reason as amazingly as our contemporary machines can calculate. This thesis explores new ground regarding the composition of relations into larger relational structures. First of all a foundation is laid by examining how networks of relations might be used for automated reasoning. We define exclusion networks, which have close connections with the areas of constraint satisfaction problems, belief propagation, and even boolean circuits. The foundation is laid somewhat deeper than usual, taking us inside the v relations and inside the variables to see what is the simplest underlying structure that can satisfactorily represent the relationships contained in a relational network. This leads us to define zipper networks, an extremely low-level view in which the names of variables or even their values are no longer necessary, and relations and variables share a common substrate that does not distinguish between the two. A set of simple equivalence operations is found that allows one to transform a zipper network while retaining its solution structure, enabling a relation-variable duality as well as a canonical form on linear segments. Similarly simple operations allow automated deduction to take place, and these operations are simple and uniform enough that they are easy to imagine being implemented by biological neural structures. The canonical form for linear segments can be represented as a matrix, leading us to matrix networks. We study the question of how we can perform a change of basis in matrix networks, which brings us to a new understanding of Valiant’s recent holographic algorithms, a new source of polynomial time algorithms for counting problems on graphs that would otherwise appear to take exponential time. We show how the holographic transformation can be understood as a collection of changes of basis on individual edges of the graph, thus providing a new level of freedom to the method, as each edge may now independently choose a basis so as to transform the matrices into the required form. Consideration of zipper networks makes it clear that “fan-out,” i.e., the ability to du- plicate information (for example allowing a variable to be used in many places), is most naturally itself represented as a relation along with everything else. This is a notable de- parture from the traditional lack of representation for this ability. This deconstruction of fan-out provides a more general model for combining relations than was provided by pre- vious models, since we can examine both the traditional case where fan-out (the equality relation on three variables) is available and the more interesting case where its availability is subject to the same limitations as the availability of other relations. As we investigate the composition of relations in this model where fan-out is explicit, what we find is very different from what has been found in the past. First of all we examine the relative expressive power of small relations: For each relation on three boolean variables, we examine which others can be implemented by networks built solely from that relation. (We also find, in each of these cases, the complexity of deciding whether such a network has a solution. We find that solutions can be found in polynomial vi time for all but one case, which is NP-complete.) For the question of which relations are able to implement which others, we provide an extensive and complete answer in the form of a hierarchy of relative expressive power for these relations. The hierarchy for relations is more complex than Post’s well-known comparable hierarchy for functions, and parts of it are particularly difficult to prove. We find an explanation for this phenomenon by showing that in fact, the question of whether one relation can implement another (and thus should be located above it in the hierarchy) is undecidable. We show this by means of a complicated reduction from the halting problem for register machines. The hierarchy itself has a lot of structure, as it is rarely the case that two ternary boolean relations are equivalent. Often they are comparable, and often they are incomparable—the hierarchy has quite a bit of width as well as depth. Notably, the fan-out relation is particularly difficult to implement; only a very few relations are capable of implementing it. This provides an additional ex post facto justification for considering the case where fan-out is absent: If you are not explicitly provided with fan-out, you are unlikely to be able to implement it. The undecidability of the hierarchy contrasts strongly with the traditional case, where the ubiquitous availability of fan-out causes all implementability questions to collapse into a finite decidable form. Thus we see that for implementability among relations, fan-out leads to undecidability. We then go on to examine whether this result might be taken back to the world of functions to find a similar difference there. As we study the implementability question among functions without fan-out, we are led directly to questions that are independently compelling, as our functional implementability question turns out to be equivalent to asking what can be computed by sets of chemical reactions acting on a finite number of species. In addition to these chemical reaction networks, several other nondeterministic systems are also found to be equivalent in this way to the implementability question, namely, Petri nets, unordered Fractran, vector addition systems, and “broken” register machines (whose decrement instruction may fail even on positive registers). We prove equivalences between these systems. We find several interesting results in particular for chemical reaction networks, where the standard model has reaction rates that depend on concentration. In this setting, we analyze questions of possibility as well as questions of probability. The question of the possibility of reaching a target state turns out to be equivalent to the reachability question for Petri nets and vector addition systems, which has been well studied. We provide a vii new proof that a form of this reachability question can be decided by primitive recursive functions. Ours is the first direct proof of this relationship, avoiding the traditional excursion to Diophantine equations, and thus providing a crisper picture of the relationship between Karp’s coverability tree and primitive recursive functions. In contrast, the question of finding the probability (according to standard chemical kinetics) of reaching a given target state turns out to be undecidable. Another way of saying this is that if we wish to distinguish states with zero probability of occurring from states with positive probability of occurring, we can do so, but if we wish to distinguish low probability states from high probability states, there is no general way to do so. Thus, if we wish to use a chemical reaction network to perform a computation, then if we insist that the network must always get the right answer, we will only be able to use networks with limited computational power, but if we allow just the slightest probability of error, then we can use networks with Turing-universal computational ability. This power of probability is quite surprising, especially when contrasted with the conventional computational complexity belief that BP P = P .

Load more