Journal of Articial Intelligence Research Submitted published LogarithmicTime Up dates and Queries in Probabilistic Networks Arthur L Delcher delchercsloyolaedu Computer Science Department Loyola Col lege in Maryland Baltimore MD Adam J Grove groveresearchnjneccom NEC Research Institute Princeton NJ Simon Kasif kasifcsjhuedu Department of Computer Science Johns Hopkins University Baltimore MD Judea Pearl pearllanaicsuclae du Department of Computer Science University of California Los Angeles CA Abstract Traditional databases commonly supp ort ecient query and up date pro cedures that op erate in time which is sublinear in the size of the database Our goal in this pap er is to take a rst step toward dynamic reasoning in probabilistic databases with comparable eciency We prop ose a dynamic data structure that supp orts ecient algorithms for up dating and querying singly connected Bayesian networks In the conventional algorithm new evidence is absorb ed in time O and queries are pro cessed in time O N where N is the size of the network We prop ose an algorithm which after a prepro cessing phase allows us to answer queries in time O log N at the exp ense of O log N time p er evidence absorption The usefulness of sublinear pro cessing time manifests itself in applications requiring near realtime resp onse over large probabilistic databases We briey discuss a p otential application of dynamic probabilistic reasoning in computational biology Introduction Probabilistic Bayesian networks are an increasingly p opular mo deling technique that has b een used successfully in numerous applications of intelligent systems such as realtime plan ning and navigation mo delbased diagnosis information retrieval classication Bayesian forecasting natural language pro cessing computer vision medical informatics and compu tational biology Probabilistic networks allow the user to describ e the environment using a probabilistic database that consists of a large number of random variables each corre sp onding to an imp ortant parameter in the environment Some random variables could in fact b e hidden and may corresp ond to some unknown parameters causes that inuence the observable variables Probabilistic networks are quite general and can store information such as the probability of failure of a particular comp onent in a computer system the prob c AI Access Foundation and Morgan Kaufmann Publishers All rights reserved Delcher Grove Kasif Pearl ability of page i in a computer cache b eing requested in the near future the probability of a do cument b eing relevant to a particular query or the probability of an aminoacid subsequence in a protein chain folding into an alphahelix conformation The applications we have in mind include networks that are dynamically maintained to keep track of a probabilistic mo del of a changing system For instance consider the task of automated detection of p owerplant failures We might rep eat a cycle that consists of the following sequence of op erations First we p erform sensing op erations These op erations cause up dates to b e p erformed to sp ecic variables in the probabilistic database Based on this evidence we estimate query the probability of failure in certain sites More precisely we query the probability distribution of the random variables that measure the probability of failure in these sites based on the evidence Since the plant requires constant monitoring we must rep eat the cycle of senseevaluate on a frequent basis A conventional nonprobabilistic database tracking the plants state would not b e appropriate here b ecause it is not p ossible to directly observe whether a failure is ab out to o ccur On the other hand a probabilistic database based on a Bayesian network will only b e useful if the op erationsup date and querycan b e p erformed very quickly Because realtime or near realtime is so often necessary the question of doing extremely fast reasoning in probabilistic networks is imp ortant Traditional nonprobabilistic databases supp ort ecient query and up date pro cedures that often op erate in time which is sublinear in the size of the database eg using bi nary search Our goal in this pap er is to take a step toward systems that can p erform dynamic probabilistic reasoning such as what is the probability of an event given a set of observations in time which is sublinear in the size of the probabilistic network Typically sublinear p erformance in complex networks is attained by using parallelism This pap er relies on prepro cessing Sp ecically we describ e new algorithms for p erforming queries and up dates in b elief networks in the form of trees causal trees p olytrees and join trees We dene two natural database op erations on probabilistic networks UpdateNode Perform sensory input mo dify the evidence at a leaf no de single variable in the network and absorb this evidence into the network QueryNode Obtain the marginal probability distribution over the values of an arbitrary no de single variable in the network The standard algorithms introduced by Pearl can p erform the QueryNode op er ation in O time although evidence absorption ie the UpdateNode op eration takes O N time where N is the size of the network Alternatively one can assume that the UpdateNode op eration takes O time by simply recording the change and the Query Node op eration takes O N time evaluating the entire network In this pap er we describ e an approach to p erform b oth queries and up dates in O log N time This can b e very signicant in some systems since we improve the ability of a system to resp ond after a change has b een encountered from O N time to O log N Our approach is based on prepro cessing the network using a form of no de absorption in a carefully structured way to create a hierarchy of abstractions of the network Previous uses of no de absorption techniques were rep orted by Peot and Shachter Queries Updates in Probabilistic Networks We note that measuring complexity only in terms of the size of the network N can overlook some imp ortant factors Supp ose that each variable in the network has domain size k or less For many purp oses k can b e considered constant Nevertheless some of the algorithms we consider have a slowdown which is some p ower of k which can b e b ecome signicant in practice unless N is very large Thus we will b e careful to state this slowdown where it exists Section considers the case of causal trees ie singly connected networks in which each no de has at most one parent The standard algorithm see Pearl must use O k N time for either up dates or for retrieval although one of these op erations can b e done in O time As we discuss briey in Section there is also a straightforward variant on this algorithm that takes O k D time for b oth queries and up dates where D is the height of the tree We then present an algorithm that takes O k log N time for up dates and O k log N time for queries in any causal tree This can of course represent a tremendous sp eedup esp ecially for large networks Our algorithm b egins with a p olynomialtime prepro cessing step linear in the size of the network constructing another data structure which is not itself a probabilistic tree that supp orts fast queries and up dates The techniques we use are motivated by earlier algorithms for dynamic arithmetic trees and involve caching su cient intermediate computations during the up date phase so that querying is also relatively easy We note however that there are substantial and interesting dierences b etween the algorithm for probabilistic networks and those for arithmetic trees In particular as will b e apparent later computation in probabilistic trees requires b oth b ottomup and topdown pro cessing whereas arithmetic trees need only the former Perhaps even more interest ing is that the relevant probabilistic op erations have a dierent algebraic structure than arithmetic op erations for instance they lack distributivity Bayesian trees have many applications in the literature including classication For instance one of the most p opular metho ds for classication is the Bayes classier that makes indep endence assumption on the features that are used to p erform classication Duda Hart Rachlin Kasif Salzb erg Aha Probabilistic trees have b een used in computer vision HelOr Werman Chelb erg signal pro cessing Wilsky game playing Delcher Kasif and statistical mechanics Berger Ye Nevertheless causal trees are fairly limited for mo deling purp oses However similar structures called join trees arise in the course of one of the standard algorithms for computing with arbitrary Bayesian networks see Lauritzen and Spiegelhalter Thus our algorithm for join trees has p otential relevance to many networks that are not trees Because join trees have some sp ecial structure they allow some optimization of the basic causaltree algorithm We elab orate on this in Section In Section we consider the case of arbitrary p olytrees We give an O log N algo rithm for up dates and queries which involves transforming the p olytree to a join tree and then using the results of Sections and The join tree of a p olytree has a particularly p simple form giving an algorithm in which up dates take O k log N time and queries p O k log N where p is the maximum number of parents of any no de Although the p constant app ears large it must b e noted that the original p olytree takes O k N space merely to represent if conditional probability tables are given as explicit matrices Delcher Grove Kasif