PGM, Fall 2006 1 Problem Set #6 - Solution 1. A polytree is a singly-connected, directed graph (singly-connected graph is a graph that does not contain any undirected cycles). (a) Show that variable elimination on polytrees can be performed in linear time, assuming that the local probability models are represented as full tables. Specifically, for any polytree, describe an elimination ordering, and show that the complexity of variable elimination with your ordering is linear in the size of the network. Note that the linear time bound here is in terms of the size of the CPDs in the network, so that the cost of the algorithm grows exponentially with the number of parents of a node. Answer: The elimination ordering we will use is as follows: Consider the skeleton of the polytree S (same structure, only the edges are undirected - S is actually a tree). At each step choose a leaf l in the undirected tree S (degree = 1) and eliminate it. In the next step we will choose a leaf from the resulting graph (where l is eliminated). Note that in this process whenever we delete a leaf we are still left with a polytree, and therefore we will always have a leaf to remove (the process will not get stuck). Observe that when using this ordering, all intermediate factors involve only subsets of the variables in the original factors. To see this, we distinguish between two cases: i. The eliminated leaf l is a parent in the polytree - l has no parents and one child c. When eliminating l we eliminate two factors - Á(l) and Á(c; P ac) and create an intermediate factor Ã(c; P ac). The scope of this factor is equal to the scope of the factor in the original polytree Á(c; P ac). ii. The eliminated leaf l is a child with no children of its own and only one parent p - we eliminate the factor Á(l; p) and create an intermediate factor Ã(l; p), which is equivalent in scope to the original factor. To conclude, the scope of the largest intermediate factor created in the process is fully contained in the scope of an original factor, and thus the size of the largest intermediate factor is less than or equal to the size of the largest original factor. As we saw in class, the running time analysis of VE results in a running time which is linear in the size of the largest intermediate factor and linear in the number of variables. Therefore, since in the case of polytrees the size of the largest intermediate factor is less than or equal to the largest original factor, we get that the running time is linear in the size of the original network. (b) Extend your result from (a) to apply to cases where the CPDs satisfy independence of causal influence. Note that, in this case, the network representation is linear in the number of variables in the network, and the algorithm should be linear in that number. Answer: Recall that any symmetric independence of causal influence CPD can be decomposed in a chain-based manner, where each node has at most two parents (as was PGM, Fall 2006 2 shown in class). Our algorithm will use this transformation. Note that providing we start with a polytree this transformation results in a polytree, since the transformation only introduces edges in one direction, without creating cycles. In addition, the number of variables in the transformed network is linear in the number of variables in the original network (there are at most n ¡ 1 edges in the original polytree, and thus the transformation will result in a graph with O(n) variables). Thus, The transformed graph is a polytree, where the largest factor is of size 3 and the number of variables is linear in the number of variables in the original network. From the analysis of VE shown in class and the result of section (a) above, we conclude that the running time is linear in the size of the original network, as required. 2. Assume that we have constructed a clique tree T for a given Bayesian network graph G, and that each of the cliques in T contains at most k nodes. Now we construct a new network G0 by adding a single edge to the Bayesian network G (The edge can be added between any pair of nodes in the network, as long as it maintains acyclicity). What is the tightest bound you can provide on the maximal clique size in a clique tree T 0 for G0? Justify your response by explaining how to construct such a clique tree. Note: You do not need to provide the optimal clique tree T 0. The question asks for the tightest bound for any clique tree you can construct for G0. Answer: Note: The question is not completely clear. What is required is to show how to transform the given clique tree T into a legal clique tree for G0, and to bound the size of the transformed clique tree T 0 The bound on maximum clique size is k + 1. Suppose the user decides to add the edge X ! Y . We must update our clique tree to satisfy the family preservation and the running intersection property. (Note that any tree satisfying these two properties is a valid clique tree.) (a) Since the original clique satisfies the family preservation property, it must contain a clique C that contains Y and the original parents of Y . Adding X to C restores the family preservation property in the new clique tree. (b) To restore the running intersection property, we simply add X to all the cliques on a path between C and some other clique containing X. Since we added at most one node to each clique, the bound on maximum clique size is k + 1..