284 Conditional Independence in Uncertainty Theories Prakash P. Shenoy School of Business University of Kansas Summerfield Hall Lawrence, KS 66045-2003USA [email protected] Abstract Anabstract framework that unifies variousuncertainty calculi is that of valuation-based systems [Shenoy 1989, Thispaper introduces the notions of independence 199Ia]. In VBS, knowledge about a setof variables is rep­ and conditionalindependence in valuation-based resentedby a valuation for that set of variables. Thereare systems (VBS). VBS is an axiomatic framework threeoperators in VBS that areused to makeinferences. capable of representingmany differentuncertainty These are called combination,marginalization, and re­ calculi.We definein dependenceand conditional moval. Combination representsaggregation of knowledge. independencein terms of factorization of the joint Marginalizationrepresents coarseningof knowledge. And valuation.The definitions of independence and removal represents disaggregation of knowledge. conditional independencein VBS generalizethe The framework of VBS is able to uniformly represent corresponding defmitions in probability theory. probability theory, Dempster-Shafer's belief-function the­ Our definitions apply not only to probability ory, Spohn'sepistemic-belie ftheory,and Zadeh'spossi­ theory, but also to Dempster-Shafer's belief-func­ bility theory. In this paper, we will develop thenotion of tion theory, Spohn's epistemic-belief theory, and independenceand conditional independence for variables in Zadeh's possibility theory. In fact, they apply to the framework of VBS. One advantage of this generalityis any uncertaintycalculi that fitin the framework that all resultsdeveloped here will apply uniformly to of valuation-based systems. all uncertaintycalculi that fit in the framework of VBS. Thus the results describedin this paper apply to, for example, 1 INTRODUCTION probabilitytheory, Dempster-Shafer's belief-function the­ The concept of conditional independencebetween two sub­ ory, Spohn's epistemic-belief theory,and Zadeh's possi­ sets of variables given a third has been extensively studied bility theory. in probability theory [Dawid 1979, Spohn 1980, What does it meanfor two disjoint subsetsof variables to Lauritzen 1989, Pearl1988, Smith 1989, Geiger 1990]. bein dependent? Intuitively,independence can definedbe in Theconcept of conditional independencein probability terms of factorization of the joint valuation. If t is a valu­ theory has been interpreted interms of relevance. If r, s ation for rus,then we say thatr and s are independent and t are disjoint subsets of variables, then to say that r with respectto t iff t factors into two valuations, one and s conditionallyindependent given means that the are t, whose domainonly involves r, and the other whose do­ conditional distributionof r, given values of sand t, are main only involves s. One implication of this is that if governed by the value of t alone-further information we are interestedin constructinga valuation for rvs, then about the value of s is irrelevant. independence of r and s allows us to constructthis valua­ The concept of conditionalindependence forvariables has tion by, frrst, constructing two valuations-one whose also been studied in Spohn's theory of epistemic beliefs domain involving only r and the other whose domain in­ [Spohn 1988, Hunter 1991]. However, the concept of in­ volving only s-and second, by simply combining the dependence for variables has not been studied inDempster­ two valuations to get the result. Shafer' s theory of belief functions [Dempster 1967, Shafer What does it mean for two disjoint subsets of variables to 1976] or in Zadeh's possibility theory [Zadeh 1979, beconditionally independentgiven a third disjoint subset? Dubois and Prade 1988].1 Conditional independencecan alsobe describedin terms of factorization of the joint valuation. Suppose't is a valua­ tion for rusul We say rand s are conditionally indepen­ 1 Dempster [1967], Shafer [1976, 1982, 1984, 1987, 1990], dent given t with tot iff factors and Smets [ 986] have defmed independence for belief respect the valuation t 1 into whosedomain functions, but not for variables on which belief functions are two valuations, one involves variables defined. Shafer [1976] has defmed independence for frames of in rut, and the other whose domain involves only vari­ discenunent, a concept further studied by Shafer, Shenoy and ables in sut. Mellouli [1987]. Belief functions in belief-function theory are analogs of probability functions in probability theory. Conditional Independence in Uncertainty Theories 285 bility theory,for example, a valuation for s is a function An outline of this paperis as follows. In section 2, we s�IR.where is the set of all numbers. describethe frameworlc of valuation-basedsystems (VBS). cr:'UI' R real TheVBS frameworkwas describedearlier in [Shenoy Zero Valuations For each s�X. there is at most one 1989, 1991a]. Herewe extendthe frameworkby defining valuation �sE '\J5 called the zero valuation for s. Let Z three new classes of valuations callednormal, proper nor­ denote { �s I S!;;;X}, the setof all zero valuations. Note mal, and positiveproper normal. Also,we introduce a that we arenot assuming zero valuations always exist. If new operatorcall ed removal,and defme some new axioms zero valuations do notexist, Z = 0. We call valuationsin for the removal operator. Shenoy [1991b]shows how '\J- Z nonzero valuations. Intuitively, a zero valuation probability Dempster-Shafer'stheory, belief-function the­ representsknowledge thatis internallyinconsistent. In ory, Spohn's epistemic-belief theory, and Zadeh's possi­ probability theory,for example, a zero valuation for sis bility theory fitin the framework of VBS. the valuation �s such that�5(x) = 0 for all xe 'UI' 5• In section3, we defmeindependence and conditional inde­ Proper Valuations For each subset s of$, there is a pendence for sets of variables. We show that thesedefmi­ subset P5 of V 5-{�5). We call the elementsof P5proper tions satisfy some well known propenies that have been valuations for s. Let P denote u{ P s I s�X ), the set of stated by Dawid [1979], Spohn [1980], Lauritzen [1989], all proper valuations. Intuitively, a propervaluation repre­ Pearl [1988], and Smith [1989] in the contextof probabil­ sentsknowledge that is partially coherent In probability ity theory. Using Pearl's terminology, the conditional in­ theory, for example,a propervaluation is a nonzerovalua­ dependence relation inVBS is a graphoid. Finally, in sec­ tion cr such that cr(x) � 0 for all xe cur5• tion 4, we make some concluding remarks. Proofs of all Normal Valuations For h s�X, there is another resultscan be found in [Shenoy 1991b]. eac subset'JI. s of V 5-{ �s}. We call the elements of 'JI.s nor­ mal valuations for s. Let 'J1. denoteu { 'JI.5 I �$ ) , the set 2 VALUATION-BASED SYSTEMS of all normal valuations. Intuitively, a normalvaluation In this section, we describe the framework of valuation­ representsknowledge that is partially coherent in a sense based systems (VBS). In aVBS, we representknowledge different from propervaluations. In probability theory,for by entities calledvariables andvaluations. We infer inde­ example, a normalvaluation is a nonzero valuation cr pendence relations using three operators calledcombina­ such that :E[cr(x) I XE curs l = 1. tion, marginalization, andremoval. We use these opera­ Proper Normal Valuations For each s�$, let� 5 tors on valuations. denote P 8n'J\.8• We call the elementsof 1\5proper nor­ Theframework ofVBS is describedin [Shenoy 1989, mal valuations for s. Let 1\ denote u(l\5I s�X) , the set 1991a]. The motivation there was to describea localcom­ of all proper normal valuations.Intuitively, a propernor­ putational methodfor computing marginals of the joint mal valuation representsknowledge that is completely co­ valuation. In this paper, we embellishthe VBS framework herent. by introducing threenew classesof valuations called nor­ Positive Proper Normal Valuations For each mal, proper normal, and positive proper normal, and by s�$, there is a subset 1\s + of 1\5• We call theelements introducinga new operatorcalled removal. Our motivation here is todefine independence and describe itsproperties. of 1\5+positive proper normal valuations for s. Let 1\ + denote u( 1\s + I sr;;;;$), the set of all positive proper Mr­ Variables We assume there is a fmite set X whose ele­ mal valuations. As we will la r, positive propernor­ ments called variables.Variables willbe denoted by see te are mal valuationsare proper normal valuations that have upper-case letters, etc. Subsets of X will be de­ X, Y, Z, unique identities. In probability theory, for example, a noted by lower-case letters, r, s, t, etc. positive propernormal valuationfor s is a propernormal Valuations For each s�$, there is a set 'If5• We call valuation crsuch that cr(x)> 0 for all xe cur5• the elements of '\15 valuations for s. Let V denote , Figure 1 shows the relations betweenthe different typesof u { V 5 I s�X } the set of all valuations. If cr is a valua­ valuations. As our defmitions, Z�'lf, ('V- ), tion for s, then we say that sis the domain of cr. per P� Z Valuations will bedenoted by lower-case Greek alphabets, 'J\.QV -Z), 1\ = Pn'n.,and 1\+!:;1\. p, t, cr, etc. Combination We assume there is a mapping Valuations are primitives inour abstractframework and, (£);'\Jx'\1 � 'JI. uZ, called combination, such that the as such, require no definition. But as we shall see shortly, following axioms are satisfied: they are objectswhich can be combined, marginalized,and (Cl) (Domain) If p anda arevaluations for r and s, re­ removed. A valuation for s represents some knowledge spectively, then p€£lcr is a valuation for rus. about variables in s. (C2) (Associative) p€£l(cr€£lt)= (pea)et. In probability theory, with eachvariable X, we associate a (C3) (Commutative) pea= aep. set '\Ifx called the framefor X. Also for each �X, we as­ sociate the set '\If5 = x { '\Ifx I Xe s) called theframe for s. (C4) (Zero)Suppose zero valuations exist, supposecr is a Elements of '\If5 are called configurationsof s.