PAIRWISE COMPARISONS, INCOMPARABILITY AND PARTIAL ORDERS
Ryszard Janicki∗ Department of Computing and Software, McMaster University, Hamilton, Ontario, Canada L8S 4K1
Keywords: Decision theory, ranking, pairwise comparisons, weak and partial orders.
Abstract: A new approach to Pairwise Comparisons based Ranking is presented. An abstract model based on the concept of partial order instead of numerical scale is introduced and analysed. Many faults of traditional, numerical- scale based models are discussed. The importance of the concept of equal importance or indifference is discussed.
1 INTRODUCTION rankings. The strange and contradictory examples of rank- The method of Pairwise Comparisons could be traced ings obtained by AHP cited in the literature (see to Marquis de Condorcet 1785 paper (see (Arrow, (Dyer, 1990)) follow from the following sources: 1951)), was explicitly mentioned and analysed by • When two objects are being compared, interval Fechner in 1860 (Fechner, 1860), made popular by scales are mainly used; but later, when recipro- Thurstone in 1927 (Thurstone, 1927), and was trans- cal matrices are constructed the values of interval formed into a kind of (semi) formal methodology by scales are treated as the values of ratio scales. The Saaty in 1977 (called AHP, Analytic Hierarchy Pro- concept of a scale was never formally defined or cess, see (Dyer, 1990; French, 1986; Satty, 1977)). analysed. In principle it is based on the observation that while • The domain of real numbers, R , has two distinct ranking (“weighting”) the importance of several ob- interpretations, and these two interpretations are jects is frequently problematic, it is much easier when mixed up in the calculi provided by AHP. The first restricted to two objects. The problem is then reduced interpretation is standard, numbers describe quan- to constructing a global ranking (“weighting”) from tative values that can be added, multiplied, etc. the set of partially ordered pairs. The name “Pair- The second interpretation is that this is just a rep- wise” is slightly misleading, since for every triple resentation of a total order, and for instance 1.0 some consistency rules are required (Dyer, 1990; is “better” than 3.51 and “much better” than 13.6, Koczkodaj, 1993; Satty, 1986). For the orderings of a, but that’s it! In this interpretation one must be b, and c are made independently, it frequently occurs very careful when any arithmetic operations are that the consistency rules are not satisfied. To deal performed. with this problem various consistency concepts have • Even though the input numbers are “rough” and been introduced and analysed. imprecise the results are treated as if they were At present Pairwise Comparisons are practically very precise; “incomparability” is practically dis- identified with the controversial Saaty’s AHP. On one allowed. hand AHP has respected practical applications, on the other hand it is still considered by many (see (Dyer, In the author’s opinion the problems mentioned 1990)) as a flawed procedure that produces arbitrary above stem mainly from the following sources: • The final outcome is expected to be totally ordered ∗Partially supported by NSERC of Canada grant (i.e. for all a,b, either a < b or b > a),
297 Janicki R. (2007). PAIRWISE COMPARISONS, INCOMPARABILITY AND PARTIAL ORDERS. In Proceedings of the Ninth International Conference on Enterprise Information Systems - AIDSS, pages 297-302 DOI: 10.5220/0002363102970302 Copyright c SciTePress ICEIS 2007 - International Conference on Enterprise Information Systems
• Numbers are used to calculate the final outcome. ta ta .q ta J . {a} J A non-numerical solution was proposed and dis- . ? J . J cussed in (Janicki and Koczkodaj, 1996), but it as- t . b JJ^ . JJ^ sumed that on the initial level of pairwise compar- t tc .?q t tc ? Jb . {b,c} b isons, the answer could be only “a < b” or “a ≈ b”, tc . J . or “a > b”, i.e. “slightly in favour” and “very strongly J . ?t J^t .?q t? in favour” were indistinguishable. In this paper we d d {d} d provide the means to make such a distinction without <1 <2 ≺2 <3 using numbers. total or weak or total neither total The model presented below is an extension of the linear stratified nor weak one from (Janicki and Koczkodaj, 1996) with some influence due the results of (Fishburn, 1985; Janicki Figure 1: Various types of partial orders (represented as and Koutny, 1993). Hase diagrams). The total order ≺2 represents the weak order <2, The partial order <3 is nether total nor weak.
2 TOTAL, WEAK AND PARTIAL classification of our rankings (defined in Section 4) ORDERS complete, as in this paper we will not use much of their rich theory. An interested reader is referred to Let X be a finite set. A relation ⊳ ⊆ X ×X isa(sharp) (Fishburn, 1985; French, 1986). partial order if it is irreflexive and transitive, i.e. if Weak orders are often defined in an alternative a ⊳ b ⇒ ¬(b ⊳ a) and a ⊳ b ⊳ c ⇒ a ⊳ c, for all way, namely (Fishburn, 1985), ⊳ a,b,c ∈ X. A pair (X, ) is called a partially ordered • a partial order (X,⊳) is a weak order iff there ex- ⊳ ⊳ set. We will often identify (X, ) with , when X is ists a total order (Y,≺) and a mapping φ : X → Y known. such that ∀x,y ∈ X. x ⊳ y ⇐⇒ φ(x) ≺ φ(y). We write a ∼⊳ b if ¬(a ⊳ b) ∧ ¬(b ⊳ a), that is if a and b are either distinct incompatible (w.r.t. ⊳) or This definition is illustrated in Figure 1, let φ φ identical elements of X. : {a,b,c,d} → {{a},{b,c},{d}} and (a) = {a}, φ φ φ We also write (b) = (c) = {b,c}, (d) = {d}. Note that for all x,y ∈ {a,b,c,d} we have a ≈⊳ b ⇐⇒ {x | x ∼⊳ a} = {x | x ∼⊳ b}. x <2 y ⇐⇒ φ(x) ≺2 φ(y). Weak orders can easily be represented by step- The relation ≈⊳ is an equivalence relation (i.e. it is reflexive, symmetric and transitive) and it is called the sequences. For instance the weak order <2 from Figure 1 is uniquely represented by the step-sequence equivalence with respect to ⊳, since if a ≈⊳ b, there is nothing in ⊳ that can distinquish between a and b {a}{b,c}{d} (c.f. (Janicki and Koutny, 1993)). (see (Fishburn, 1985) for details). We always have Following (Fishburn, 1985), in this paper a ⊳ b is a ≈⊳ b ⇒ a ∼⊳ b, and one can show that (Fishburn, 1985): interpreted as “a is less preferred than b”, and a ≈⊳ b is interpreted as “a and b are indifferent”. a ≈⊳ b ⇐⇒ {x | a ⊳ x} = {x | b ⊳ x} ∧ {x | x ⊳ a} = {x | x ⊳ b} The preferable outcome of any ranking is a total ⊳ A partial order is (Fishburn, 1985) order. For any total order , ∼⊳ is the empty rela- tion and ≈⊳ is just the equality relation. A total order • total or linear, if ∼⊳ is empty, i.e., for all a,b ∈ has two natural models, both deeply embedded in the ⊳ ⊳ X. a b ∨ b a, human perception of reality, namely: time and num- • weak or stratified, if a ∼⊳ b ∼⊳ c ⇒ a ∼⊳ c, i.e. bers. Most people consider the causality relation and if ∼⊳ is an equivalence relation, its time related version “earlier than” as total orders, • interval, if for all a,b,c,d ∈ X, a ⊳ c ∧ b ⊳ d ⇒ even though their formal models are actually only in- a ⊳ d ∨ b ⊳ c, terval orders (Janicki and Koutny, 1993). Unfortunately in many cases it is not reasonable to • semiorder, if it is interval and for all a,b,c,d ∈ X, insist that everything can or should be totally ordered. ⊳ ⊳ ⊳ ⊳ a b ∧ b c ⇒ a d ∨ d c. We may not have sufficient knowledge or such a per- Evidently, every total order is weak, every weak order fect ranking may not even exist (Arrow, 1951). Quite is a semiorder, and every semiorder is interval. We often insisting on a totally ordered ranking results in mention semiorders and interval orders to make the an artificial and misleading “global index”.
298 PAIRWISE COMPARISONS, INCOMPARABILITY AND PARTIAL ORDERS
Weak (stratified) orders are a very natural gener- relation ∼<2 are {a}, {b,c}, and {d}, and <2 can be inter- which leads to the definition (Janicki and Koczkodaj, preted as a sequence {a}{b,c}{d}. 1996) below (for both weak an total orders). There are, however, cases where insisting on weak orders may not be reasonable. Physiophysical mea- A weak (or total) order ⊳w ⊆ X × X is a proper surements of perceptions of length, pitch, loudness, weak (or total) order extension of ⊳ if and only if : and so forth, provides other examples of qualitative ⊳ ⊳ (x y ⇒ x w y) and (x ≈⊳ y ⇒ x ∼⊳w y). comparisons that might be analysed from the perspec- If X is finite then for every partial order ⊳ its tive of semiorders and interval orders rather than the proper weak extension always exists. If ⊳ is weak, more precise but less realistic weak and total orders. than its only proper weak extension is ⊳ = ⊳. If ⊳ if The reader is referred to (Fishburn, 1985; Janicki and w not weak, there are usually more than one such exten- Koutny, 1993) for more details. In this paper we will sions. Various methods were proposed and discussed only use total, weak, and general partial orders. in (Fishburn, 1985). For our purposes, the best seem to be the method based on the concept of a global score function, which is defined as: 3 PARTIAL AND WEAK ORDER g⊳(x) = |{z | z ⊳ x}| − |{z | x ⊳ z}|. APPROXIMATIONS Given the global score function g⊳(x), we define g the relation ⊳w ⊆ X × X as Let X be a set of objects to be ranked. The problem g a ⊳ b ⇐⇒ g⊳(a) < g⊳(b). is that X is believed to be partially or weakly ordered w ⊳g but the data acquisition process is so influenced by Proposition 1 ((Fishburn, 1985)) The relation w ⊳ informational noise, imprecision, randomness, or ex- is a proper weak extension of a partial order . pert ignorance that the collected data R is only some Some other variations of g⊳ and their interpreta- relation on X. We may say that R gives a fuzzy pic- tions were analyzed in (Janicki and Koczkodaj, 1996). ture, and to focus it, we must do some pruning and/or From Proposition 2 it follows that every finite partial extending. Without loss of generality we may as- order has a proper weak extension. The well known sume that R is irreflexive, i.e. (x,x) 6∈ R. Suppose procedure “topological sorting”, popular in schedul- that R is not transitive. The “best” transitive approx- ing problems, guarantees that every partial order has + Ë∞ i a total extension, but even finite partial orders may not imation of R is its transitive closure R = i=1 R , where Ri+1 = Ri ◦R (c.f. (Fishburn, 1985)). Evidently have proper total extensions. Note that the total order R ⊆ R+ and R+ is transitive. The relation R+ may not ⊳t is a proper total extension of ⊳ if and only if the re- be irreflexive, but in such a case we can use the fol- lation ≈⊳ equals the identity, i.e a ≈⊳ b ⇐⇒ a = b. lowing classical result (which is due to E. Schroder,¨ For example no weak order has a proper total ex- 1895, see (Janicki and Koczkodaj, 1996)) . tension unless it is also already total. This indicates that while expecting a final ordering to be weak may Lemma 1 Let Q ⊆ X ×X be a transitive relation. De- be reasonable, expecting a final total ordering is of- fine: x 299 ICEIS 2007 - International Conference on Enterprise Information Systems 4 THE MODEL a given ranking. b b Let X be a finite set of objects to be “ranked”, and let Let ⊏w be a proper weak extension of ⊏ and let b b ≈, ⊏, ⊂, <, and ≺ be a family of disjoint relations ⊏w = ⊏w \ ⊂, on X. The interpretation of these relations is the fol- Rankw = (X,≈,⊏w,⊂,<,≺). lowing (compare (Satty, 1986)), a b : a and b are ≈ Proposition 2 The relational structure Rank = of equal importance, a ⊏ b : slightly in favour of b, w (X,≈,⊏ ,⊂,<,≺) is a weakly ordered ranking. a ⊂ b : in favour of b, a < b: b is strongly better, a ≺ b w : b is extremely better. The lists ⊏, ⊂, <, ≺ may be We will call Rankw = (X,≈,⊏w,⊂,<,≺) a weak shorter or longer, but not empty and not much longer order extension of Rank = (X,≈,⊏,⊂,<,≺). (due to limitations of the human mind(Satty, 1986)). We define the relations ⊏b, ⊂b, 300 PAIRWISE COMPARISONS, INCOMPARABILITY AND PARTIAL ORDERS “almost” a ranking. Only some pairs violate the rank- Table 1: The first pre-ranking. It is not a ranking, the gray ing axioms, the majority of pairs satisfy the ranking cells indicate problematic relations. axioms. Checking those axioms is in principle a triad A B C D E F G H analysis, and the major violators are usually easy to ⊏ < ⊐ detect. A ≈ ⊂ ≈ ⊃ ≈ B ⊃ ≈ > ≈ ≻ ⊐ ≈ < After finding the pairs that violate more axioms C ≈ < ≈ ⊂ ⊐ ⊏ < ⊏ than the other pairs (violations of axioms propagate, D ⊐ ≈ ⊃ ≈ ≻ ≈ ⊏ > but “innocent” pairs violate much less frequently), we E ⊂ ≺ ⊏ ≺ ≈ < ≺ ≈ can either: F ≈ ⊏ ⊐ ≈ > ≈ ⊂ ⊃ • Repeat the pairwise comparison process for those G > ≈ > ⊐ ≻ ⊃ ≈ ≻ pairs and change the judgment. Or H > ≺ ⊐ < ≈ ⊂ ≺ ≈ • Introduce a “moderator” process which will change the relationship between those pairs, to Table 2: The second (revised) pre-ranking which is a rank- satisfy the axioms. ing. The grey cells were revised. In both cases, the changes are usually minor, like from A B C D E F G H ⊂ to ⊏, etc. A ≈ ⊂ ≈ ⊏ ⊃ ≈ < ⊐ The resulting ranking Rank is usually not weak, in B ⊃ ≈ > ≈ ≻ ⊐ ≈ < most cases it is only semi- or intervally ordered, so, if C ≈ < ≈ ⊂ ⊐ ⊏ < ≈ we expect the outcome to be weak or total ranking, D ⊐ ≈ ⊃ ≈ > ≈ ⊏ > we need to compute Rankw. E ⊂ ≺ ⊏ < ≈ < ≺ ≈ Computing Rankw is an extension process which F ≈ ⊏ ⊐ ≈ > ≈ ⊂ ⊃ is expected to identify correctly the ordered pairs that G > ≈ > ⊐ ≻ ⊃ ≈ ≻ are not part of the data. The order identification power H > ≺ ≈ < ≈ ⊂ ≺ ≈ of weak extension procedures is substantial and vastly underestimated. If the ranked set of objects is, by its nature, expected to be totally ordered, the weak ex- shows that the only violations are caused by relations tension can detect it, even if the pairwise compari- between C and H, and D and E (grey cells in Table son process is not very precise, and often results in 1). The person was asked to repeat the comparison “indifference” (see example from Table 3 in the next of pairs (C,H), and (D,E). This time he produced a section). It is a serious error to attempt to find a slightly different answers. The new pre-ranking pre- total extension without going through a weak exten- sented by Table 2, is now a ranking. However, it is not sion process (see comments at the end of the previous a weakly ordered ranking. The partial orders ⊏b, ⊂b, chapter). In general, admitting incomparability on the 301 ICEIS 2007 - International Conference on Enterprise Information Systems Table 3: The third pre-ranking, where only ≈, < and ≺ a “moderator” might replace lower preferences by in- were recorded. It is a ranking. It can be obtained from difference, and check if the outcome is a ranking. If Table 2 by replacing ⊏ and ⊂ with ≈. it is, a weak extension may be constructed and on its bases the initial pre-ranking might be modified. Al- A B C D E F G H ternately, a problematic decision could be re-judged. A ≈ ≈ ≈ ≈ ≈ ≈ < ≈ B ≈ ≈ > ≈ ≻ ≈ ≈ < C ≈ < ≈ ≈ ≈ ≈ < ≈ D ≈ ≈ ≈ ≈ > ≈ ≈ > 6 FINAL COMMENTS E ≈ ≺ ≈ < ≈ < ≺ ≈ F ≈ ≈ ≈ ≈ > ≈ ≈ ≈ Apparently, all of the popular techniques based on G > ≈ > ≈ ≻ ≈ ≈ ≻ pairwise comparison principles suffer from many se- H > ≺ ≈ < ≈ ≈ ≺ ≈ rious problems and may lead to strange and contradic- tive results. These problems follow mainly from their G B F D B G treatment of imprecision, knowledge incompleteness t t t t t¨ t A ¡ ¨ and indifference (incomparability) (Dyer, 1990; Jan- A ¡ ¨ icki and Koczkodaj, 1996). CttA F tt D ¨¨ ¡A ¨ The method proposed in this paper does not use t?= ¨ t? = t? t? ¡ A numbers at all, it is entirely based on the concept of t?¡ AU t E H C A E H partial orders. It emphasizes and advocates using in- ≺b 302