A Distributed Protocol for Collective Decision-Making in Combinatorial Domains

Swinburne Research Bank http://researchbank.swinburne.edu.au

Author: Li, Minyi; Vo, Quoc Bao; Kowalczyk, Ryszard Title: A distributed protocol for collective decision- making in combinatorial domains Editor: Takayuki Ito, Catholijn Jonker, Maria Gini, and Onn Shehory Conference name: 12th international conference on Autonomous agents and multi-agent systems (AAMAS 2013) Conference location: Saint Paul, Minnesota, United States Conference dates: 06-10 May 2013 Series title and volume: Vol. 2 Publisher: International Foundation for Autonomous Agents and Multiagent Systems Year: 2013 Pages: 1117-1118 URL: http://hdl.handle.net/1959.3/378574

Copyright: Copyright © 2013. International Foundation for Autonomous Agents and Multiagent Systems. All rights reserved.This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, vol. 2, 2013, http ://dl.acm.org/citation.cfm?id=2484920.2485099&c oll=DL&dl=ACM

This is the author’s version of the work, posted here with the permission of the publisher for your personal use. No further distribution is permitted. You may also be able to access the published version from your library.

The definitive version is available at: http://dl.acm.org/citation.cfm?id=2484920.24850 99&coll=DL&dl=ACM

Swinburne University of Technology | CRICOS Provider 00111D | swinburne.edu.au

Minyi Li, Quoc Bao Vo, and Ryszard Kowalczyk

Swinburne University of Technology

Abstract. In this paper, we study the problem of collective decision- making over combinatorial domains, where the set of possible alternatives is a Cartesian product of (finite) domain values for each of a given set of variables, and these variables are not preferentially independent. Due to the large alternative space, most common rules for preference aggregation cannot be directly applied to compute a winner. In this paper, we focus on a particular rule, namely Smith/Minimax. We introduce a distributed protocol for collective decision-making, which enjoys the following desirable properties: i) the final decision chosen is guaranteed to be a Smith member, and is sufficiently close to the Smith/Minimax candidate; ii) it enables distributed decision-making and works under incomplete information settings; iii) it significantly reduces the amount of dominance testings (individual outcome comparisons) that each agent needs to conduct, as well as the number of pairwise comparisons; iv) it is sufficiently general and does not restrict the choice of preference representation languages.

1 Introduction

In this paper, we focus on the problem of collective decision-making in combinatorial domains, where the set of candidates is a Cartesian product of finite value domains for each of a given set of possibly inter-dependent variables. For instance, a research group plan to order several personal computers (PCs) and the group members need to decide on a common group PC configuration. Consider for example that the common PC configuration to be agreed on is composed of the processor, hard disk, RAM and operating system, with a choice of six possible options for each, which makes 64 = 1296 candidates. The agents’ preferences on variables and the collective decisions are not independent, because, perhaps, the preferred operating systems may depend on the given processor type. The inter-dependency between variables and the exponential alternative space in such domains lead to a big challenge of computing the social aggregation rules. When the domain is combinatorial, a straightforward way to compute a social decision rule might be using issue-by-issue (a.k.a. seat-by-seat) sequential election. However, as soon as the variables are not preferentially independent, it is very likely that deciding on the issues separately will lead to suboptimal choices [4]. The only way to avoid inefficiency is ”deciding on combinations [of variable values]” [4]. Nonetheless, most common aggregation methods need a number of operations at least linear (sometimes even quadratic or exponential) 2 Minyi Li, Quoc Bao Vo, Ryszard Kowalczyk in the number of possible alternatives [9]. When the set of alternatives has a combinatorial structure and the agents’ preferences are described in a compact representation language, generating the whole relation from those inputs and directly applying such rules to compute a collective decision is impractical. Collective decision-making in combinatorial domains has been studied in a number of papers. Many of them consider logical or structured preferences, and mainly focus on analysing negative complexity results or finding reason- able restrictions on the preference relations such that sequential winners and direct winners always coincide, e.g., [10, 13, 12]. Some later works [12, 11] relax those restrictions and introduce the notion of local Condorcet winners 1. They also propose (and implement) algorithms for computing them. Nevertheless, there is still another problem that has not been addressed: with a limited number of agents and a large alternative space, collective preference cycles (i.e., paradoxes) generally occur. Therefore, investigating techniques to generate a “good” alternative in the presence of paradoxes is certainly of interest in the field. In this paper, we address the aforementioned problems by proposing a protocol that enables the agents to identify efficient collective decisions. The proposed protocol enjoys the following desirable properties: i) Regarding the social properties, the final chosen collective decision is guaranteed to be a Smith member. Moreover, we show experimentally that the collective decision chosen by the proposed protocol is sufficiently close to the Smith/Minimax candidate i.e., the Minimax candidate in the Smith set. ii) From the perspective of computation and communication, the proposed protocol enables the agents to make proposals on partial assignments (i.e., an assignment of a subset of variables) until all variables are assigned. As a result, it does not require counting candidates as employed in a majority of simple aggregation mechanisms [8]. It reduces the total number of alternatives that each agent needs to consider in practice, and requires significantly fewer outcome comparisons compared to an exhaustive search in most decision-making instances. iii) Last but not least, the proposed protocol supports real-world applications. It works under incomplete information settings and does not require the agents to submit their preferences; and thus, it can be implemented in a distributed manner. Moreover, the proposed protocol is sufficiently general and applicable to most preference representation languages in combinatorial domains. There are no restrictions on the agents’ preferences: their preference orders could be complete or partial, and they can extend certain structural properties, e.g., CP-nets, soft constraints, etc.

2 Preliminaries

2.1 Combinatorial domains and preferences

Let V = X1,..., Xm be a set of m variables in a combinatorial domain, where each variable{ X takes} values in a local variable domain D(X). A variable X is 1 A local Condorcet winner is an alternative that beats all its neighbours. And a neigh- bour of an alternative is another alternative that differs only on a single variable from that alternative. Collective decision-making in combinatorial domains 3 binary if D (X) = x, x¯ . An alternative is uniquely identified by the combination of variable values.{ Hence,} the possible alternatives (outcomes) are D(X ) 1 × · · · × D(Xm), denoted by O. If X = X ,..., X and X V, with σ < < σ then DX { σ1 σ` } ⊆ 1 ··· ` denotes DX DX and x denotes an assignment of variable values to X, σ1 × · · · × σ` i.e., x DX. If X = V, x is a complete assignment (corresponds to an alternative); otherwise∈ x is called a partial assignment. We allow concatenations of disjoint sets of variable values: for instance, let X = A, B , Y = C, D , x DX and x = ab¯, { } { } ∈ y DY and y = cd¯, then xye¯ denotes an assignment abc¯ d¯e¯. If X Y = V, we call xy∈a completion of assignment x. We denote by Comp(x) the set∪ of completions of x. A preference order is a reflexive, transitive and antisymmetric relation on (recall that is antisymmetric iff x, y , x y and y x implies x = y). X denotes the strict relation induced∀ from∈ X: x y iff x yand not y x. An order is complete (linear order) iff , eitherx y or y xholds for all x, y ; otherwise, it is a partial order, i.e., there existx, y , such that neither x∈ Xy ∈ X nor y x holds, written as x Z y (incomparable). Since direct assessment of the preference relations in combinatorial domains is usually infeasible, several compact languages have been proposed to avoid the exponential blow up. Some of these languages are cardinal, e.g., utility networks [1] and soft constraints [7, 5]; some others are purely qualitative, e.g., CP- nets [2] and CI-nets [3]. Some representation models are based on conditional preferences, for instance CP-nets [2] and its variants; some others are based on propositional logic (or possibly a fragment of it), for instance prioritized goals, distance-based goals, weighted goals, bidding languages for combinatorial auctions. For brevity, we don’t present an exhaustive list here. The reader is referred to Lang ([9]) for a detailed survey of compact preference representation languages in combinatorial domains. In the following, we only briefly describe CP-nets, a widely studied compact graphical language for representing qualitative preferences in combinatorial domains. A Conditional Preference Network (CP-net) [2] over a set of variables V is an annotated directed graph , in which nodesN stand for the variables. Each node X is annotated with aG conditionalN preference table CPT(X), which associates a strict total order over DX given every combination values u of parents Pa(X) in . For instance, let V = A, B, C , with variables all three being binary, beG aN CP-net whose structural{ part is} the directed acyclic graph = V, (C, AN), (C, B), (A, B) . This means that the agent’s preference over the GvaluesN { of C is unconditional,} preference over the values of A (resp. B) is fully determined given the value of C (resp. the values of C and A). The preference statements contained in the conditional preference tables are written with the usual notation, that is, ac¯ : b¯ b means that when A = a¯ and C = c, then B = b¯ is B ac¯ preferred to B = b. Such a relation is denoted by b¯ | b. A CP-net generally induces a partial preorder (written as ), such that N N x, x0 DX, u DPa(X), and z DV Pa(X) X , xuz x0uz if and only if ∀ X u ∈ ∀ ∈ ∀ ∈ − −{ } N x | x0[2]. In most literature, the dependency graph is assumed to be acyclic. When is acyclic, is transitive and antisymmetric,GN i.e., a strict GN N 4 Minyi Li, Quoc Bao Vo, Ryszard Kowalczyk partial order. A linear order extends a CP-net , if it extends the partial order that induces. Finally,we note that CP-netsN are not fully expressive, because someN preferenceN relations are not expressible by CP-nets, however, any linear order extends some (possibly cyclic) CP-net.

2.2 Collective decision-making

Given a set of alternatives O, an aggregation mechanism can be used to ag- gregate multiple agents’ preferences and finally decide on one as the collective decision. In order to determine a “good” collective choice, Condorcet criterion has usually been used. A Condorcet winner is an alternative that when compared with every other candidate, is preferred by more agents. A preference aggregation method satisfies the Condorcet criterion, i.e., is Condorcet-consistent, if it chooses the Condorcet winner when one exists. When the Condorcet winner exists, it is unique. However, it is possible for a paradox to form, known as the Condorcet Paradox, in which collective preferences can be cyclic (i.e. not transitive), even if the preferences of individual agents are not. We denote “ i” the preference of an agent i and “ maj” the majority preference. For instance, consider three agents (agent 1, agent 2 and agent 3) with the following preferences respectively: o o0 o00; o0 o00 o; and o00 o o0. In this case, 1 1 2 2 3 3 o maj o0 as agent 1 and 3 prefer o to o0, o0 maj o00 as agent 1 and 2 prefer o0 to o00, and o00 maj o as agent 2 and 3 prefer o00 to o. Therefore, there is no Condorcet winner in this example. When a Condorcet winner does not exist, Smith criterion is often considered, which states that the social choice mechanism should always choose the alternative from the Smith set. Smith set is the smallest set of alternatives that every member of the set is pairwise preferred to every alternative not in the set. Smith set is originally introduced to make group decisions for complete tournament graphs. However, the definitions is easily extended to handle general graphs (incomplete tournaments). Smith criterion implies Condorcet criterion, since if there is a Condorcet winner, then it is the only Smith member. Additionally, Smith criterion also implies Condorcet loser criterion, i.e., if a candidate loses to every other candidate in pairwise comparison, it should never be chosen as the final collective decision. We notice that the Smith set can have more than one candidate, either because of pairwise ties or incomparability, or because of cycles, such as that in a Condorcet’s paradox. In social choice theory, Minimax Condorcet, also called the Simpson method, is one of the most studied Condorcet-consistent methods, which chooses an op- tion that is defeated by the fewest agents in its worst defeat. However, Minimax method fails Smith criterion (and thus also fails Condorcet loser criterion). A straightforward way to address this issue is to apply Minimax rule to only the alternatives inside the Smith set, called the Smith/Minimax candidate. From a computational point of view, computing the Smith set from a directed graph (e.g, a majority graph) is polynomial in the number of alternatives (i.e., exponential in the number of variables). Moreover, as shown by Lang [8], under Collective decision-making in combinatorial domains 5 an assumption of polynomial outcome comparison, checking whether an alternative is a Minimax candidate alone (not necessarily a Smith member) in a combinatorial domain is NP-hard. Therefore, checking whether an alternative is a Smith/Minimax candidate or computing a Smith/Minimax candidate is likely to be harder than NP or CoNP problems. Furthermore, in many preference representation languages, e.g., CP-nets, outcome comparison is computationally difficult [6]. Therefore, finding good algorithms or protocols to compute or ap- proximate such a collective decision in combinatorial domains is certainly an important and promising research topic.

3 The MDTreeS/M protocol

We now introduce our proposed protocol, called MDTreeS/M, for collective decision-making in combinatorial domains.

3.1 The framework

Majority decision tree (MDTree) For a collective decision-making problem over m variables V = X1,..., Xm , we conceptualize the assignment of the variable values as a tree,{ known as the} majority decision tree (MDTree). A MDTree assigns values to variables level by level. Let k be the maximum size of the variable domain: X V, D (X) k, the MDTree is then a k-ary tree. The depth of is∀m with∈ the| root| ≤ being at depth 0.T Assume an order over T variables σ = Xσ1 > > Xσm is given for making the collective-decision. Then a MDTree is generated··· following σ, and each level ` considers the value assigns T to variable Xσ` . A node Φ at depth ` represents a unique value assignment, denoted by Assg , to the set of variables X ,..., X specified by the path Φ { σ1 σ` } from the root to Φ, i.e., AssgΦ D(Xσ1 ) D(Xσ` ). The root node represents an empty assignment, when a node∈ Φ at× depth · · · × m represents a unique alternative (outcome), as every variable has been assigned a value ( Assg = m). | Φ| The procedure starts from a root node with an empty assignment. The MDTree is then iteratively created in a top-down process following the order T σ = Xσ1 > > Xσm , where in each iteration, each agent can propose on a leaf node in .··· Note that different agents may propose on different nodes (possibly at diTfferent depths) in the same iteration. Once a leaf node Φ at depth ` (` < m) receives a majority of the participating agents’ proposals, the subtree of Φ will be expanded with possible values assigned to the next variable Xσ`+1 . The child nodes of Φ will then be explored by the agents and be available for them to consider making proposals on in the next iteration. Notice that we say a node Φ receives a majority of agents’ proposals, these proposals may arrive at Φ in different iterations. We formally define open nodes and winning nodes in a MDTree. Definition 1 (Open node). A node Φ is marked as open if and only if it is a leaf node at depth ` that ` < m and Φ receives a majority of agents’ proposals. 6 Minyi Li, Quoc Bao Vo, Ryszard Kowalczyk

Deﬁnition 2 (Winning node). A node Φ is marked as a winning node if and only if it is a leaf node at depth ` = m (i.e., the deepest level), and Φ receives a majority of agents’ proposals.

A winning node, in essence, represents a complete assignment (alternative) that receives a majority of agents’ proposals. Notice that once an open node being expanded, it is not an open node any more, as it is no longer a leaf node. Moreover, if a node is marked as a winning node, it will be put in a set W and may be chosen as a node for the agents to converge. That means, if a winning node Φ∗ is chosen from W, the agents will considered it as a reference point (i.e., status quo), and an agent will continue to propose on a node Φ in an iteration if and only if it is better than Φ∗ (Φ will potentially bring him to a better outcome Assg than Φ∗ ). While when there is no winning node in the MDTree, Φ∗ = Null. This can be considered as a disagreement or a failure of the collective decision- making. We assume that all the agents prefer any alternatives to this failure. Therefore, all the agents will keep making proposals until they identify at least one winning node. And once they have identiﬁed one, they try to converge to that node unless they ﬁnd some other more socially preferred nodes. Finally we know that if a node is marked as a winning node, it will not be feasible for the agents to make proposals on during later iterations. Therefore, in each iteration, a node Φ in is feasible for an agent i (i 1,..., n ) to make a proposal on if: T ∈ { } – it is a leaf node and not marked as a winning node; and – agent i has not proposed on Φ.

In the following sections, we denote Li as the set of leaf nodes that are currently feasible for agent i to make proposals on.

Best possible alternative and BATCD Strategy In this paper, we consider a distributive implementation and incomplete information settings. There is no prior information about the preferences of the other agents. Moreover, the proposals made by an agent during the decision-making process are invisible to other agents. Therefore, in each iteration, the only information that each agent i (i 1,..., n ) obtains is Li and a reference point Φ∗. Based on Li and Φ∗, agent i will∈ { decide} on whether or not he will make a proposal, and on which node he will propose. Given Φ∗, agent i will propose only if there is some node Φ in Assg Li that will potentially lead agent i to a more preferred outcome than Φ∗ . Assume Φ∗ , Null, then this at least requires that the most preferred alternative Assg from the subtree of Φ is better than Φ∗ for agent i. Therefore, we investigate the optimistic alternative that a node Φ can lead an individual agent to. At each node Φ Li, agent i (i 1,..., n ) has a best possible alternative ∈ ∈ { } (BPA), denoted by BPAi(Φ). It is the optimistic alternative that agent i can obtain from the subtree of Φ, i.e., the best (most preferred) alternative for agent i among the completions of Assg (Comp(Assg )). For instance, assume V = A, B, C , all Φ Φ { } variables are binary: DA = a, a¯ , DB = b, b¯ and DC = c, c¯ . Consider a node Φ with assignment a (resp. a¯b¯),{ the} completions{ } of Assg {is abc} , abc¯, abc¯ , ab¯c¯ (resp. Φ { } Collective decision-making in combinatorial domains 7 a¯bc¯ , a¯b¯c¯ ). If an agent i’s preference is represented by a CP-net: c : a¯ a, c¯ : a a¯; b{ b¯; and} c c¯. Then the BPA of Φ is abc (resp. a¯bc¯ ), because as induced from that CP-net, abc 1 abc¯, abc 1 abc¯ and abc 1 ab¯c¯ (resp. a¯bc¯ 1 a¯b¯c¯). Moreover, the BPA of the root node of agent i corresponds to the best (most preferred) alternative of agent i in the entire alternative space. With the aforementioned CP-net, it is abc¯ . BPA Assg Consequently, if Φ Li, such that i(Φ) i Φ∗ , agent i will propose in the current iteration.∃ Moreover,∈ during the decision-making process, an agent can always go backtrack to other nodes as long as the BPA of the current node is less preferred than that of another node. Therefore, we can assume that the agents use an optimistic strategy for making proposals on the MDTree. That means, if agent i decides to propose, he will choose a node Φ that BPAi(Φ) is the most preferred (non-dominated) among those of the nodes in Li. Consequently, we consider the following BATCD strategy (Best alternative to a collective decision):

Deﬁnition 3 (BATCD strategy). Given a list of feasible leaf nodes Li and a reference point Φ∗ in the current iteration, agent i (i 1,..., n ) will make a proposal on a node ∈ { } Φ Li only if: ∈

– Φ0 Li, BPAi(Φ0) BPAi(Φ); and ∀ ∈ – Φ∗ = Null or BPAi(Φ) i Assg . Φ∗ As an implementation, each agent i can maintain a priority queue consisting of nodes, known as the fringe. A fringe is a sorted list of feasible nodes Li w.r.t. agent i’s preferences. That is, the more preferred the BPAi(Φ) of a node Φ is, the higher priority it is in agent i’s fringe. In each iteration, given the reference point Φ∗ (possibly be Null), agent i consider the highest priority nodes. Once a node in is expanded, each agent will add the new child nodes into his fringe followingT his preference over the BPAs of the nodes. Moreover, the nodes that become infeasible will be removed from his fringe, which includes the node that he proposes on, the created open nodes and winning nodes in the current iteration. We emphasize here that the agents’ preferences can be represented by most compact languages in combinatorial domains, and diﬀerent agents can use diﬀerent representation languages. Moreover, the agents do not need to reason about the entire preference order during the decision-making process, but only need to conduct a few outcome optimization queries (as to compute the BPAs) and dominance queries when adding new nodes into their fringes.

Remark 1. When the preference ordering of an agent is strict and total, or extends an acyclic CP-net, the best possible alternative (BPA) of a node in the MDTree is unique. However, when an agent’s preference contains indifference and incomparability, it is possible that there may exist more than one BPAs of a single node, which are indifferent to or incomparable with each other. Even when each node has a unique BPA, it may also be the case that the BPAs of two nodes are indifferent or incomparable. In this work, we assume: 8 Minyi Li, Quoc Bao Vo, Ryszard Kowalczyk

Protocol 1: MD-TreeS/M 1 [Protocol] Input: σ = X > > X , an order over variables; σ1 ··· σm Output: Assg , a collective decision (outcome); Φ∗ 2 Initiate a MDTree with a root node; T 3 foreach x D(X ) do σ1 ∈ σ1 4 Φ CreateNode(Φ, x ); AppendTo( , Φ); ← σ1 T 5 foreach agent i A do ∈ 6 add Φ to Li w.r.t. his preferences; 7 end 8 end 9 Φ∗ Null; W ; ← ← ∅ 10 repeat // Step 1 11 foreach agent i A do ∈ 12 Φ MostPreNode(i, Li); ← 13 if Φ∗ = Null or BPAi(Φ) i Assg then Φ∗ 14 propose on node Φ; remove Φ from Li 15 end 16 end 17 foreach agent i A do ∈ 18 remove all created open nodes and winning nodes from Li 19 end // Step 2 20 foreach open node Φ do 21 X NextVar(σ,Φ); σk ← 22 foreach x D(X ) do σk ∈ σk 23 Φ0 CreateNode(Φ, x ); AppendTo( , Φ0); ← σk T 24 foreach agent i A do ∈ 25 add Φ0 to Li w.r.t. his preferences; 26 end 27 end 28 end // Step 3 29 add all winning nodes created to W; 30 Φ∗ Smith/Minimax(W); ← 31 until all agents do not propose in Step 1; 32 return Assg ; Φ∗

– given a reference point Φ∗, and a most preferred node Φ Li, agent i (i 1,..., n ) will propose on Φ if at least one of the BPA of Φ ∈is better than Assg∈ { } Φ∗ . If more than one most preferred node has this property, then:

– agent i will give higher priority to a most preferred node Φ than another most preferred node Φ0 if Φ is at a deeper level than Φ0, i.e., Assg > Assg . | Φ| | Φ0 | Collective decision-making in combinatorial domains 9

3.2 The process of collective decision-making We now present the process of collective decision-making using the proposed MDTreeS/M protocol. From a broad view, the proposed MDTreeS/M protocol is an iterative procedure, where the agents iteratively make proposals on the nodes in and makes the nodes in being expanded. Once there exist one or some winningT nodes, the agents thenT try to converge to one of the them, denoted by Φ∗, until they could not find any other alternatives that is socially preferred to Φ∗, then they stop making proposals, and the process ends with Assg a collective decision Φ∗ . Notice that the reference point Φ∗ is originally Null (representing a disagreement), and is not fixed during the decision-making process, as the agents may identify other more socially preferred winning nodes. Protocol 1 provides an algorithmic implementation of the collective decision- making process, as well as the ongoing update in the agents’ fringes. The input of the protocol is an order σ = Xσ1 > > Xσm over the domain variables following which the MDTree will be formed.··· Without any knowledge about the agents’ preferences, this order can be randomly chosen or decided by a neutral third party. Notice that different order chosen to generate the MDTree may lead to a different final decision, but as we will prove later, the final decision chosen by MDTreeS/M protocol is guaranteed to be a Smith member. Nonetheless, it may also affect the size of the MDTree, and thus, the total number of alternatives that each agent needs to consider during the decision-making process (i.e., the total number of leaf nodes being explored in the MDTree) . Initially, a MDTree is initiated with a root node. We create and add all T possible D(Xσ1 ) child nodes (each branch assigns a distinct value to the variable X ) to | (line 2–7).| Each agent i (i 1,..., n ) adds these nodes into his fringe σ1 T ∈ { } w.r.t his preferences. The reference point Φ∗ is assigned Null and the set of winning nodes W is assigned (line 5). Each iteration of the MDTreeS/M protocol then performs the following three∅ steps: Step 1: Offering. Each agent i (i 1,..., n ) propose on a feasible leaf node ∈ { } Φ Li and remove Φ from Li (line 10–15), unless: –∈agent i has made a proposal on every feasible leaf node, i.e., his fringe currently is empty (Li = ); or ∅ Assg – none of the node in Li has a BPA that is preferred to Φ∗ : Φ BPA Assg ∀ ∈ Li, i(Φ) i Φ∗ . After this step, some leaf nodes in will be open (if it receives a majority of agents’ proposals); and some leafT nodes will become a winning node (if it is at depth m and receives a majority of agents’ proposals). Each agent then removes all recreated open nodes and winning nodes from his fringe Li, as they are no longer feasible for making proposals on. Step 2: Expanding. We expand all the open nodes created in the current iteration (line 19–27). For an open node Φ at depth ` (` < m), let k = ` + 1, it will

be expanded with every possible value xσk of the next variable Xσk . Each agent i then add these child nodes to Li. Step 3: Choosing Φ∗. All the winning nodes created will be added into a set W (line 28). Then in the third step, we run a pairwise comparison among 10 Minyi Li, Quoc Bao Vo, Ryszard Kowalczyk

(a) Agent 1

(b) Agent 2

Fig. 1. Agents’ preferences root a a ф ф1 2 ab c abc ab c ab c abc abc 4 1 1 1 P1 P2 P1 P3

b b b b ф ф7ф 8ф 3 4 ab c ab c ab c abc ab c abc ab c abc abc abc abc abc 5 6 5 2 2 2 P1 P2 P2 P1 P3 P2 c c c c ф9 ф10ф 5ф 6 ab c ab c ab ca bc ab c ab c ab c ab c ab c abc abc abc 7 8 7 3 9 3 3 P1 P1 P2 P1 P3 P2 P3

Fig. 2. MDTree T the alternatives in W, and select a Smith/Minimax candidate of W to be the reference point Φ∗ for convergence (line 29). As such, in the next iteration, an agent will continue to propose only if there exists any possibilities that Assg lead him to a more preferred alternative than Φ∗ . Notice that during the entire decision-making process, only the alternatives in W require direct pairwise comparisons . After Step 3, the protocol continues and goes back to Step 1, until every agent stops making proposals (either because his fringe is currently empty or Assg there is nothing better than Φ∗ ). In such case, the process ends, and returns Assg the alternative Φ∗ as the ﬁnal collective decision (line 30-31).

3.3 Illustrations Now, we demonstrate the execution of MDTreeS/M protocol with an example. Assume agent 1, 2 and 3 are making decision over a set of three binary-valued variables V = A, B, C . The domains are D(A) = a, a¯ , D(B) = b, b¯ and D(C) = c, c¯ , respectively.{ For} the sake of simplicity, we assume{ } the agents{ } have linear {orders} over O, which are depicted in Figure 1. In essence, these linear orders extend the following CP-nets respectively: agent 1: c : a¯ a, c¯ : a a¯; b b¯; c c¯. agent 2: a a¯; b¯ b; b : c¯ c, b¯ : c c¯. agent 3: b : a¯ a, b¯ : a a¯; b b¯ ; a : c c¯, Collective decision-making in combinatorial domains 11

ф* Agent 1 Agent 2 Agent 3 ф ф ф ф I1 : 2 1 ф1 ф2 2 1 I2 : ф3 ф1 ф4 ф4 ф3 ф3 ф1 ф4 I3 : ф5 ф1 ф6 ф4 ф6 ф5 ф6 ф5 ф1 ф4 ф ф ф ф I4 : 6 ф1 ф4 ф5 5 1 4 ф ф ф ф I5 : ф6 ф7 ф4 ф8 ф8 ф7 ф5 5 7 8 4 I6 : ф6 ф4 ф8 ф7 ф5 ф5 ф7 ф ф4 ф ф ф ф ф ф I7 : ф6 ф9 ф10 4 ф8 10 ф9 5 5 9 10 ф8 ф4 ф ф ф ф ф ф ф I 8 : ф6 10 4 ф8 9 5 5 9 10 ф8 ф4 ф ф ф ф ф ф ф ф ф I 9 : 10 4 8 9 5 5 9 8 4 ф ф ф ф ф ф ф I10 : ф6 4 8 9 5 9 8 4

Fig. 3. Agents’ fringes a¯ : c¯ c. As all variables are binary, the MDTree is a binary tree. Figure 2 shows the formation of the MDTree . Each node Φ in associates with a proposal table, in which the second rowT of the table displaysT the proposals that the agents make on that node: the 1st, 2nd and 3rd column depicts the proposal that agent 1, agent 2 and agent 3 makes, respectively; and the proposal made by agent i in th i the j iteration is denoted by Pj. For explanation purposes, we also attach the best possible alternative (BPA) of each agent on each node in the first row of the table: the 1st, 2nd and 3rd column depicts the BPA of agent 1, agent 2 and agent 3, respectively. However, notice that the information including the proposal that an agent makes and the BPA of a node of an agent is his private information and invisible to other agents. The nodes in in red are the winning nodes in W. Figure 3 provides an illustration of theT ongoing changes occuring in each th agent’s fringe. The number Ij denotes the j iteration, and the bullet in different colors denotes whether that agent makes proposal in that iteration: “” denotes the agent proposes, and “” means the agent does not make any proposal in that iteration. Suppose that an order σ = A > B > C is chosen for the formation of a MDTree . Initially, a root node as well as two children nodes Φ , Φ with assignment a T 1 2 and a¯, respectively are created in . For agent 1, BPA1(Φ1) = abc, BPA1(Φ2) = abc¯ . Since abc¯ abc (see Figure 1(a)),T the order of nodes in agent 1’s fringe is Φ -Φ 1 2 1 (agent 1 prefers Φ2 to Φ1). Similarly, agent 2’s fringe is Φ1-Φ2, and agent 3’s fringe is Φ2-Φ1. st In 1 iteration, agent 1 and 3 propose on Φ2 (the first node in their fringes), agent 2 proposes on Φ1 in Step 1. Thus, Φ2 is marked as open as it received a majority of agents’ proposals. Φ2 is then expanded with the possible values b, b¯ of the second variable B. Two children nodes Φ3, Φ4 are created in , and{ } they are added into each agent’s fringe (see Figure 3). Notice that agent 2’sT fringe does not contain Φ2, although it has not made proposal on Φ2. This is because Φ2 has been expanded and is not a leaf node any more. Therefore, it is not feasible for agent 2 to make a proposal on. nd Similarly in 2 iteration, agent 1 and 3 propose on Φ3, agent 2 proposes on Φ4. Φ3 becomes open and two child nodes Φ5 and Φ6 are created. 12 Minyi Li, Quoc Bao Vo, Ryszard Kowalczyk

rd In 3 iteration, agent 1 proposes on Φ5, agent 2 and 3 proposes on Φ6. Φ6 becomes a winning node and is added into W. Since it is the only member in W, the node chosen for convergence is Φ∗ = Φ6. Notice that agent 1 will remove Φ6 from his fringe, as it is not a feasible node any more. th Assg In 4 iteration, both agent 2 and 3 do not propose, as Φ∗ is preferred to the BPA of the most preferred node in their fringe: Assg 2 BPA2(Φ5) and Φ∗ Assg 3 BPA3(Φ5). Agent 1 proposes on Φ1 because BPA1(Φ1) 1 Assg . Φ1 is Φ∗ Φ∗ then marked as open and being expanded in Step 2. Two child nodes Φ7 and Φ8 are created in . th T In 5 iteration, agent 1 proposes on node Φ7, agent 2 proposes on Φ8, and agent 3 still does not make any proposal. In 6th iteration, agent 1 and agent 3 do not make proposals, agent 2 proposes on Φ7. Φ7 is become open and is expanded. Two child nodes Φ9 and Φ10 are created in . th T In 7 iteration, agent 1 proposes on Φ9, agent 2 proposes on Φ10, and agent 3 does not make any proposal. In 8th iteration, agent 2 and agent 3 do not make any proposal, agent 1 proposes on Φ10. As such, Φ10 become another winning node. Φ10 is added into W. Φ10 is chosen to be the node for convergence Φ∗ = Φ10 (since it is the Smith/Minimax solution in W = Φ ,Φ : Assg maj Assg ). { 6 10} Φ10 Φ6 In the 9th iteration, agent 1 and agent 2 do not make proposal, agent 3 proposes on Φ5. Φ5 becomes a winning node. Therefore, currently W = Φ6,Φ10,Φ5 . Assg Assg {Assg } In this example, Φ5, Φ6 and Φ10 form a circle: Φ6 maj Φ5 , Φ5 maj Assg Assg Assg Φ10 , and Φ10 maj Φ6 , and the maximum defeats of all of them are two. Consequently, they are all Smith/Minimax candidates in W. We randomly choose one of them for convergence, for instance, consider Φ∗ = Φ6. In the 10th iteration, all three agents stop making proposals: i 1,..., n , BPA Assg Assg ∀ ∈ { } Φ Li, i(Φ) Φ6 . The process ends and returns Φ6 = ab¯ c¯ as the ﬁ- ∀nal collective∈ decision. In this illustrative example, ab¯ c¯ is an actual Smith/Minimax candidate in the alternative space. Using MDTreeS/M protocol, the number of alternatives that each agent needs to consider (i.e., the number of leaf nodes in the MDTree) is 6. Moreover, the process only requires pairwise comparisons on Assg Assg Assg a set of three alternatives W = Φ5 , Φ6 , Φ10 , which is less than half of the alternative space. { }

3.4 Properties of MDTreeS/M protocol To prove the ﬁnal chosen decision using MDTreeS/M protocol is a member of the Smith set, we ﬁrst have the following proposition and corollary.

Proposition 1. For an individual agent i (i 1,..., n ), given a winning node Φ∗ BPA∈ { Assg} and a node Φ (not necessarily at depth m), if i(Φ) i Φ∗ (agent i does not prefer any BPA of Φ to Assg ), then: o Comp(Assg ), o i Assg . Φ∗ ∀ ∈ Φ Φ∗ Proof. For agent i, o Comp(AssgΦ), BPAi(Φ) i o. As individual preference is Assg∀ ∈ BPA Assg transitive, if o i Φ∗ , then we will have i(Φ) i o i Φ∗ , contradicting BPA Assg the fact that i(Φ) i Φ∗ . Collective decision-making in combinatorial domains 13

Corollary 1. For a majority of agents A∗ 1,..., n , given a winning node Φ∗ ⊆ { } and a node Φ (not necessarily at depth m), if i A∗, BPAi(Φ) i Assg , then: ∀ ∈ Φ∗ o Comp(Assg ), o maj Assg . ∀ ∈ Φ Φ∗ Assg Theorem 1. The ﬁnal collective decision Φ∗ chosen by MDTreeS/M protocol is a member of the Smith set.

Proof. When all the agents stop making proposals and stick with Φ∗, Φ < W, Φ does not receive a majority of agent’s proposals. Therefore, o ∀Comp(Φ), Assg Assg ∀ ∈ o maj Φ∗ . Let W denotes the set of alternatives speciﬁed by the nodes Assg Assg Assg in W. The ﬁnal outcome Φ∗ is a Smith member in W. Φ∗ is not Assg Assg majority dominated by any outcome in the set O W. Therefore, Φ∗ must also be a Smith member in the set Assg (O− Assg ) = O. W ∪ − W

4 Experiments

We now describe some results of experiments with respect to (i) the number of variables, the size of the alternative space O , and the number of agents; (ii) the average number of alternatives that each| | agent has considered during the decision-making process, denoted by Altcon; (iii) the average number of dominance queries (outcome comparisons) that each agent needs to answer, denoted by DQ; (iv) the average number of alternatives that requires direct pairwise comparisons over, i.e., the average size of W ( W ); and (v) the average running time. Regarding the running time, we compare| the| proposed MDTreeS/M protocol with an algorithm, called SMDesc. SMDesc algorithm first computes the Smith set from a set of descending orders induced from the agents’ preferences, and then chooses a Minimax candidate from the Smith set. In these experiments, we focus on binary variables. For the sake of simplicity, we consider the representation language ”SLO SCPnet” [5] to represent the agents’ preferences in collective decision-making, which induces a linear order over the alternative space. SLO SCPnet is a compact representation language that combines the nature of soft constraints and ceteris paribus structural preferences. Moreover, as outcome comparison in SLO SCPnet is easy, the experiment results reflect the reduction of the total number of dominance queries using MDTreeS/M protocol compared to using SMDesc algorithm. Obviously, when the dominance queries in the preference representation are more difficult (e.g., dominance queries in CP-nets), this saving will be even bigger, since longer time is needed for answering a single dominance query. In these experiments, the order σ used to generate the MDTree in MDTreeS/M protocol is randomly chosen, and there are no restrictions on the agents’ preference structures. We run 20000 rounds of experiment with 2-10 variables, 5 and 15 agents, and with both MDTreeS/M protocol and SMDesc algorithm. How- ever, when the number of variables is more than 10, we only run 100 rounds of experiments to compare the execution time of two methods. From Table 1, we can see that the average number of outcomes Altcon that each agent need to consider, the average number of dominance queries DQ, the 14 Minyi Li, Quoc Bao Vo, Ryszard Kowalczyk

Table 1. Using random variable order in MDTreeS/M

5 agent 15 agent Var. OS Time (sec.) Time (sec.) Altcon W DQ Altcon W DQ | | SMDesc MDTreeS/M | | SMDesc MDTreeS/M 3 8 5.5 2.3 5.4 0.007 0.006 6.75 3.3 8.0 0.03 0.02 6 64 24.8 9.9 128.2 0.17 0.073 44.92 16.9 379.4 0.41 0.37 9 512 106.6 51.5 2097.9 7.96 0.59 234.3 83.9 10173.3 12.9 4.18 12 4096 441.1 322.7 29939.0 1764.2 8.20 1142.4 399.4 240605. 2174.9 74.20 average size of W , and average running time of MDTreeS/M protocol are in- creasing as the number| | of variables increases. Nonetheless, making a collective decision using MDTreeS/M protocol is more efficient than using SMDesc algorithm. Compared to the large outcome space, by using the proposed protocol MDTreeS/M, the average number of alternatives Altcon that each agent needs to consider is significantly reduced in both 5 and 15 agent cases. When the number of variables is large (e.g., 12), on average, the average size of W is less than 330 in 5 agent cases, (resp. less than 400 in 15 agents cases), which| | is less than one tenth of the entire alternative space. Moreover, according to the experiment data, when the number of variables is 12, on average, the entire collective decision-making process ends within 9 seconds with 5 agents (resp. 75 seconds with 15 agents), while using SMDesc, this will take more than 1700 seconds (2000 seconds). In these experiments, for each number of agents and variables, in more than 99.8% cases the final decision chosen by MDTreeS/M protocol is the Smith/Minimax candidate. Moreover, even in the cases that MDTreeS/M protocol fails to choose a Smith/Minimax candidate, the maximum defeats of the alternative chosen by MDTreeS/M is sufficiently close to that of a Smith/Minimax candidate. For instance, in 33 cases out of 20000 runs with 15 agents and 9 variables, on average, the maximum defeats of the alternative chosen by MDTreeS/M is only 1 larger than that of the Smith/Minimax candidate.

5 Conclusion

In this paper, we proposed a very general distributed protocol for collective decision-making in combinatorial domains. The proposed protocol sat- isfied Smith criterion and the final outcome chosen is sufficiently close to the Smith/Minimax candidate. It works under incomplete information and significantly reduces the search space. An important issue for future research is manipulation: whether the protocol is strategy-proof, or an agent can get a better alternative by not using the BATCD strategy. Moreover, although the protocol can be applied to most preference representation languages, the running time of the protocol strongly depends on the running time of answering dominance queries and optimization queries. Thus, it is important to investigate the feasibility of the proposed Collective decision-making in combinatorial domains 15 protocol when applied to different representation languages with different computational properties on preference queries.

Acknowledgments.

This work is partially supported by the ARC Discovery Grant DP110103671.

References

1. Fahiem Bacchus and Adam Grove. Graphical models for preference and utility. In Proceedings of the Proceedings of the Eleventh Conference Annual Conference on Uncer- tainty in Artificial Intelligence (UAI-95), pages 3–10, San Francisco, CA, 1995. Morgan Kaufmann. 2. Craig Boutilier, Ronen I. Brafman, Carmel Domshlak, Holger H. Hoos, and David Poole. Cp-nets: a tool for representing and reasoning with conditional ceteris paribus preference statements. J. Artif. Int. Res., 21:135–191, February 2004. 3. Sylvain Bouveret, Ulle Endriss, and Jer´ omeˆ Lang. Conditional importance networks: A graphical language for representing ordinal, monotonic preferences over sets of goods. In IJCAI, pages 67–72, 2009. 4. Steven J. Brams, D. Marc Kilgour, and William S. Zwicker. The paradox of multiple elections. Social Choice and Welfare, 15:211–236, 1998. 5. Carmel Domshlak, Steven David Prestwich, Francesca Rossi, Kristen Brent Venable, and Toby Walsh. Hard and soft constraints for reasoning about qualitative conditional preferences. J. Heuristics, 12(4-5):263–285, 2006. 6. Judy Goldsmith, Jer´ omeˆ Lang, Miroslaw Truszczynski, and Nic Wilson. The computational complexity of dominance and consistency in CP-nets. J. Artif. Int. Res., 33(1):403–432, 2008. 7. Piero La Mura and Yoav Shoham. Expected utility networks. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, UAI’99, pages 366–373, San Francisco, CA, USA, 1999. Morgan Kaufmann Publishers Inc. 8.J er´ omeˆ Lang. From preference representation to combinatorial vote. In KR, pages 277–290, 2002. 9.J er´ omeˆ Lang. Logical preference representation and combinatorial vote. Annals of Mathematics and Artificial Intelligence, 42:37–71, September 2004. 10.J er´ omeˆ Lang and Lirong Xia. Sequential composition of voting rules in multi-issue domains. Mathematical Social Sciences, 57(3):304–324, May 2009. 11. Minyi Li, Quoc Bao Vo, and Ryszard Kowalczyk. Majority-rule-based preference aggregation on multi-attribute domains with CP-nets. In Proceedings of the 10th Inter- national Conference on Autonomous Agents and Multiagent Systems, volume 1, pages 659 – 666, Taipei, Taiwan, 2011. Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems. 12. Lirong Xia, Vincent Conitzer, and Jer´ omeˆ Lang. Voting on multiattribute domains with cyclic preferential dependencies. In AAAI’08: Proceedings of the 23rd national conference on Artificial intelligence, pages 202–207. AAAI Press, 2008. 13. Lirong Xia, Jer´ omeˆ Lang, and Mingsheng Ying. Strongly decomposable voting rules on multiattribute domains. In AAAI’07: Proceedings of the 22nd national conference on Artificial intelligence, pages 776–781. AAAI Press, 2007.