Trends in the Theory of Decision-Making in Large Systems

Home , Decision theory

This PDF is a selection from an out-of-print volume from the National Bureau of Economic Research

Volume Title: Annals of Economic and Social Measurement, Volume 1, number 4

Volume Author/Editor: NBER

Volume Publisher: NBER

Volume URL: http://www.nber.org/books/aesm72-4

Publication Date: October 1972

Chapter Title: Trends in the Theory of Decision-Making in Large Systems

Chapter Author: Pravin Varaiya

Chapter URL: http://www.nber.org/chapters/c9448

Chapter pages in book: (p. 493 - 500) A iinalsof!ionofltu anti Social Aleasure,,,e,,i,1/4, 1972

TRENDS IN THE THEORY OF DECISION-MAKING IN LARGE SYSTEMS'

IW PRAVIN VARAIYA

Thispapersun,nlari:ei the literature dealin' wit!, the niathe,natical aspect of decision-making by a set of agentseachof whom haspartialinformation and pariial co,,ttol. At! tlu' agents hate the same utility function.Theliterature is classified into 3 classes: !ean, 1/lear V. systemswhose prototypeis thecompetiiil'e econnohi. and syston,s arroned in a l,ierarc In. An ecaluationofthe literature is attenipri'd

1. INTROI)UCTION ANDSUMMARY This paper is a brief summary of the literature dealing with the theoreticaland mathematical aspects of decision-making in large systems. There is also an attempt to evaluate the work done so far, and to guess the potentialof different lines of investigation. We apologize in advance for any omissions, only some of which are forced by considerations of space. The generally agreed-upon framework for decision theory consists of the following: I. a set ofagents(decision-makers, controllers, actors), for each agent a pre/'rence ordering (utility function, performance criterion, pay-otT function, reward function). for each agent a set of permissible decisions (actions), each agent makes some observation of the environment (state-oFflie- world) in which the agent is operating, we call the mapping from thestate-of-the- world to the various observations theinformationstructure, finally each agent has a description or model of the system i.e.,the way in which the state-of-the-world and the combined actions of the agentaffect coisequences. Each agent's behavior is expressed by a decision rule which is aspecification of thu action to be taken when a certain observation is made. Weborrow from Marsciak and Radner [I] the term organi:ationalforin to denote aparticular choice of information structure and set of decision rules, one foreach agent. Most of the literature with which we are concerned assumes as exogenous tothe decision problem being studied the set of agents. their preferences, and thepermis- sible decisions. The focus of the study is the analysis anddesign of "good" organizational forms for one of two cases: many (2) agents with the samepreference ordering, many agents each with a differentpreference ordering. The situation envisioned in case B is usually called a game,and we do not deal with it further. There is a third possibility, a single agentwith more than one pieference ordering. This can arise in dealing with questions ofincentives, or with

Researchsupportedin part by the U.S. Naval Electronics Command. Contract N00039-71-C- 0255 and by a Fellowshiprrornthe John Simon Guggenheim Foundaiion 493 I

trying to find decisionrulesfora either. public agency. We donot dealwith this case The literature dealing withcase A can he classified into three teamtheory, categories: orgaflj,ationJ formswhose prototypeisthe system, and Competitive economic organizatjç,naj formsarranged in a hierarchy. This classificationis guided by the differences in areas ofapplication and in the mathematical techniquesused. Thus learn theory is the most abstract andformal treatment of our subject.The literature classified under class (b) deal withdecision problems whosestructures are specified in contains little abstract considerable detail. The lastclass theory and is mostlya collection of "case" studies.

2. IHEORYOF TEAMS Asatk learn2 may be formalizedas follows. Let S be the world. Let}' set ofstates-oftIie he the space ofobservations and generates agent i's observation, :S } the mapping which f=I.....N. Thus lion structure. For q= (,.....)is the informa- each I let D1 be theset of permissible decisions decision rule is so that the team's a function ((5k = An organizationa' where (5 D is is decisionrule. form is apair(ti, (5). subjective Let p(s), .seS, bethe team'scommon probability3distribution concerning occur From the model which state-of-theworldwill of the system andthe team's preference arrives in the usualway at a function u(s, d) ordering one =u(s, d1,.., d,il), such that the team's objective is o maximizethe expected utility

f(i, (5;p, u)= p(s)u(s, (5'(

Typically the entire state S is not relevanttoii. function z= Rather, there isa (many-to-one) C(s) such that i4s, d)= o(C(s),d) for a suitable (I) can herewritten as (2). function w, and then

(5 P,w) = p(s)w((s), ES ( ,(s))...... N('1(s))). = CO(Z,ö1(y1),(5\,(y))p(ç =z,1 =y1 = What sorts of questions can be raisedin this formal level, assuming,, P,w set-up? At the simplest fixed, we cansearch for a (5 which and then there is maximizes QØj,. P,a), nothing conceptuallydifferent which is a single agent. Letus call such a (5an not present in thecase of of a per.son-by_pgrso,j oplima!r:,le A moreinteresting satisfactory (.pbps) decision concept is that rule which isany rule (5* such that .ö...ô;P,) .ô, P,w) for all(51and all i. 2 The main reference is{lJ, which contains analmost exhaustive The common subjectiveprobability is crucial for the bibliography. lions, arising from ditfering theory. Thus expectations cannot be formulated many important con.sjdera. within this framework. 494 In words, is pbps if a unilateral decision change on the part of any agentwill not increase the team's expected utility. IfS is finite, then obviousconvexity and differ- entiability conditions will imply that every phps rule is optimal.Another result is that if p is a Gaussian distribution, if ,i is linear, and if u is a concavequadratic form, then there are linear pbps rules which arc optimal.4 But these results are of mild interest only. There seem to be threepotential useful directions. The first involves considering the information structurejas variable. Let V(: P, o4=Max iS; P. w) a be the value ofq (relative to (P. w)). One may then seek to maximizeV( ; P, w) over a suitable class, or better yet, try to maximize P, o) - where C(ii) is the "cost" of the information structurej. Butboth of these questions turn out to be technically not well-posed. Theappropriate questions are of the following kind.Given two information structures, q'find conditions such that V(j: P, o4 V(i1' : F, w) forallw. For the case of a single agent there is an extensiveliterature5 addressed to these sorts of issues since they arise in statistical decisiontheory, information theory, and economics. Extensions of some of these results to the caseof teams appear to be quite straightforward. However, the form of the set of all decision rulesis such that extensions of most of these results wifl be difficult. In many situations the agents of the team take theirdecisions sequentially rather than simultaneously. Further, the observationmade by an agent depends upon the actions taken by other agents. Such asituationcannotbe described as a static team. Consider the following simple formulation.Let S. Y, D1, u be as before. However q. is now of the form ai) (4) y=q1(s; where7i=(d1,. .. ,d1_1,di,...,d). It turns out that the class of organizational forms that can be placed in this form is much wider thanthe class arising in static teams. And there are surprises. Suppose s is aGaussian vector, the rare linear, and 14 is concave and quadratic. Then the optimal decisionrule neednotbe linear [7]. Since the appearance of this "perverse" resultconsiderable effort has been spent in discovering which information structuresimply linearity of the optimal rule. To state a recent result say thatjis a precedent of i,j > i, if either jd3 0 or there exists k such thatj> k and k > i. The information structure is said to be nestedif1contains as a component everyfor everyj precedent of 1. It has been proved [8] that for nested information structuresthere exist linear

This is surprising in that the statement is not in general truefor dynamic teams (see below). A very nice survey, together with new results, is found in [2]. [3] isstill a basic reference. Some new developments are reported in [4] I For amore general formulation sec [5]. Note that theformulation (3) may lead to an ambiguous situation since y1 may depend on d which may depend on d1 whichdepends on y,. For a discussion of this point see [61. 495 optimal decision rules, andthe proof consists in Showingthat the team problem is equivalent to anotherstatic team problem. The literature results of a general Contains almost no nature which are relevant to "truly"dynamic teams To motivate the thirdline of developmentwe hazard a gencraliation based upon the results in static and dynamicteams concerning the Gaussian, linear, quadratic case. It appears that the computationtl effortnecessary for calculating an optimal rule for a team (two or more agents) is considerablygreater than for centralized decision-makingthat is, when there is only that from consideratiojs one agent (see e.g 19]), so of design one woulduse a team decision framework only if decentralization of decision-makingwere enforced because of physicalor institutionalconstraints.7But implicit in the assertion that some of the literature in team theory is decentralized decision-making willresult in savings in information-processing and computation ii this assertion isto be treated seriously,we need to study decisionprocesses in teams where the messages to each other (and receive agents communicate some messages from the environment) whileat the same time the agents updatetheir decisions in the light received.8 of the new information We now mentiontwo examples [13. 14] where studied.9 such processesare The problem consideredin [13] is that of synchronizing (i.e., acheivinga common frequency among)a finite set of geographically separated oscillators. A centralized solution might heto choose one of the oscillators and transmit its frequency as a "master clock" to the rest. Each one ofthesewould lock onto (tune into) the master clock. Thedecentralized scheme that to each oscillator the was studied in [13] was to transmit frequencies of some of therest. Each oscillator locks avcrage' of the received frequencies. onto an It is shown thatsome simple feedback loops around each oscillatorcan achieve stability in the verge to a common frequency. sense that the oscillatorscon- Note that an additionalbenefit of the scheme is it is reliable sincea break in a transmission link that network synchronje will leave eachconnected sub- Reference [14] develops a decentralized scheme for findingthe maximum flow ofa singlecommodity through a node of the network, the capacitated network. Anagent is located at each agent observes the capacitiesand the Ilows in those branches of the networkwhich are incident to that node, and inpart controls the flow on these branches.Each agent continuously and based upon the sends messages to itsneighbors, received messages updatesits decision. The interesting about the schemeare that (l)the messages points are selected from a finitealphabet whose size depends onlyupon the maximum number of on the size of the network, and edges incident toa node and not (2) the messages neednot be synchronized i.e., order and the timeat which messages the are transmitted or receivedare unimportant We need manymore results of the kind modify the team-theory represented in [I 1-16] so thatwe can framework presented earlierin such away that these problems can he formulatedon an abstract level. This may be a reason for the paucity of applications Althoughthe subject was introduced 1955 [tO], one finds onlytwo "appljcatons" papers in consistently referenced it I,12] This is quite different tromthe usual tatônnernent literature, since theprocesses we are discussing are occurringprocesses one encounters in thecconornj to the tatônnemcnt processes where in rs'al time Howeverthese are similar "recontracting" takes place. 9Two examples fromeconomics can be found in [15, 16]. both compare alternative processes 496 3. ORGANIZATIONAL.FonisREMINISCENT 01 TIlE 0 )MPFTITIVE ECONOSIY Aside froni its social implications, the most celebrated propertyof the corn- petitive economy derives from the proposition thatit achieves an efilcient allocation among thousands of agents by transmitting toeach agent a minimal amount of information, namely, the prices of the various commodities in the economy.This has lead to a tremendous amount of research directed todevising algorithms for computing optimal decisions in large systems which stimulatethe competitive economy. With the growth of non-linearprogramming the search has broadened to include any algorithm which leads todecentralization in the sense that the algorithm decomposes into many "parallel" procedures eachdealing with a small number of variables the motivation being that such algorithmswill pernhit current computer facilities to solve problems of increasingsize. Since there is available an excellent survey of this entire area [171 we will not pursueit any further except for three remarks. First of all, most of these decentralizingschemes ("decomposition algorithms'' in nonlinear programming parlance) imposefairly stringent con- vexity conditions although promising recent developments areovercoming some of these ([18], [19]) at the cost of increasedinformation transmission. Our second point is to emphasize that almost all of theseschemes are suitable for static decision problems only. Finally, uncertainty has not been introduced in any interesting way.

4. ORGANIZATIONAL FORM ARRANGEI) IN A HIERARCHY We start by assuming that the system is to consist of anumber of interacting subsystems each one of which is under the control of one agent.Furthermore, the subsystems are arranged in a hierarchy of levels. Thesubsystem in each level communicates with several subsystems in the level under it andwith one sub- system in the level above. The stratificationinto levels is not arbitrary. It is implied that the agents at different levels perform diflerent"tasks" or "functions." The agents at the lowest level perform "routine"tasks while those above take "long- term" decisions. A significant portion of controlengineering designs have the structure of a hierarchical organizational form.However, there are no well- articulated design philosophies. There is no mathematicaltheory of such forms (although [21] is an attempt to formalize the notion ofahierarchy and also gives a useful classification of different designs), and thereis no serious attempt to explain why hierarchical forms are vorthwhile.'' Becauseof the lack of any substantive mathematical theory we merely give an example of asimple hierarchical form due to Minsky and Papert [23]. The perceptron is a pattern recognition devicearranged in a two-level hierarchy as follows. The pattern is displayed on aplane divided into a square grid. Each square is either black (I) or white (0), and a patterncorresponds to a particular

I [20] is one of the excepiions It is shown that the problemof the optimal control of a dynamic linear system has a structure which makes the Dantzig-Wolfedecomposition prtnctp!e applicable- For a thought-provoking statement explaining the presenceof hierarchical structures in the natural and man-made universe in terms of reliability, adaptabilityand reduction of computational requirements sec [22].

497 S

distributionof black andwhitesquares. Corresponding to every set of a squares there is alower-levelagentwhoobserves these asquares (i.e., observesabinary numbers) and computes a binary functionof thesei numbers and reports its value to the higher level.Thusif there areNlowerlevel agents, theycan be represented by N Boolean functions , .....çb where4:O,I} O, l. There isonlyone agent at the higher level. This agent receivesthe N binarynumbers ..... from the lower level, computes a linear function '= .+ and makes a binary decision using the sign of'. Thus thisagent's decision rule is given by [Yes ifX1çb>O Decision is lNo Every organizational form ,..., ,i/i)divides the class S of all patterns into two disjoint jets S1. S by scS1 if1(s).....(s)) > 0 SE Sif IIifr/1(s).....4(s)) 0. We say that Sr is the set ofpatterns recognized by the organizational form. Within this framework it is shown in [23]that (here is no organizational form ..... ,s1i} which recognizes exactly theset of all connected patterns. However, there is an organization form of order 3(i.e. a = 3) which recognizes theset of all convex patterns. Similar questions and other classes oforganizational forms are also considered. This study [23] suggests that we may make interesting discoveries pickingan appropriate class of organizational formsand ask which set of taskscan be accomplished within the chosen class. Weend with illustrating this idea withone more example. Consider a linear system

t(t) Ax(t) + Bd(:) where (1(1) is the decision vector of dimension N.The ith component ofd(t), d1(t), is the under control of agent 1. Suppose that thestate vector x(t) is decomposed into N subvectors x1(t),...,x,(t), andsuppose that the ith subvector x1(t) is observedby the ith agent who is also in charge of"controlling" it.' 2 Now consider the following class of organizational forms. Suppose all the agents report their observationsto a "managing" agent who chooses a linear feedback control scheme representedby a matrix C so that the other agents now face the task of controlling thesystem = Ax(t) + BCx(t) + Bd(t). Does there exist a matrix C which permits the task to be accomplished?We can enlarge the class of organizational forms even further to allowsystems of the form (t) = A.x(t) + BCx(t) + BKd(t) which means that we are allowing the lower level agents to affectevery input. A fairly complete theory for these classes of systems isnow available ([25], [26]). 1For example the ith agent's task may be to stabilize .xat a prespecified larget x7. For application to a problem in economicssee [24]. an

498 5. A CON'I.UDING RIl,tRK Our objective has been to show that researchersin (Ilverse liekis are facing the prohkms of dcsigmng organizational formsfor decision-making in large svstenis. The probleim; that they face are varied and theyhave devised niatiy ingenious I schciiic for tackling them. Perhaps the most acuteneed is to develop an abstract framewot k within which many of these schemes appear as"special cases.'' At the same time the frameworkshould besufficientlyoperationally usefulSOthat some detailed questions of comparisons oforganizational forms can he formulated and answered. Dc'latrt?iu'tlt0/E!c'cti'h'a! Eogvteet'ing and ('mnputer Science's Linit'e'rsitv of Catitornia, Berkele'v

R1FIRINCUS

19721, New flaxen Cowles 1-otinda- I]J. Marschak and R. Radner, i/u' I'COPnIOhiC tlit'(,r' of (conic, ( tion Monograph 22, Yale University Press. 12] J Marschak and K. Miyasawa. "Economic comparabilityof information systems. meL &un. Rer. 9 (1968), pp. 137--l74. D llackwell and sl. A. Girsehick. T/u'crv of ganiecanti .suJtLo,dal t/t'cisiu.t. (1954) New York. Wiley. (eds.) C. B. McGuire. Comparison of information structures.in C. [3. McGuire and R. Radner Decision and organi:aliun (Iorthconhiflg). H. S. Witsenhausen. "Separation of estimationand control for discrete time systems." Proc. IEEE59(1971)pp. 1557-I56f'. -----, 'On information structures.feedback and causality," S/AM J. Control 9 (1971) pp. 149 160. 147. --.--- -,"A counterexainpleinstochasticoptimum control." S/A Il J. ('onirolO ( 1968)pp. 131 Y -C Flu and K. C. Chu. "Team decision theory andinformation structures in optmal control problems Parts I," IEEE Trans. on Auloma ticControl A C'- 17 (1972) pp. 15-21. C.-Y. Chong and M. Athans. "On the stochastic controlot' linear systems with different informa- tions Sets." IEEE Tra,is. on ,4iaonwiic Control AC-17 (1972) to appear. J Marschak. "Elements for a theory of teams," MwtagenientScience 1(955) pp. 127 137. [Ii] M. Becknsann. "Decision and team problems in airlineresersations." Econome!rito 26 (1958) pp. 134-145. C. B. McGuire. "Some team models of a salesorganization." Managenu"iI Science 7 (1961) pp. 101-130. 1. W. Sandberg. "On conditions under which it ispossible to synchronize digital transmission systems." Bell St'sr. Tee/i. i 48(1969) pp. 1999-2020. R. Lau. R. M. Pcrsiano. and P. Varaiya. "Decentralizedinformation and control: a network flow example," iEEE Tra,is, on Anwnial it' ControlAC-17 (1972) to appear. R. Radner. "Allocation of a scarce resource under uncertainty: artexample of a team" in C. B. McGuire and R. Radner op. (-it. T A Marschak. "Computation in organizations'. the comparisonof price mechanisms and other adjustment processes" in C. B. MeGuire and R. Radner op. cit. A Geoffrion, "Elements of large-scale mathematicalprogramming PartI Concepts, Part II Synthesis ol' algorithms and bibliography." ManagementScience 16 (1970) pp. 652-691. A. Camacho. "Externalities, optimality, andinformationally decentralized resource allocation processes." Int'l. Ewn. Rec. //(1970) pp. 318-327. M. Aoki. "Two planning processes for an economywith production externalities." DiscussiOn paper 157. Harvard Inst. for Econ. Res.(1970). G. B. Danizig and R. M. Van Slyke,"Generalized linear programming." in D. A. Wismer (ed.) Opiimi:a;ion methods for large sisu'ms (1971) New York:McGraw-Hill pp. 75 120. M. D. Mesarovic. 1). Macko, and V. Takahara. Thecrr of'!iicrarclihal. ,uultilei'eI st'.steflLs (1970). New York: Academic Press. (Sec also the review of thisbook in IEEE Trwps. apt ,4u(oimitic Control AC-17(1972) pp. 280-281.) 499 H. A. Simon, "The architecture of complexity," Proc. Am. Phil. Soc. /06 (1962)pp. 467-482. reprinted in J. A. Litterer (ed.)Organi:anons: systems, control, and adaptation Vol II (1969) New York: Wileypp. 98-114 NI. Minsky and S. Papert, Perceptions,(1969) Cambridge, Mass.: MiT. Press. D. McFadden, "On thecontrollability of decentralized macroeconomic syStems: the assignment problem" in H. W. Kuhn and G. P.Szego feds.) MatJ,en,auea/ srvu'mc ,henrt' and ecfn'mivc / 1969), Lecture notes in OperationsResearch and Mathematical Economics, Number II, Berlin: Springer-Verlag, pp. 221-240. W. M. Wonham and A. S. Morse."Decoupling and pole assignment in linear rnultivariablc systems: a geometric approach," S/A Mi.Control 8 (1970) pp. 1-18. M. Haymanu, "On the input andoutput reducibility of multivariable linear s sterns," IEEE Trans on .4ufmna,jc ('ont:U(, .4C-/5 (1970)pp. 563-569.

500