AGENT-BASED COMPOSITION OF BEHAVIOURAL MODELS

Principal Investigator: Katia Sycara

Project Duration: 1st March 2002 – 30th June 2002 Project Cost: Agent-based Composition of Behavioural Models

1. Background and Motivation

Cognitive models are models of human performance that represent human knowledge, and internal information management processes. It provides an integrated representation of the knowledge, procedures, strategies, and problem-solving skills used by a human in a domain or task situation. The execution of the model takes into consideration the cognitive capabilities and limitations of humans. The cognitive modelling community has developed several frameworks and techniques for the modelling of human behaviour as subjects engage in problem-solving or decision-making behaviour. Such frameworks facilitate the construction of human cognitive performance models, which have been used in a variety of ways: (a) as embodiments of theories of cognition, whose performance matches and predicts human behaviour [10, 13, 14], (b) as models of expert human performance, (e.g. COGNET) to be used, for example in tutoring systems [34], (c) as models of students, (d) as models of “autonomous manned entities” (e.g. helicopters, airplanes) in SAFOR research. In general, these models have been the basis for design and evaluation of computer systems and the human-computer interface, and as training tools [10]. Most cognitive architectures break down human information processing into three parallel gross activities: perception, cognition, and motor activity. The perceptual process receives information from the outside environment, internalises it into symbolic information, and stores it into a local information store that can be accessed by both the perceptual and cognitive processes. This internal symbolic representation of the outside world is manipulated by the cognitive process using existing procedural knowledge, potentially resulting in modifications to the human’s mental model (as part of its reasoning processes), and also the invocation of actions through commands or instructions to the motor system (arms, hands, speech, etc.), allowing the human to manipulate the environment. There are three primary approaches to modelling the cognitive process: Soar, ACT-R and COGNET. The Soar architecture [10] uses production rules as the basic unit of long-term knowledge. These rules are used to select and apply ‘operators’, which correspond to the actions and goals of a human on a mission. Abstract operators, such as ‘clear hallway’, act as goals, which are dynamically decomposed by rules that use more primitive operators to achieve the abstract actions. These rules assess the current situation, including the available sensors and mission parameters, and propose appropriate actions. Thus, Soar supports dynamic hierarchical decomposition and execution of complex operators. This organisation appears to correspond to the way humans organise their knowledge and provides reasonable traces of behaviour (in terms of operator selections and decompositions.) Soar also supports operators that are not part of the explicit goal hierarchy; but are opportunistic operators, corresponding to implicit goals such as survival, which can be selected independently of the currently active, explicit goals. The ACT-R architecture is a goal-directed production system theory that models cognition as a sequence of production rules that fire to coordinate retrieval of information from the environment and from memory. It has been used to model tasks as diverse as memory retrieval and visual search to learning physics and designing psychology experiments. ACT-R is situated at a level of aggregation above basic brain processes but considerably below significant tasks such as air traffic control. An ACT-R system essentially has three kinds of memories: a goal stack, that encodes the hierarchy of intentions guiding behaviour, a procedural memory containing production rules and a declarative memory containing chunks. Access to these memories is coordinated around the current goal that represents the focus of attention. Productions are selected to fire through a conflict resolution process that chooses one production from among the productions that match the current goal. COGNET (COGnition as a NEtwork of Tasks) was developed by Zachary, Ryder and colleagues at CHI systems et al. [34]. A COGNET system consists of a blackboard for problem representation, containing declarative knowledge, a number of tasks representing procedural knowledge about the situation and mechanisms for sensing the external environment, called perceptual demons, and for acting on the environment (actions). COGNET has been used in the modelling of human operators in certain real-time multi-tasking domains, such as air traffic control and telephone operations. An alternative framework that has also been used for simulations is MicroSaint, which uses primarily mathematical concepts, such as 2 Agent-based Composition of Behavioural Models queues and stochastic models, as units of models. However, since it is not based on a cognitive architecture, there is no obvious conceptual mapping onto human cognitive processes. Several major aspects of tactical decision-making have been identified by Zachary et al. [34] as being relevant to their cognitive analysis. These determine a number of requirements for cognitive modelling that have much in common with software development. Cognitive modelling must be

 real-time: In simulations, environmental data arrives in real-time and must be processed in a timely manner. Decisions must therefore be made with temporal constraints.

 opportunistic and uncertain: The external events faced are typically unpredictable and uncertain. Furthermore, the results of one’s own actions may be uncertain. Thus, decision-making must take into account both the unfolding situation and the results of the actions taken.

 multi-tasking: The pace and uncertain nature of events requires the decision maker to be prepared to interrupt the cognitive processing or sub-behaviour underway at any time.

 situated in simulation environments and interact through verbal/perceptual interactions: Decisions are implemented either through direct actions on the underlying simulation environment or through interactions with other agents. The cognitive analysis must be able to capture decision processes that are based on these types of input/output stimuli. Software systems have addressed similar requirements in very different application areas ranging from software for embedded systems, to assembly line control and coordination, where multiple entities have to be coordinated with strict constraints on the use of resources, such as time, processing power and other shared resources. The key problem in developing models of human behaviour is that [34] “there is no clear 'standard' method that is appropriate for all situations and domains. Rather, the method used must be selected (or developed) to meet the specific needs of the analysis”. In other words, the development of a model for every new task and every new domain has to be begun from scratch. Usually, labour-intensive task analysis is undertaken to make a detailed mapping of what humans are doing, or should be doing, in order to complete a set of tasks. Every model is coded as a new set of production rules, set of executable procedures, e.g. operators and methods. Not only is this time-consuming and effort-intensive, it does not take advantage of the possibilities for model reuse and model combination and does not scale well for the development of large models. Decomposition would allow models that are common across applications to be reused and composed to solve the requirements of new applications. Current modelling frameworks (e.g. SOAR) do decompose the task in hierarchical fashion of goals and subgoals, but the set of rules that implement this task decomposition is represented in a single data store in a single system. For example, in [10], for a squadron of airplanes, each rule set is copied into each entity, although not all rules are needed or used. In that sense, by the fact of copying the model into another entity in the same domain (e.g. another airplane), the model is reused, but not across different applications. As computer simulation environments become more complex and as these environments are increasingly used for training, the need for increased automation in the construction of these models is becoming painfully apparent. There are several benefits to being able to treat portions of models as software components. As in the software engineering world, modular decomposition of a model minimizes adverse interactions between the subcomponents. Each portion of the model can be tested independently, thus contributing to increased overall reliability of the system. Construction of a new model requires less time and effort. Therefore increased automation in model decomposition, reuse and composition of component models across new applications will allow cognitive models to scale to complex tasks and simulation environments.

3 Agent-based Composition of Behavioural Models 2. Research Objectives

The objective of our research is to provide (a) a methodology, grounded in computational theory and meeting cognitive requirements of human performance to decompose behaviour functionality and map it to a set of software agents, and (b) robust computational methods and software tools that enable increased automation in reuse and composition of the agents, so that the composite system reflects the observable behaviour of the modelled humans. A key feature of agent-based development is the encapsulation of knowledge and processing in autonomous units of behaviour/processing. This distinguishes agents from traditional software components that are typically passive pieces of code. The current cognitive modelling frameworks do separate knowledge, as stored in short-term memory, from processing, which is contained procedurally in the production rules. However, current cognitive models are “monolithic” rather than “compositional”. This research objective is extremely challenging for multiple reasons. First, automated software reuse and composition is an open research area in Software Engineering. Second, human behaviour in complex domains often includes multi-tasking for which, as Pew and Mavor have remarked [19], “relevant theories and models are not well developed or validated and the computational models are somewhat ad hoc”. Third, different models of cognition incorporate assumptions about the decomposition of tasks into smaller subtasks and units done to the elementary keystroke level, but there has not been a theory or practice of how these subtasks/behaviour fragments can be reused across different applications. 1 Therefore, the proposed research has to go beyond notions of composability of components in Software Engineering, i.e. functional composition, to notions of “cognitive composition” that include the software framework (computational model infrastructure) the requisite knowledge and the human processing characteristics. A key point here is that we are interested in generating human-like observable behaviour, rather than developing models that explain human thought and skill acquisition at a detailed level, e.g. ACT-R [13, 14]. Earlier experiences in modelling entities to be used in large simulation environments, e.g. [10], has shown that attempting to model human behaviour to the greatest possible detail would greatly slow development and increase the required computational resources of the system and may not guarantee increased accuracy. Thus, in our research, the computational model architecture will correspond to human cognitive architectures, but abstract away from modelling low level eye and finger movements retaining the basic decision processes and mimicking human observable behaviour. Our overall research hypothesis is that cognitive and behavioural functionalities can be decomposed in appropriate ways and that these fragments of behaviours can be composed in a semi-automated fashion in similar ways that software components can be composed together. Instead of writing production rules from scratch and having to consider primitive actions, certain behaviours can be treated as modules and composed to simulate behaviour that is more complex. In particular, our hypothesis is that such reuse and composition will be facilitated by agent-based software development methods. In the short term, our experience in this area predicts that the composition of behavioural components will involve some degree of manual adaptation by software programmers to repair possible software interface mismatches between agents. However, we would like to extend the proposed short-term research to enable cognitive modules to be automatically invoked and composed in accordance with the task and situation at hand. We have taken some important steps in this direction with the development of the RETSINA multiagent infrastructure (see Section 4.1).

In particular, we will investigate the following research hypotheses:

1 Many skill acquisition theories make the implicit or explicit hypothesis that complex tasks consist of many small parts each of which is learned according to basic learning principles. However, as remarked in [14], there has been rather sparse testing of this hypothesis. In the proposed research, however, we will not deal with learning per se as we are dealing with expert task performance.

4 Agent-based Composition of Behavioural Models

1. Behavioural decomposition: That it is possible to decompose behavioural functionality in meaningful ways and map it onto agents that can be reused across applications and can be composed to support the overall task.

2. Semantics-based interoperation: That semantically rich agent interfaces can be constructed to support flexible and meaningful composition. It is clear that semantic agent interoperation is a prerequisite for meaningful agent composition: unless the agents can make their behaviour explicit so that it can be “understood” by another agent (or an agent developer), agent composition would be “semantically incoherent”. This hypothesis embraces heterogeneity, in that it assumes that agents would be developed by different developers and per force would be heterogeneous.

3. Automated agent composition: That it is possible to “discover”, select and compose application- suitable agent components, given some appropriate description of the cognitive task and its requirements. To test the above research hypotheses, we must address a number of challenging questions. What is an appropriate agent processing architecture? What kind of behaviour decomposition method is most appropriate for a plausible mapping of fragments of cognitive behaviour to agents? Should the decomposition be based on tasks? Should the decomposition be based on psychologically motivated structural components of cognition, such as short term memory, or instead on some sort of hybrid containing task fragments integrated with appropriate cognitive processing constraints? Who performs the decomposition? A human designer/programmer? Another agent? How can these decomposition schemes be compared and evaluated? What would be appropriate representations for agent software interfaces that express the agent’s functional behaviour? How would these interfaces be utilized in agent composition? How can agents discover others with requisite functionality to form a “composite”? Can agents be stored, indexed and meaningfully selected automatically based on specified requirements? Who generates the requirement specification and selection criteria for the discovery of a desirable component agent? A human programmer? Another software agent? If different development groups develop agents, how can ontological mismatches in agent behaviour descriptions be mitigated? What are the const/benefit tradeoffs of semantic interoperability in terms of computational performance, software design time, system reliability and robustness? In order to fulfil our research goal and test our research hypothesis, we propose to leverage our prior work in the RETSINA agent-based infrastructure [27] and develop agent-based methodology for modularised construction and composition of behavioural models of human performance.

5 Agent-based Composition of Behavioural Models

3. Research Backdrop: Brief Overview of RETSINA

RETSINA stands for Reusable Environment for Task Structured Intelligent Networked Agents [23, 28]. It provides a multiagent infrastructure that has been developed under the following assumptions: (a) the agent environment is open and unpredictable, i.e. agents may appear and disappear dynamically; (b) agents are developed by for a variety of tasks by different developers that do not collaborate with one another, (c) gents are heterogeneous and could reside in different machines distributed across networks, and (d) agents can have partially replicated functionality and can incorporate models of tasks at different level of decomposition and abstraction. For example, there can be a single agent that provides all kinds of weather information (including barometric pressure, wind direction etc) and for all cities in the world. On the other hand, there could also be weather agents that provide only temperature. Alternatively, there can be an agent that provides radar operator functionality, and agents that provide only target tracking functionality for a particular environment. These agents could vary in fidelity to the task constraints (e.g. the target tracking agent could operate at a more refined resolution level for tracking). The goal of RETSINA is to provide multiagent infrastructure so that agents can dynamically find application-appropriate agents and automatically at run-time compose a system configuration to accomplish user-defined tasks. As shown in Figure 1, each agent has a communications module [21], which is

Figure 1: Architecture of a RETSINA agent responsible for interactions and the exchange of messages with other agents. These messages could contain new objective from other agents or from the environment. The communicator uses the input/output message queue to modify the agent’s set of high-level objectives in its knowledge store. The planner module [17] uses the objectives and a plan library of pre-specified plan fragments. The planner composes these plan fragments to construct alternative possible plans for the agent, stored as task structures. The scheduler module uses the task structures determined by the planner module to create a schedule of

6 Agent-based Composition of Behavioural Models primitive actions for execution, that the agent can then execute. The execution monitor module monitors action execution in the operating environment and suggests repairs, if actions in the schedule fail. The four modules operate in parallel, as multi-threaded code. Thus, the agent can receive messages from other agents through the communicator module while the planning module simultaneously constructs plans. In this way, an agent can interleave deliberative planning with information gathering and execution monitoring, an important capability in dynamically changing environments. Most current cognitive approaches model the existence of multiple goals held on a stack, which can influence and change the current set of production rules to be executed. However, true multi-tasking in the form of parallel planning, execution and communication cannot be represented. In contrast, a RETSINA agent can model potentially concurrent behaviours, such as the simultaneous listening to an audio signal and reading a text. Behaviours that cannot take place at the same time, but must be serialised, such as reading a text and viewing a video signal, can also supported by the RETSINA architecture. The green box in Figure 1 represents the knowledge store of the agent, which consists of a goal stack, where incoming or internally generated goals are stored and a task database, where task fragments (could also be viewed as previous abstracted cases) relevant to the agent’s functionality are stored and reused to construct plans. It contains an additional belief database that stores the current beliefs of the agent, as information from the environment or through internal processing changes. Although the RETSINA infrastructure has been used to construct multiple agents for applications ranging from financial portfolio management [24], to non-combatant evacuation operation [7], to agents interacting with ModSAF simulations for joint mission scenarios, we have not explicitly tried to make RETSINA agents model cognitive human performance. However, the RETSINA agent architecture possesses many similarities with the cognitive architectures reviewed in previous sections. A goal stack, for example, is common to most cognitive architectures. The task database corresponds to a procedural knowledge store. The belief database can be considered as a long-term memory store. Our long-term research agenda is to extend the representation schemes and potentially the agent infrastructure as needed to conform to cognitive requirements and evaluate the utility of RETSINA-like agents as computational models of human performance. For example, simulating latencies in human processing might require the introduction of artificial delays in the execution and communication of agents.

User 1 User 2 User u Goal and Task Specifications Results

InterfaceInterface Agent Agent 1 1 InterfaceInterface Agent Agent 2 2 InterfaceInterface Agent Agent i i Tasks Solutions

TaskTask Agent Agent 1 1 TaskTask Agent Agent 2 2 TaskTask Agent Agent t t

Info & Service Information Integration Requests Conflict Resolution Replies

MiddleAgentMiddleAgent 2 2 InfoInfo Agent Agent n n Advertisements Answers InfoInfo Agent Agent 1 1 Info Info Info Queries Info Info Info Source m Source 1 Source m 7 Source 1 SourceSource 2 2 Agent-based Composition of Behavioural Models

Each RETSINA agent provides a set of services that are defined by the agent’s capability/functionality. The figure above shows a typical multi-agent interaction, where a number of users will specify goals and tasks to be accomplished. These specifications are received by interface agents, that decompose the task and hand off portions of the task to task agents. The task agents could accomplish the tasks themselves, with the help of information agents [3]. If a task agent is unable to perform a task (due, for example to overloading), or the task is too complex, it may further decompose the task and delegate subtasks to other task agents. For example, an agent that plans possible routes for tanks, so they arrive at a rendezvous point at some particular time, may request the services of a weather agent for weather predictions in the area, since tanks can move at different speeds in wet soil than in dry soil. On the other hand, the route planner is also a service provider of route planning services that can be requested by a mission-planning agent. Since every agent can both plan and execute action sequences, the above architecture enables deliberation and reaction to the environment to be performed as needed at every stage of the task decomposition. In addition, the system does not impose on agents a particular granularity of task decomposition. Since RETSINA agents are meant to compose with agents developed independently by other groups, imposing commitments to a “uniform” task decomposition granularity would be unrealistic. Based on the environment and on constraints passed down to from other agents, a (task) agent can plan and choose the best course of action. This enables it to flexibly and dynamically adapt to changes in the environment. Furthermore, this promotes the modular composition of agents. Thus, an agent involved in the earlier stages of a task decomposition need only be concerned about high-level actions, whereas an agent in the ‘final’ stages will only consider low-level primitive actions. Such modularisation is difficult, if not impossible, in the current cognitive modelling frameworks. The single set of production rules needs to be able to handle everything from abstract actions to low-level primitive actions. In addition to agents that request and provide services, RETSINA includes a category of agents called middle agents [2, 32]. We have identified different types of middle agents, such as matchmakers, brokers, facilitators, recommenders, and have evaluated their performance tradeoffs experimentally [2]. Middle agents are registries of capabilities of service providers. They also include matching algorithms to automatically match dynamically arriving agent requests for a particular service against stored descriptions of service provider capabilities, called advertisements. When an agent is instantiated, it automatically advertises its capabilities with one (or more) middle agents. In this way, an agent becomes known to the agent society and enables a middle agent to subsequently match its capability advertisements against incoming requests. We have developed matching algorithms that combine information retrieval techniques with logic-based formalisms for agent capability matching [LARKS??].

4. Technical Approach

Our approach is based on the following key ideas: First, that cognitive models can be decomposed into smaller units and mapped onto agents that are active entities that include process descriptions and also mechanisms for instantiating themselves. Second, that these agents can be reused and composed in meaningful ways for new tasks so that the resulting composite system is a plausible cognitive model for the new task. Third, a crucial requirement for composability is meaningful interoperability of the composed entities. Meaningful composition of components must go beyond current Software Engineering notions of components having compatible APIs (e.g. for a serial composition of components A and B, the output variable type of A must be same/compatible with input of B), to notions of semantic interoperability. Our approach to semantic interoperability will be to develop rich descriptions of component interfaces to support more flexible forms of dynamic composition based on partial semantic matches that allow components to exhibit the desired behaviour even in the face of mismatches. Fourth, that automated selection of agents from agent registries is possible, given task requirements. These agents encapsulate pieces of needed behaviour and they can be composed to create a cognitive model of the overall new task. The impact of such semantic interoperability and composability of cognitive models will be significant. It will change the way of thinking about and building models: (a) it will enable instantiation and seamless reusability of models across a variety of heterogeneous applications and possibly execution environments, 8 Agent-based Composition of Behavioural Models

(b) it will free the cognitive engineer from dealing repeatedly with many low-level details, (c) it will foster a paradigm of computational cognitive model building, where software agents will be built with the explicit expectation that they are going to be used with other agents to achieve high-level goals, and (d) it will enable adaptive management of service and resource conditions to achieve a variety of human and system performance objectives. Because of the high levels of training and experience typically found in military personnel we assume that models we are composing are restricted to skilled performance. As skilled performance, the behaviour will tend to be highly stereotyped (deterministic) and largely determined by characteristics of the task rather than those of the performer. Our models will incorporate perceptual and motor latencies and movement times as principal determinants of task times. An attentional filter that discards tasks and events exceeding a numerical threshold will be used to simulate the effects of working memory limitations. Tasks will be indexed by type of resource demand [30] to accommodate compositions with multi-tasking. For example, we would allow visual search and a communications task to co-occur while procedure reading and communications would be scheduled sequentially. Our initial investigation will involve investigation of decomposability of cognitive models, along which dimensions this decomposition will be done, and the mapping of models to agents. In addition, we will examine agent interoperability at “design time”. Adaptation of agents to different mismatches will probably done semi-manually, by a human programmer/cognitive engineer. Our longer-term research effort, however, is aimed at enabling totally automated flexible selection, interoperation and adaptive composition of agents at run time and in open and changing environments. To flexibly interoperate, agents must: discover providers with appropriate functionality/capabilities, select the set of providers to work with, possibly adapt interfaces and inputs/outputs to bridge mismatches, transact/interact with the selected providers in an efficient manner, monitor execution, compensate/repair/recover from faults, and compose services to create more complex ones. The challenges that underlie these investigations are many, including: a suitable decomposition scheme for cognitive models; appropriate mapping mechanisms to map model components to agents; suitable representation of component models; structure of the agents to include processing mechanisms, agent knowledge and human performance constraints; representation scheme for representing agent capabilities; interface negotiation and adaptation strategies; protocols and mechanisms for agent interaction; schemes for detecting and handling mismatches. In the next few subsections, we describe in some detail the technical approach and research investigations we will undertake to address these challenges.

4.1. Behaviour Composition

Cognitive models are generally comprised of two parts: architecture and knowledge. The architecture of a system presents the system’s overall structure. At its core, a software architecture is a graph where nodes represent the different computational processing or data storage elements. Arcs represent pathways of interaction between computational elements. At a basic level, an architecture must provide basic input and output functionality and the ability to process knowledge. In cognitive models, the architecture is a domain- independent set of theoretical commitments as to cognitive process elements and their interactions. Knowledge represents the set of symbols on which the processing components act. In cognitive models, the knowledge is represented in terms of programmer-supplied rules, frames or procedures. The knowledge elements represent the overall tasks that must be accomplished by the human and is placed in the appropriate data stores indicated by the architecture. The architecture uses a set of mechanisms that control processing of the various parts of the architecture on the knowledge elements and determine what course of action the system should pursue based on its current knowledge. Changing different parts of the model, be it parts of the architecture or the knowledge, results in different system performance and different predictions. The design of model operation is guided by cognitive task analysis, which is the cornerstone of cognitive modelling. Cognitive task analyses yield psychologically plausible descriptions of peoples’ task performance. They are guided by formal descriptions of the task (assumptions and goals), constraints on task performance and 9 Agent-based Composition of Behavioural Models most importantly “think-aloud” protocol studies of task performance in which people continuously report what they are thinking about throughout task completion. These analyses can yield both high-level descriptions and fine-grained step-by-step descriptions of the task features the participant is processing, the goals that the participant sets, and the intermediate states that the participant achieves. By studying multiple participants at different levels of expertise and at different levels of resource availability, we can develop extensive descriptions of successful (and unsuccessful) task completion strategies, as well as capture heuristics as to how users solve or adapt to different problems. We propose to investigate whether and how cognitive behaviour can be decomposed in meaningful ways so the sub-components, implemented as agents, can be reused across applications and can be plausibly composed to accomplish new cognitive tasks. This decomposition could possibly be done in different ways. First, the decomposition could be done in terms of components of a cognitive architecture, e.g. short term memory, long term memory, goal stack etc. being represented by an agent. The functionality of each of these components would be incorporated in an agent. It is rather difficult to see up front how such a decomposition can be useful in alleviating the burden of developing behaviour models from scratch, since the labour intensive part is involved in the cognitive task analysis and task representation. Second, the decomposition could be done based on tasks, e.g. “radar operation” can be decomposed into “screen monitoring”, “target identification” and “target tracking” behaviours. Agents encapsulate particular behaviours that could co-occur at many different operator positions. In the above example of “radar operation”, this behaviour occurs in radar operators in CIC, radar operator on plane, radar operators at SAM and artillery sites, sonar operator on submarines, etc. Each of these positions also incorporates other different behaviours, which could be accounted for by other agents. Decomposition along task lines would seem at first glance to provide leverage since, once a task such as “radar operator” has been modelled for one situation and mapped onto an agent, the hope is that this task would not have to be modelled again and that the agent can be reused as a component of a new overall task. However, inclusion of the already modelled task-based agent in a cognitive model for a new application may give rise to additional requirements and constraints, which will have to be incorporated. Incorporating application-dependent constraints into the agent is an issue we will investigate. More importantly, task execution in a new context may give rise to different cognitive performance requirements. For example, imagine that the radar operator of a CIC must engage in verbal communication of his observations to his superior while watching the radar. These two tasks can be executed in parallel (multitasked). Now, assume that a sonar operator on a submarine is not allowed to engage in verbal communication of his observations, but must type them out on a adjacent screen. These two tasks cannot be so easily shared. Therefore, if the radar operator task has been modelled in the CIC context and mapped into a “radar operator agent”, it would not include any constraints on the possibility of multitasking. This could lead to problems when an attempt is made to compose the “radar operator agent” with the “typing agent”. One potential solution to this mismatch is to represent the constraints of possible context of use of each agent. However, this seems infeasible. Another approach would be to develop techniques to recognize the mismatches and have repair agents to fix them. The fact remains however, that human performance processing constraints must be somehow incorporated into task- based agents. This leads to the third decomposition and mapping approach as a hybrid approach that involves both task knowledge and also human processing architecture constraints. Our initial hypothesis is that the hybrid approach would be the most viable. We propose to investigate and evaluate each one of these decomposition approaches. In this investigation, many questions will arise. For example, are there tradeoffs of these three decompositional approaches with respect to reuse, composition, time spent in analysis and design of the software? What kind of cognitive fidelity characteristics each one of these approaches affords? How can one validate and evaluate these decomposition schemes? In our initial 4 month exploration, we propose to take a mutually agreed upon task, e.g. radar operation in CIC or modelling some aspect of adversaries in a MAUT. Since time is short for a full cognitive task analysis, we propose to either (a) be given an already conducted task analysis, or (b) conduct a coarse analysis of the task from publications, manuals etc. After we analyse the task, we will perform the decomposition of behaviours according to the three schemas mentioned above. This might not involve

10 Agent-based Composition of Behavioural Models software implementation, but “paper and pencil” study. Each of these decompositions will presumably make different predictions about human performance. One ways to evaluate the different decompositions with respect to cognitive plausibility would be to perform a cognitive walk-through of each one and compare against data and predictions previously obtained using another cognitive model, e.g. COGNET on the same task. To evaluate the potential for agent reusability and composition, we will need an additional new task that has common subtasks with the previous task. We need to have the cognitive task analysis of the new task also made available to us (or use crude analysis methods, as above). We will then try to reuse and substitute a common sub task agent from the first task for its “mirror agent” into the model of the second task. To test composability of agents, we will need yet a third task that can be composed of already available agents (modelled in the context of the two previous tasks). We will then analyze the feasibility of composing agents to perform the third task. One key research issue we plan to address in this regard is to what extent task taxonomies depend on the particular decision maker involved. We know that different people solve the same problems in different ways. It would seem therefore that different combinations of agents and resources might work best for different groups of decision makers. At the same time, we hope that at some high-enough level of abstraction user-independent taxonomies can be identified. The findings of this research will be used to drive the design of our (semi-)automated configuration functionality. Such taxonomies will be used to automatically derive default optimal configurations (based on available resources).

4.2. Task Representation to Facilitate Interoperability and Composability

Effective interoperability of agent functionality must be user-, task- and context driven. A task could be a simple computational task or one that involves sequence and composition of multiple activities, such as organizing a conference, or planning the transportation of supplies from an origin to a destination. These tasks may involve a single decision maker, or a team of decision makers, who coordinate to utilize their heterogeneous computational infrastructure and resources to accomplish the task at hand. Context is time- dependent and at any point in time consists of the following parts: (a) user related parameters, such as user goals, user role with associated access control parameters; (b) task related parameters, such as task specification, task state; (c) cognitive resources, (d) domain specific operational context, such as number, type and location of targets in an air campaign mission. To accomplish the interoperability vision described in this proposal, we claim that explicit representation of task and context is crucial. By explicit, we mean a representation that can be inspected by humans and systems and provide “data” to be reasoned about. There are three main benefits for explicitly representing tasks: 1. Explicit task representation allows different service agents/behavioural units with partially overlapping capabilities to recognize, extract, and reason about task-related information. 2. Explicit context representation allows different agents/behavioural units to specify what resources they need. 3. A task environment may have certain fidelity preferences that can be explicitly represented and reasoned about. For example, in a speech recognition task, latency of a single utterance recognition may be satisfactory if it is 0.5 seconds, but nor if it exceeds 5 seconds. To explicitly represent tasks, these aspects must be considered, each creating research challenges:

 Representation: Tasks can be simple or complex. A simple task is a primitive action that can be carried out by a single agent, while a complex task may need to compose multiple agent functionalities.. Questions that need to be answered for this aspect include: What is the appropriate representation for tasks? What are the reusable building blocks (primitives) that should be used? How can the task and workflow representation allow customisation to additional applications or simulation environments? 11 Agent-based Composition of Behavioural Models

How can the representation enable consistent semantic interpretation across differing ontologies? How can one represent intended effects and side effects of tasks and actions? How can exceptions be represented and reasoned about?  Composition: Complex tasks can be composed as sequences or other forms of workflow involving simple tasks. When and how should a task be added to the system? How can we safeguard that such an addition does not result in erroneous behaviour? What kind of flow language should be used for composition of tasks? How can the representation allow for structured and incremental updating in the face of new requirements? How is composition effected and validated? What architectures should be used for agent composition in support of a given task?

In the proposed investigation, we study the utility of using the RETSINA agents task representation formalisms and enriching it with constraints (both temporal and resource) relating to human performance In addition, we propose to explore different representation schemes and analyse various tradeoffs, such as expressive power and computational efficiency; imposing a standard representation of terms versus allowing potentially heterogeneous ontologies to emerge and provide tools for semantic reconciliation.

4.3. Automated agent selection and composition

One can think about a task as a hierarchy of logical models. For example, the task of “Arranging a trip to D.C. Nov 30th - Dec 2nd” can be represented as the sequential composition of the agent model for arrange air travel, arrange car rental, and arrange hotel. Each of the agents can in turn need the results of agents further down the decomposition hierarchy. Such a static decomposition can be used as a blueprint for determining what agent capabilities are needed at each point in the decomposition and issuing requests for agent capabilities to an agent repository (a middle agent in the RETSINA infrastructure). These requests will be matched by the middle agent to the stored capability advertisements and the available suitable agents will be returned to the caller to be part of the required composition. As a long-term objective, for research proposed after the initial 4-month performance period, we propose to investigate creation of new workflows by deliberative agents to guide agent composition. This can be viewed as semi-automated task analysis. In this investigation, we will explore using AI planning formalisms (e.g. [4]). In RETSINA, an interface type agent gets user goals and constraints and then, through AI planning mechanisms decomposes the overall goals to subgoals dynamically. These goal and subgoal specifications are mapped to requests for agent with the requisite capabilities. These requests then are sent to a matchmaker that matches the request to its stored agent advertisements and returns appropriate agents to be used in the fulfilment of the dynamically constructed plan. RETSINA agents already include planning mechanisms based on the Hierarchical Task Network representation that allows representation of tasks and their decompositions. During planning, different task decompositions are generated, and tested for fulfilling single or conjunctive goals. The planning algorithms in RETSINA [28,39] interleave planning, information gathering, execution and re-planning in the face of changes. The great advantage of such planning is that the same generic procedure invoked with different tasks and context parameters, can generate very different workflows and different compositions depending on the available agents.

4.4. Semantic Descriptions, Matching and Negotiation of Interfaces

In addition to the operational view of interoperability, which determines whether or not a composition of agents may be assembled in a plausible fashion (based on interface definition, granularity of services, etc.), there is the semantic view, which ensures that these exchanges make sense: i.e., that the requester agent and the agent being requested, called the service provider, have a common understanding of the “meanings” of the requested capabilities and data. Since the semantics of components and data are rarely explicit in current software engineering practice, it is difficult, if not impossible, to determine whether the assumptions of the components’ developers are compatible. Similarly, when systems are built by independent development efforts, there will be little semantic agreement between the systems.

12 Agent-based Composition of Behavioural Models

We propose to build on our previous research on creating RETSINA agent advertisements and requests, which are expressed in declarative and semantically meaningful ways, to provide metadata in which the semantics of agents and data can be made explicit. Making semantics explicit in metadata would allow agent developers and, in the future, agents themselves to detect mismatched assumptions and to create required mappings to overcome them. This principally means moving away from interfaces as simply APIs —a specification of formal parameters that describes how two pieces of a system exchange data—to a notion of capability-based interfaces. To support this we need to make fundamental architectural changes to how systems are built, as capability-based interfaces involve knowledge beyond the provider and consumer of the interface. A capability-based interface describes the purpose of the interface (what the provider can do and what the requester desires), but leaves the binding to a specific set of formal parameters initially unspecified. That binding is a product of the negotiation of the provider and requester, as well as discovery regarding what agents are available, what resources are available, and taking into consideration constraints and requirements of human performance. In addition, a big issue in such capability-based interfaces is the construction and use of ontologies that are application and task dependent and can be used in matching capability advertisements and requests. We will leverage our experience with the DARPA Agent Markup Language (DAML), which is aimed at providing semantic annotations for documents, services and agents on the Web. Sycara is part of the DARPA DAML program where she is developing DAML-S, a language for description of services that agents provide, (i.e. agent capabilities or behaviours) that allows automated capability-based discovery, transactions, and composition [1]. The W3C consortium intends to propose DAML-S as a Web standard. DAML-S goes beyond XML to provide well-founded semantics for describing the interface and interaction behaviour of web based agents and facilitating the composition of complex workflows from simpler ones. A capability advertisement, which has a concrete implementation in a programming language, is described within the DAML-S behaviour profile as preconditions, inputs, outputs, constraints, and effects. This also includes parameters that denote access control restrictions, agent availability, etc. Such descriptions will need to be extended to include information that is needed to express cognitive modelling requirements and constraints. Requesters also use behaviour profiles to submit requests to distributed registries. Adaptive interface negotiation can be realized through relaxed semantic matching algorithms and direct negotiated interaction between provider and requester. We propose to develop partial matching algorithms that match behaviour profiles and requests at the conceptual and semantic level. For this, we will leverage our experience with the Larks Project [25], where different types of partial matching were defined and partial matching algorithms were developed.

5. Statement of Work

The 4-month initial investigation will use ‘paper and pencil’ models and a series of walkthroughs with problem scenarios for the analysis of the compositional behaviour and performance of the three decomposition schemes proposed in Section 4.1We view the overall work as comprised of the following three parts: 1. Investigate the feasibility of the three different kinds of decomposition schemes. a. Select a known task that has been studied previously by the cognitive modelling community and has existing cognitive task analyses that will be made available to us. Alternatively, we can perform a ‘coarse-grained’ analysis of a task from documents, such as [16]. Call this task, Task A. b. Use the task analysis from 1a to develop task representations for each of the three decomposition schemes.

13 Agent-based Composition of Behavioural Models

c. Evaluate the performance of the three decompositions against each other for accuracy of performance predictions. d. Evaluate the performance of the three decompositions against a known model, e.g. COGNET, of Task A. 2. Investigate the substitutability of component agents a. Select and perform a similar analysis (to the one in 1a) for a task, Task B, which has common tasks with Task A. b. Use the task analysis from 2a to develop task representations for each of the three decomposition schemes. c. Analyse the feasibility of substituting a common component agent from Task B into Task A. d. Identify the tradeoffs and potential limitations involved with such substitutions. 3. Investigate the composability of component agents to accomplish a new task. a. Select a task, Task C (once again as in 1a) that has many common component agents with Tasks A and B. b. For the possible non-common components of Task C, develop agents for the three decompositions and call this set D. c. Reuse and compose component agents from A, B and D to get back a Task C’. d. Evaluate the extent to which Task C’ matches the initial specification of Task C. We hope to achieve the above parts in the four months. However, we view this investigation as an initial step in a longer-term research effort. If the 4-month feasibility analysis seems promising, we would be interested in extending the work to construct software that implements the three decomposition schemes and evaluate the automated reuse and composition of agents.

6. References

1. Ankolekar, A., Burstein, M., Hobbs, J.R., Lassila, O., Martin, D.L., McIlraith, S.A., Narayanan, S., Paolucci, M., Payne, T.R., Sycara, K., and Zeng, H. “DAML-S: Semantic Markup For Web Services”, in 1st International Semantic Web Working Symposium, July 2001, pp411-431. 2. Decker, K., Sycara, K. and Williamson, M. "Middle-Agents for the Internet", Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97), Nagoya, Japan, August 1997 pp. 578-584. 3. Decker, K., Pannu A., Sycara, K., and Williamson M., "Designing Behaviors for Information Agents", Proceedings of the First International Conference on Autonomous Agents (Agents-97), February, Los Angeles, CA., 1997. 4. Eilbert, J., Bracken, K. “A Cognitive Agent that Delivers Human-Centric Advice about System Design,” http://www.manningaffordability.com/S&tweb/PUBS/CognitiveAgentHumanCentricAdvice/CognitiveAgentHum anCentricAdvice.htm 5. Garlan, D., Allen, R., Ockerbloom, J. “Architectural Mismatch or Why it's hard to build systems out of existing parts,” Proc. the 17th International Conf. on SW Engineering (ICSE-17), April 1995. 6. Ghallab, M. et. al. “PDDL-The Planning Domain Definition Language V. 2”. Technical Report CVC TR-98- 003/DCS TR-1165, Yale Center for Computational Vision and Control, 1998. 7. Giampapa, J., Paolucci, M. and Sycara, K “Agent Interoperation Across Multagent System Boundaries”. In Proc. the 4th International Conf. on Autonomous Agents, 2000, pp179-186. 8. Giampapa, J. A., Juarez-Espinoso, O., and Sycara, K., “Configuration Management for Multi-Agent Systems”. Proc. the 5th International Conf. on Autonomous Agents, 01, pp230-231. 9. Jennings, N., Sycara, K. and Wooldridge, M., “A Roadmap of Agent Research and Development”, Autonomous Agents and Multi Agent Systems, Vol. 1, No. 1, July 1998 14 Agent-based Composition of Behavioural Models

10. Jones, R. M., Laird, J. E., Nielsen, P. E., Coulter, K. J., Kenny, P. and Voss, F. V. “Automated Intelligent Pilots for Combat Flight Simulation,” 11. Kraus, S., Sycara, K. and Evanchik, “Argumentation in Negotiation: A Formal Model and Implementation”, Artificial Intelligence, (104)1-2, pp. 1-69, September 1998. 12. Langley, B., Paolucci, M., and Sycara, K., “Discovery of Infrastructure in Multi-Agent Systems.” In Agents 2001 Workshop on Infrastructure for Agents, MAS, and Scalable MAS, 2001. 13. Lebiere, C., Anderson, J.R., Bothell, D. “Multi-tasking and Cognitive Workload in an ACT-R Model of a Simplified Air Traffic Control Task,” To appear in Proc. the 10th Conf. On Computer Generated Forces and Behavioural Representation. 14. Lee, F. J., Anderson, J. R. “Does Learning a Complex Task Have to Be Complex?: A Study in Learning Decomposition,” Cognitive Psychology 42, 267-316 (2001). 15. Lenox T., Hahn S., Lewis M., Payne T. and Sycara, K. "Task Characteristics and Intelligent Aiding", Proceedings of the 2000 IEEE International Conference on systems, man, and Cybernetics, October 8-11, Nashville, TN pp. 1123-1127. 16. Military Operations on Urbanized Terrain (MOUT), Marine Corps Warfighting Publication (MCWP) 3-35.3 17. Paolucci, M., Onn Shehory and Sycara, K., "Interleaving Planning and Execution in a Multiagent Team Planning Environment". In the Journal of Electronic Transactions of Artificial Intelligence, May 2001. 18. Payne, T., Sycara, K. and Lewis, M. "Varying the User Interaction within Multi-Agent Systems" , In Proceedings of the Fourth International Conference on Autonomous Agents, June 3-7, Barcelona, Spain, 2000. pp 412-418 19. Pew, R. W. and Mavor, A. S. Modelling Human and Organisational Behaviour: Application to Military Simulations, Washington D. C.: National Academy Press, 1998. 20. Shaw, M., and Garlan, D. Software Architecture: Perspectives on an Emerging Discipline, Prentice Hall, 1996. 21. Shehory, O. and Sycara K. "The RETSINA Communicator", In Proceedings of the Fourth International Conference on Autonomous Agents, June 3-7, Barcelona, Spain, 2000. pp 199-200. 22. Sycara, K. "Multiagent Systems", AI Magazine, Vol. 10, No. 2, pp. 79-93, Summer 1998. 23. Sycara K., Decker, K., Pannu A., Williamson M., and Zeng D., "Distributed Intelligent Agents", IEEE Expert: Intelligent Systems and their Applications, Vol. 11, No. 6, December 1996, pp. 36-46. 24. Sycara, K., Decker, K. and Zeng, D. "Intelligent Agents in Portfolio Management". In Agent Technology: Foundations, Applications, and Markets, N. Jennings and M. Woolridge (eds). Chapter 14, Springer Fall 1998, pp. 267-283. 25. Sycara, K., Klusch, M., Widoff, S. and Lu J., “Dynamic Service Matchmaking Among Agents in Open Information Environments”, SIGMOD Record (ACM Special Interests Group on Management of Data), Vol. 28, No. 1, March 1999, pp. 47-53. 26. Sycara, K., and Lewis, M. "Calibrating Trust to Integrate Intelligent Agents into Human Teams", Proceedings of The 31st Hawaii Systems Conference (HICSS-98 ), Hawaii, January 5-9, 1998. 27. Sycara. K, Paolucci, M, Giampapa, J, and van Velsen, M. “The RETSINA MAS Infrastructure” in the Journal of Autonomous Agent and Multiagent Systems (2001). 28. Sycara, K. and Zeng, D. "Coordination of Multiple Intelligent Software Agents", International Journal of Intelligent and Cooperative Information Systems, Vol. 5. Nos 2 and 3, pp. 181-211, 1996. 29. Tsvetovat, M., Sycara, K., Chen, Y. and Ying, J. “Customer Coalitions in the Electronic Marketplace”. In Proc. the 4th International Conf on Autonomous Agents, 2000, pp263-264. 30. Wickens, Chris & Justin Hollands. Engineering Psychology and Human Performance. NJ: Prentice Hall, 1999. 31. Wong, H. C., and Sycara, K. “Adding Security and Trust to Multi-Agent Systems”. In Proc. Autonomous Agents '99 (Workshop on Deception, Fraud and Trust in Agent Societies). May 1999, Seattle, Washington, pp. 149-161. 32. Wong, H.C., and Sycara, K., “A Taxonomy of Middle-agents for the Internet”. In Proc. of the 4th International Conference on Multi-Agent Systems, 2000, pp. 465-466. 33. Yamamoto, J., and Sycara, K., “A Stable and Efficient Buyer Coalition Formation Scheme for E- Marketplaces”. In Proc. of the 5th International Conf. on Autonomous Agents, 2001, pp576-583. 34. Zachary W. W., Ryder, J. M., Hicinbothom, J. H. “Cognitive Task Analysis and Modelling of Decision Making in Complex Environments,” To be published in Decision making under stress: Implications for training and simulation. 35. Zeng D. and Sycara, K. “Bayesian Learning in Negotiation”, International Journal of Human Computer Systems, Vol 48, pp.125-141, 1998.

15