Agent Based Computing for Autonomous Intelligent Software James Hendler, Defense Advanced Research Projects Agency (DARPA) and Laura Douglass, Schafer Corporation

1. Background 1.1 Defense Advanced Research coalition operations, improved Projects Agency (DARPA) intelligence gathering, and more In the complex realm of modern Agent Based Computing timely command and control. military operations, commanders are dealing with increasingly diverse 2. Control of Agent Based missions, including operations other Systems (CoABS) than war, expeditionary missions, and controlling dangerous situations The CoABS program consists of in dynamic and uncertain three elements—the agent grid, environments. All of these missions agent interoperability standards and are further complicated by the the scaling of agent control requirement for joint and coalition The Defense Advanced Research strategies. CoABS has developed coordination. Achieving decision Projects Agency (DARPA) has a novel tools for run-time superiority in these situations is proven record of developing interoperability among becoming increasingly difficult and revolutionary new capabilities in the heterogeneous systems and is complex. The need to gain the right area of software agents, and developing new tools to ensure information at the right time for each continues to take a leadership role in rapid, real-world system integration type of decision maker in this this field. DARPA is currently with other software agents and complex scenario is leading the focusing its research in this area on entities such as servers, databases, Department of Defense (DOD) several initiatives in Agent-Based legacy systems and sensors. toward distributed information Computing (ABC). The ABC suite systems that are managed and of programs will provide the accessed in a network-centric building blocks for understanding manner. and implementing the Software Agents that are the foundation for A key technological innovation intelligent distributed information capable of handling the complexity systems and thus will enhance our of modern warfare is that of software ability to reach decision superiority. agents. Agent-based computing The three initiatives in this effort are focuses on the development of the Control of Agent-Based Systems distributed computational entities (CoABS) program, the DARPA (software agents) which can act on Agent Markup Language (DAML) CoABS has focused on supporting behalf of, mediate or support the program and the Taskable Agent joint and coalition military actions of human users and Software Kit (TASK) program. autonomously carry out tasks to operations. In these domains it is achieve goals or assist the activities Using these programs DARPA will necessary to assemble disparate of the users in achieving those goals. create an environment where information systems into a coherent In the military, using these agents intelligent distributed computing interoperating whole without will improve our information and will be as easy and ubiquitous in the redesigning or reimplementing the decision management capabilities future as data exchange is now. The systems into a single common and thus drastically reduce the payoffs to DOD will be seen in architecture. To achieve this, it is complexities of modern warfare. improved interoperability of necessary to construct an heterogeneous systems, support for information environment where

2 STN 4-4: Software Agents these systems are accessible to each component for Millennium with Berners-Lee and the World other and their users, resulting in an Challenge 2002, operating in the Wide Web Consortium to ensure intelligent distributed information Agile Commander ATD sponsored DAML fits with its plans for the system. by Army CECOM, and contributing semantic web and becomes the to the Air Forces’ ATD in effects recommended web language for To achieve an operational based operations and several of the semantic interoperability. The most environment for such a distributed Joint Battlespace Infosphere-related up to date version of the language is information system, a mechanism is projects. available on the web, as are needed to enable the dynamic, documents on its use, tools, and runtime integration of agent, object, 3. DARPA Agent Markup demonstrations (www.daml.org). and legacy software components into Language (DAML) applications. The key to this An exciting military application for capability is the CoABS Grid, which Currently, the languages used on the DAML is the Horus initiative, a is software that provides a service- World Wide Web impose a joint DARPA-Intelink Management based middleware infrastructure for significant limitation on agent based Office project, which introduces a managing complex information computing. Going far beyond XML, new way of organizing and flows. The Grid supports the the DAML program’s goal is to presenting intelligence information development of applications for develop a language that allows to users of Intelink, the DOD dynamic domains such as military information content to be expressed network for distributing classified command and control, which require on the web in machine-readable intelligence information. Other early the composability, adaptability, and ways. This will make for a whole adopters of DAML include the autonomy provided by software new generation of Internet Center for Army Lessons Learned agents. capabilities. Prototype tools are and the Oak Ridge National being developed to show the Laboratory. Current “legacy” systems can be potential of such markups to provide brought to the grid through software revolutionary applications that will 4. Taskable Agent wrappers and service descriptions, change the way humans interact with Software Kit (TASK) allowing their functionality to be information. tapped without major recoding. In Developing agent-based software is addition, the cooperative nature of currently something of a “black art” the problem solving, using — an arbitrarily complex process existing software that has many ad hoc methodologies components, allows both but no significant modeling military and industrial solutions. Very few rules or protocols users to develop large scale have been agreed on for analyzing applications without large and modeling complex, large-scale, scale software agent systems that interact with development efforts. distributed heterogeneous information systems. As a result, In the next year, CoABS will focus As reported in PC Week, the World DARPA is investing in on transitioning those run-time Wide Web Consortium (W3C) has understanding the principles integration capabilities to the embraced DAML as a step toward underlying agent-based computing. military, particularly for command what Tim Berners-Lee, the creator of and control applications. Among The TASK program’s goal is to the World Wide Web, calls the others, CoABS will be providing C2 extend the current scientific and “semantic web.” DARPA is working infrastructure in the Navy’s Continued on page 4 Data & Analysis Center for Software (DACS) 3 Agent Based Computing for Autonomous Intelligent Software Continued from page 3 mathematical foundations of agent- DARPA, and run in cooperation Foundation Fellowship, is a member based computing and to add with the services and other DOD of the US Air Force Science precision to the engineering of agent- agencies. These programs provide Advisory Board, and is a Fellow of based systems. TASK researchers funding to over fifty universities, the American Association for will explore the application of companies, and research institutes Artificial Intelligence. mathematics not traditionally used who are cooperating in developing Ms. Laura Douglass is currently at for computer modeling, such as both underlying science and Schafer Corporation, where she statistical physics and chaotic implemented technology for provides technical and analytical dynamics, to explore the modeling of developing and fielding agent based support to the Agent Based multi-agent systems. systems for the Department of Computing program at DARPA. Defense. Previously, she was the Special Biographies Assistant to the Deputy Under Secretary of Defense for Science and Dr. James Hendler is currently Technology, and spent several years working at the Defense Advanced providing program management Research Projects Agency (DARPA) support to the Office of Naval as a Program Manager in the Research while on contract from the Information Technology Office. Pennsylvania State University’s Dr. Hendler is on a leave of absence Applied Research Laboratory. from the University of Maryland Contact Information TASK researchers will demonstrate where he continues to retain his their algorithms and approaches status as a professor and head of Dr. James Hendler against a set of “research exploration both the Autonomous Mobile Program Manager frameworks” chosen to be of great Robotics Laboratory and the DARPA / ITO relevance to a very wide range of Advanced Information Technology 3701 Fairfax Drive DOD problems. Specific focus will Laboratory. He has joint Arlington, VA 22203-1714 be placed on dynamic command and appointments in the Department of E-mail: jhendler@darpa.mil control problems, the fundamentals Computer Science, the Institute for Phone: 703-696-2238 of cooperative systems, and Advanced Computer Studies and the Fax: 703-696-2203 understanding information-agent Institute for Systems Research, and behavior. In this way, new he is also an affiliate of the Electrical Laura Douglass techniques for resource assignment Engineering Department. Sr. Program Manager against chaotic problems, hybrid Schafer Corporation He is the author of the book control of dynamic systems, and 4301 North Fairfax Dr., Suite 700 Integrating Marker-Passing and modeling the information dynamics Arlington, VA 22203 of complex IT systems will be Problem Solving: An Activation explored and exploited. Spreading Approach to Improved E-mail: ldouglass@schafercorp- Choice in Planning. He has also ballston.com 5. Conclusion authored over 100 technical papers Phone: 703-516-6033 in artificial intelligence, robotics, Fax: 703-516-6065 In this short article, we have intelligent agents and high described a vision of the software-of- performance computing. Hendler the-future that drives the agent-based was the recipient of a 1995 Fulbright computing programs supported by

4 STN 4-4: Software Agents ©Msg*Log: E-mail-based Agent Messaging to Improve Robustness in a Distributed Logistics Planner Tom Bannon, Steve Ford, Craig Thompson, and David Wells, Object Services and Consulting Inc. 1. Introduction move. The existing Cougaar 2. The Logistics communications mechanism is based Problem The Department of Defense (DOD) on Java Remote Method Invocation Joint Vision 2020 [1] identified (RMI) extended with message Logistics planning and execution is a “focused logistics” as a key queues and retry policies. This huge and complex problem, with component for achieving full performs well under stable operational and financial spectrum dominance in future conditions, but rapidly loses its implications for the DOD. The DOD conflicts. To that end, the Defense ability to deliver messages as chaos states that: Advanced Research Projects Agency increases due to inherent properties (DARPA) Advanced Logistics of the RMI protocol. This failure If Desert Shield/Storm logistics Project (ALP) has successfully had optimized lift scheduling, will ultimately make it impossible detailed coordination between demonstrated the use of an for Ultra*Log to achieve its stated planning and execution, and extensible collection of loosely robustness and performance goals of visibility into the logistics pipeline, coupled agents to generate, monitor, suffering “no more than 20% significant improvements would and dynamically modify complex, capability degradation and 30% have been possible in deployment multilevel logistics plans. ALP surges and resource sequencing, performance degradation under planning and replanning driven by agents reside on an agent conditions of 45% information changing requirements, and infrastructure called Cougaar infrastructure loss in an environment improved control over the logistics (Cognitive Agent Architecture). The of 90% of maximal real-world pipeline. [11] current DARPA Ultra*Log program chaos”. Estimates are that this would have is improving the robustness, allowed the campaign to have scalability, and security of the This article describes the ongoing Msg*Log (Messaging Logistics) concluded 100 days sooner, reduced Cougaar agent infrastructure with the quantity of material transported the goal of allowing ALP agent- effort to improve the robustness of Cougaar agent communications by by 1M Tons, and reduced logistics based planning to survive and costs by $800M. These represent perform in a highly chaotic transparently adding additional message transport mechanisms based improvements of approximately environment in which unforeseen or 45%, 33%, and 40%, respectively. uncontrollable events interfere with on the robust and ubiquitous E-mail the ALP planners’ operation, and and Netnews protocols. The Technically, logistics planning and with the ability of the military resultant three transport mechanisms monitoring poses a challenging procurement, transport, and have different operational properties problem. It is clearly a large warehousing organizations to that make each suitable under problem as evidenced by the Desert physically execute the plans different conditions. To capitalize on Storm example. The types of generated by ALP. this, Msg*Log provides a higher planning activities are varied, level Adaptive Message Transport including inventory, packing, route Because the ALP planning process is capability to automatically switch planning, and scheduling, each highly distributed, reliable and among transport mechanisms to cope subject to multitudinous and varied timely message delivery to multiple with degradation of the constraints. The problem is highly planning agents is mandatory, even communications infrastructure and distributed, yet interconnected; the as the communications infrastructure to adjust the level of redundancy. supported organizations are degrades and as the planning agents

Msg*Log is copyrighted by Object Services and Consulting, Inc. All rights reserved. © Copyright 2001 Continued on page 6 Data & Analysis Center for Software (DACS) 5 Msg*Log: E-mail-based Agent Messaging Continued from page 5 autonomous and can reside More information about ALP and interaction. Thus, robust ALP anywhere on the globe, but not only Ultra*Log can be found at [5] and planning requires “survivable” does an organization need to know [6]. Complete details of the internode and intercommunity its own part of the overall plan, it Cougaar agent infrastructure on communication that will continue to needs to know enough about other which ALP planning agents reside operate as networks partition or organizations and their plans to be can be found at [7]. become swamped as planning agents able to collaborate in planning and and nodes migrate along with their The Cougaar architecture provides execution. Complicating matters owning organization or as part of an the technical basis for a loose further, these organizations can overall survivability plan. Figure 1 confederation of ALP planners that move during the course of an features a graphic by Dr. Todd interoperate via message passing. operation, sometimes in an Carrico of DARPA, which illustrates ALP planners, corresponding to unexpected manner. Additional some of these concepts. Cougaar agents can be grouped into dynamism enters the system in the nodes and communities that perform The current Cougaar form of changed logistics requests more complex planning or implementation uses Java RMI and the potential loss of logistics monitoring. Agents generally (Java’s RPC mechanism) for resources as the military operation perform a very specific task (e.g., internode communication. The progresses, any of which may job-shop scheduling). A node is strength of RMI is its tight require plans to be modified on-the- Cougaar software that supports one integration with Java and its speed. fly. The tempo and scope of or more agents within a single Java Its weakness is its dependence on operations can change rapidly as the Virtual Machine (JVM); typically point-to-point direct connectivity situation progresses from peace to several related agents will be placed between sender and receiver and on deployment to combat to into a single node to optimize accurate knowledge of the peacekeeping and back to peace. communications (e.g., a scheduler, recipient’s IP address and port ID. Finally, all this must be packing planner, and associated data Under low chaos conditions, these accomplished in an environment in access supporting a warehousing requirements are generally met and which opponents are attempting to organization could reside in a single when violated are easily remedied, prevent logistics activities from node). A community is a logical making RMI a good choice, being planned and executed by both grouping of agents, typically especially when combined with kinetic and information attacks. associated with a DOD organization message queues and retry policies 3. Agent-based (e.g., the US Transport Command). (which Cougaar currently supports). Logistics Planning Communities can be hierarchical. However, as chaos increases, these Communities have no physical requirements are less likely to be A problem of this nature calls for a interpretation in Cougaar, but since met: partitions become more highly flexible, scalable, and robust they correspond to an organization, frequent and take longer to repair, computing solution that can be there is often at least some physical while at the same time, nodes are incrementally improved and “closeness” between members of a more likely to migrate. In the worst expanded. The DARPA Advanced community (e.g., on a LAN or case, end-to-end connectivity Logistics Project (ALP) and its within a single base). A society is a between critical pairs of nodes may successor Ultra*Log are developing collection of agents and communities never exist, at least not often enough an agent-based computing solution working on a common problem. to produce plans in an acceptable time. Thus, even though an RMI- to distributed logistics under stress. All but the simplest plans will based approach can tolerate highly involve interaction of multiple The following presents sufficient chaotic conditions, it will not be able agents, node, or communities and information about ALP and Cougaar to accomplish much during them, to understand the role of Msg*Log. will involve considerable remote 6 STN 4-4: Software Agents not requiring control over intermediate nodes already exist, namely Simple Message Transfer Protocol (SMTP) and Network News Transport Protocol (NNTP), the E-mail and UseNet newsgroup protocols. STMP and NNTP-based internode communications mechanisms have two characteristics that recommend them. Like RMI, SMTP and NNTP are both well defined, mature protocols with many implementations, so any system built on top of them will have industrial- strength underpinnings. The many SMTP and NNTP servers already in place ensure wide “reach” for the ALP system; in fact allowing Figure 1: Technical Vision - Applying Agent Technology to the Logistics Domain delivery of messages to places unreachable by RMI. More importantly from a survivability which will prevent Ultra*Log from Two ways to address this suggest perspective, SMTP and NNTP have reaching its goal of “no more than themselves. RMI can be further different operational characteristics 20% capability degradation and extended with intermediate servers that, while making them slower than 30% performance degradation under and naming and redirection RMI, allow them to continue to conditions of 45% information mechanisms to provide a store-and- function with reasonable infrastructure loss in an environment forward capability and support for performance at levels of chaos that of 90% of maximal real-world mobile agents, or alternate transport would preclude RMI from delivering chaos”. mechanisms that already provide any messages at all. these capabilities can be adapted for 4. Msg*Log Cougaar use. RMI extensions would Unlike RMI, which requires end-to- Clearly, an important requirement require the ability to install software end connectivity between sender and for overall ALP/Cougaar on intermediate network nodes; a receiver, SMTP and NNTP use survivability [5][6][7] is for inter- degree of access not generally store-and-forward mechanisms that agent messaging to be reliable and available in third party networks. only require individual hops along a timely even when the Since use of such networks is delivery path to be functioning. A communications infrastructure is required for many applications, consequence is that SMTP and stressed. Since Cougaar’s RMI- including logistics, RMI extensions NNTP can deliver messages even if based message transport is not alone will be insufficient. sender and receiver are never reliable under those conditions, Fortunately, robust protocols that connected at any point in time. In a Cougaar needs a better way to allow message delivery across frequently partitioning network perform remote messaging when frequently partitioning networks and environment, this is clearly an chaos increases. that support mobile recipients while advantage. Continued on page 8 Data & Analysis Center for Software (DACS) 7 Msg*Log: E-mail-based Agent Messaging Continued from page 7

Second, RMI requires that a sender existing Cougaar RMI-based a Java VM) in which multiple have knowledge of its recipient’s IP transport, Msg*Log uses Java Cougaar agents can run as threads. address and port ID, which is serialization as an efficient external Each Cougaar node has at least one difficult to ensure if ALP agents/ representation for both SMTP and E-mail address (an E-mail account nodes and logistics organizations are NNTP message transport. Other on at least one E-mail server used for moving around in cyberspace or the encodings are certainly possible; a sending/receiving messages); real world. Since in both SMTP and previous OBJS E-mail-based agent multiple accounts at multiple servers NNTP, messages are “picked up” communications system called are possible in the interest of sending rather than delivered, all they require eGents used XML as en external and receiving messages via multiple is that a sender know how to contact encoding in the CoABS 24x7 Grid routes. Both the SMTP and NNTP- some “mail drop” where a recipient [15] where inter-language portability based transport mechanisms include can look for its messages. This and human readability was more the ability to pick up E-mail or allows message delivery in cases crucial than performance. news-encode inter-agent messages on where the sender does not know an appropriate schedule and will Msg*Log’s implementation uses where a mobile recipient resides, but have the ability to try alternate sites. existing E-mail and news servers, the recipient knows where it can and existing Java SMTP, POP3, and The SMTP and NNTP-based pick up its mail. Since there can be NNTP clients. This enables transport mechanisms integrate many mail drops and these can be applications to control routing at a easily into the Cougaar architecture, relatively stationary, this provides a level to which they have access, via which provides explicit support for much more robust solution. This established Internet services that swapping transport mechanisms by can be exploited to further improve route public messages between subclassing the Cougaar robustness by sending the same servers out of line with the direct MessageTransport class. This message to multiple servers, making path between two Cougaar nodes. packaging makes the existence and it more likely that a recipient will be With respect to caching, it should be use of transport alternatives able to access its messages from noted that Internet routers do not transparent to Cougaar agents and in somewhere, even in the face of many cache packets for significant periods fact, to the rest of the Cougaar failures. NNTP in particular is of time waiting for a connection to infrastructure. Cougaar agents need efficient at distributing messages the next router, so an additional not know which transport with high fan out through multiple advantage of our design over mechanism is being used. If layers and large numbers of servers, dynamic network routing is that ours additional transport mechanisms making it particularly useful under operates in the presence of great were needed (e.g., based on highly chaotic conditions. latency and full disconnectedness. broadcast or groupware), they would The new Msg*Log SMTP and be added in the same manner. Cougaar’s NameServer is NNTP-based transport mechanisms implemented (NameServer is an The ability to route messages via convert Cougaar inter-agent interface, not a class) to map alternate message transports messages into E-mail messages or Cougaar nodes to one or more E- improves the flexibility of internode news postings, transmit them via the mail addresses. While there are communications. However, the existing E-mail or news several possibilities for mapping bigger benefit comes by providing an infrastructure, convert them back to Cougaar agent addressing to E-mail adaptive mechanism to dynamically Cougaar messages on the receiving addressing, for compatibility with choose between those message end, and hand them off to the the Cougaar architecture, our current transports and parameterize their use existing Cougaar mechanisms for approach requires an E-mail address on a message by message basis using delivery to the appropriate local for every Cougaar node (effectively network QoS information and higher agents. For compatibility with the

8 STN 4-4: Software Agents level application-dependent messaging via RMI, SMTP, and it is never possible to state that a intelligence about the relative NNTP, but also to be extensible with given plan could not be improved importance of the message and the future MessageTransport with more time or better planning likelihood of failure of its delivery implementations. tools or smarter logisticians. Thus, for reasons not detectable via QoS an appropriate baseline for planner metrics. For instance, if direct 5. Metrics behavior is: (1) the quality of the connectivity is not available between The effectiveness of Msg*Log in logistics plan, (2) generated in some two nodes when a message is sent, improving Cougaar robustness is nominal time, (3) by a given ALP then RMI will not work; if the need evaluated by how well it increases society when, (4) requested to satisfy to send the message immediately is the survivability of ALP logistics a given set of (possibly changing) critical, then alternate routes to the planning capability in the face of logistics requests using (5) a given recipient could be considered. stresses to the computing and set of (possibly changing) logistics Knowledge of the network communications environment. To resources (truck, ports, etc.), when topography combined with the judge this, we want answers to the (6) the computing resources partial information about the failure following questions: (computers, networks, data sets) might suffice to identify a reliable used in the planning process are not route to the remote node via a chain l When Msg*Log alone is added to stressed in any way (no failure, of SMTP servers, or a set of routes Cougaar, how much do logistics degradation, or corruption). of probable reliability. Alternatively, plans degrade in quality and A test measures the effectiveness and a security agent might have advised timeliness of generation when operational behavior of Msg*Log that a route over which direct subjected to various kinds and across a range of stresses to the communication would travel is not intensities of communications computing environment. A test uses to be trusted, and alternate routes stress, and how does that compare a single test case (an ALP society, a should be considered. Finally, to identical planning conducted computing environment, a set of general policies could state that under similar conditions without (time varying) logistics requests, and under certain operational conditions Msg*Log? a set of (time varying) logistics (an InfoCon), that public networks When all Ultra*Log l resources) to exercise the system. are not to be used for any data unless enhancements are added to Testing will be done twice this year, encrypted, regardless of the Cougaar, how much do logistics once with a moderate size test case information’s inherent sensitivity. plans degrade in quality and and the other with a 1000 node ALP timeliness of generation when This is the idea behind Msg*Log’s society. In future years, tests of subjected to various kinds and Adaptive Message Transport increasing complexity will be intensities of communications (AMT), a higher level transport defined by the Ultra*Log program’s stress, and how does that compare mechanism with the ability to select assessment team. Within a test, to identical planning conducted among, parameterize, and monitor several individual experiments inject under similar conditions without the behavior of the various lower- varying types and intensities of Ultra*Log enhancements? level transport mechanisms known to stress. A model for injecting it. This mechanism merges disparate Robustness in the face of synthetic stresses can be found in sources of relevant information, environmental stresses can only be [14]. As the computing environment plans, and executes an appropriate measured by having a baseline of is stressed, it is anticipated that communications strategy using the functionality of the unstressed logistics plan quality will degrade. available transport mechanisms. system. There is no formal notion of The extent to which logistics plan AMT is designed to manage not only a “best possible logistics plan”, since

Continued on page 10 Data & Analysis Center for Software (DACS) 9 Msg*Log: E-mail-based Agent Messaging Continued from page 9 quality does not degrade is a conditions. These operational Three kinds of information must be measure of Cougaar robustness. statistics are to be used to set collected to answer the above Stresses to be applied in the tests Msg*Log policy [3] and to direct questions: include: future development. For these l Message delivery information purposes, we want answers to the Kinetic attacks (when the message was sent, the following questions about the transport mechanism used, l Permanent loss of behavior of the Msg*Log transport message size, delivery time, geographically adjacent mechanisms: distance between sender and communications links l What is the mechanism’s receiver and the distance the l Permanent loss of target agents performance under various kinds message actually traveled (in (what will M*L do if the and intensities of communications terms of LAN and WAN hops), recipient dies?) stress, and how does that compare whether the recipient was fixed or l Permanent loss of SMTP and/ with the performance of the mobile (not necessarily known or NNTP servers original RMI-based transport until message is received), and Information warfare attacks under similar conditions? the environmental stresses l What is the mechanism’s resource present. l Denial of service attacks place utilization under various kinds l Disk and memory utilization at excess load on communications and intensities of communications the SMTP & NNTP servers at links stress, and how does that compare various time points (for l Network partitioning with the resource utilization of correlation with message send l Rolling network partitions the original RMI-based transport? times to create a map of disk (partition moves across l Under what kinds and intensities usage as a function of number network) of communications stress do each and size of messages) l IW attacks crash SMTP and/or of the Msg*Log transport l Bandwidth utilization NNTP servers mechanisms become unable to From the raw statistics captured l Spamming of STMP and/or perform reliable and timely during testing, the following NNTP servers (causes delays message delivery, and how does statistics should be computed: or overflows) this compare to the behavior of the original RMI-based transport? l For each transport mechanism, a l Protocol attacks on RMI, scatter plot of message delivery l Under what kinds and intensities SMTP, and/or NNTP protocols times as a function of distance of communications stress does (renders a transport unusable) between sender and receiver Msg*Log (including the ability to l Loss of stored messages at (actual distance, not how far the select the “best” transport SMTP and/or NNTP servers message went) mechanism) become unable to In addition to measuring the perform reliable and timely l For Msg*Log, a scatter plot of effectiveness of Msg*Log in message delivery, and how does message delivery times as a increasing Cougaar robustness, we this compare to the behavior of function of distance between are interested in collecting the original RMI-based transport? sender and receiver (actual operational or infrastructure level distance, not how far the message l How does Msg*Log affect the statistics about the concrete behavior went) performance of the original RMI- of Msg*Log under different based transport?

10 STN 4-4: Software Agents a function of the stress level. Service [12][13] and evolve the Figure 2 shows the expected shape Msg*Log design and implementation of the graphs (read “stress level” for in concert with the evolving Cougaar “chaos level; the terms have been architecture over the life of the used interchangeably in the past, program. although the term “stress” is preferable as “chaos” has a specific, 7. Plans and different, meaning). Only the In the near term, we will be mean delivery time is shown, but the completing the integration of actual graphs will be scatter plots. Msg*Log with the remainder of the Figure2: Mean Delivery Time as a For all of these, standard statistics Cougaar architecture, participating Function of Environmental such as mean delivery time in survivability testing, doing Stress (Sample) (excluding undeliverable messages) general performance improvements, and writing some simple policies for l For each of the above, the and fraction of messages delivered Adaptive Message Transport. number of undeliverable in less than a given time T can be messages at each distance (factor computed. If T is chosen Beyond that we will be developing out messages that were not meaningfully with regard to the more sophisticated adaptation delivered due to agent failure, Ultra*Log concept of operations, it mechanisms and developing since Msg*Log can’t do anything becomes a useful predictor of communications policies compatible about those) whether a given transport with evolving higher level mechanism, or Msg*Log itself, is The different experiments in a test survivability and security policies. likely to be able to provide timely We will also be investigating other will be conducted at different “levels message delivery under given stress of stress”. The scatter plots techniques such as stealthy message and distance conditions. The delivery, traffic analysis masking, produced for the experiments can be presumed graph of this for a given sliced to produce scatter plots of the and service bidding that could well distance and time T is shown in be supported by message transport effect of stresses as follows. For Figure 3. each inter-agent distance, a scatter schemes that allow delivery to plot is made that maps delivery 6. Status multiple (perhaps intentionally times (at that inter-agent distance) as spurious) addresses and delayed A Preliminary Msg*Log Design is message pickup at convenient (and complete, and implementation of safe) times. the E-mail and NNTP Message There is a strong possibility that we Transports are underway. A proof- will add additional message of-concept implementation of transport mechanisms, in particular E-mail Message Transport has been one based on extensions to RMI as demonstrated using a sample discussed in Section 4. As noted Cougaar society. The first software there, RMI could be extended with delivery will include a small intermediate servers, name mapping, number of hand-coded Message and redirection to achieve Transport policies to select and functionality similar to the SMTP parameterize the three Message and NNTP transport mechanisms Figure 3: Probability of Message Transports. We expect to integrate Delivery as a Function of with BBN’s QoS Monitoring Environmental Stress (Sample) Continued on page 12 Data & Analysis Center for Software (DACS) 11 Msg*Log: E-mail-based Agent Messaging Continued from page 11 discussed in this article. However, cost routes that reconfigure when the degrade. The severity of this the operational behavior might be network is disturbed appear problem would depend on how quite different. In particular, SMTP interesting. frequently routing plans would have and NNTP servers are optimized to to change in response to network Another area of interest is using minimize resources required per loads and disruption. It is possible Msg*Log to support groupware such message (cpu/message and storage/ that “normal” selection and routing as ISIS, Horus, and Ensemble [10]. message), but not necessarily for could be done by Msg*Log policies Groupware systems allow the throughput speed (time/message), as described above, with only “hard” definition of groups of distributed although that is generally pretty problems delegated to the logistics processes and guarantee certain good as a side effect. An RMI-based tools. properties of message delivery to the transport mechanism using store- group members, including message and-forward servers built specifically 9. Conclusion ordering, quorum calls, and handling to relay a message as fast as possible of network partitions and This article has described a system might be a useful alternative in reconnections. This might well be a to improve the survivability of an proprietary networks where it is useful capability for both logistics agent infrastructure (Cougaar) that possible to install the servers at applications and agent systems in is being used to support a complex intermediate nodes. general, since it would allow logistics planning system. 8. Potential Future synchronized communication with Techniques to measure the improved survivability were also presented. Work groups of agents working on a single task – say logistics “what-iffing”. There has long been speculation that There are a number of more Groupware systems all rely on so-called “ilities” (survivability, speculative avenues of research message delivery by a lower level scalability, quality of service, etc.) which we wish to pursue. transport mechanism; as such can best be added to software Msg*Log is complementary. In the architectures by exercising control We plan to examine the possibility of case of Horus, this possibility is exercising direct control over over the communication paths in a enhanced by a replaceable protocol complex system. This conclusion message routing at the network level stack and that could be adapted for traffic that can be limited private was partially born out by our own relatively easily to use Msg*Log Object-Oriented Database System networks. It is certainly feasible that transport. Interesting technical some logistics communities might (Open OODB) [16] work on questions arise when different sentries, later similar work at Object have dedicated intranets supporting members of a group must be reached at least part of their computing Management Group (OMG) on via different transport mechanisms interceptors for adding security to assets. We have not chosen this with different delivery properties. approach initially because over the systems, complex ACME connectors open Internet today, there is It would also be interesting to replacing simpler ones, and our insufficient access to routing explore using the ALP planning Object Services and Consulting information and insufficient control capabilities to choose message (OBJS) work with MCC on the of routing at the network level transport protocols and routing Object Infrastructure Project. If available to the application to make strategies; in effect to treat message Msg*Log can significantly improve it work. In this regard, recent work delivery itself as a logistics problem the survivability of a complex agent on intelligent swarms [9] in which occurring at the infrastructure level. infrastructure by exercising such insect simulations based on simple A concern is that routing plans could control, it would serve as an rules and weighted pheromone trails not be made fast enough, with the important data point in this larger are applied to networks to find low result that performance would architectural quest.

12 STN 4-4: Software Agents Biographies Tom received a B.S. in Electrical Prior work includes the first Engineering from Texas A&M Common Lisp implementation for Tom Bannon is a Senior Member of University in 1984, and a M.S. in the Sun workstation and a few years the technical staff at Object Services Computer Science from The of business application development and Consulting Inc. (OBJS). He is University of Texas at Dallas in in both commercial and academic currently working on a survivability 1991. environments. project, where OBJS is developing technology to allow graceful At OBJS, Steve Ford researches, Steve received an M.S. in Computer reorganization and restoration of develops, and administers software Science and a B.A in Biological threatened or failed pieces of and hardware. He is currently Sciences from Indiana University. distributed object systems. It can be working on eGents, an agent system Dr. Craig Thompson is the described as fault tolerance growing that communicates over E-mail. President of Object Services and into the distributed object world Past projects at OBJS include Consulting, Inc. Craig’s background (such as engendered with CORBA), developing techniques to improve spans compositional middleware with dynamic resource, situation, the Survivability of CORBA-based architectures, federation, web and threat, and failure models. systems and a survey of object integration, virtual office and programming languages and Tom has a strong interest in visual virtual enterprise, annotations, query environments for development of design, and specifically 3D. Tom architectures, object database Internet-based applications. works in his spare time on Metal systems, and natural language Sculpture. Prior to joining OBJS, Steve worked interfaces. on the DARPA Open OODB and its Prior to OBJS, Tom worked in He taught database and AI at the predecessor projects at Texas various corporate research labs at University of Tennessee, Knoxville, Instruments’ Computer Science Texas Instruments over the last 8 from 1977 to 1981. He joined the Laboratory. He was involved in the years. His main projects were in the Central Research Laboratory at design and implementation of areas of object-oriented databases, Texas Instruments (TI) in 1981 and virtually all components of those virtual reality game systems, people was elected Senior Member of systems, but was primarily interested tracking vision and mapping Technical Staff in 1985. At TI, he in issues related to programming systems, and video codecs (H.263 co-invented menu-based natural language API, storage and memory and MPEG4). Before that, he language interfaces, deployed in the management, communications, worked for 2.5 years in the defense DARPA/USN FRESH program in object representation and translation, area of TI, primarily building a the mid-1980s; productized an namespace management, database system for a board-level extensible object-relational DBMS performance, and the application of ECAD system. Immediately prior to on the TI Explorer Lisp Machine in OODBs to image understanding joining OBJS he spent 6 months as a 1985; led TI research projects in environments. contractor to a large telecom hypermedia and engineering company building web and Java- Before that, Steve developed system databases in 1987-89. based GUIs for a network software for the TI Explorer Lisp From 1990 to 1995 Craig was management testing system. Machine from 1983 to 1988. His program manager and co-principal primary duties were fixing bugs and investigator on the DARPA Open fleshing out Richard Stallman’s Lisp Object-Oriented Database System Machine code.

Continued on page 14 Data & Analysis Center for Software (DACS) 13 Msg*Log: E-mail-based Agent Messaging Continued from page 13

(Open OODB). The project was an Craig holds six patents, is an IEEE OODB, which was deployed at 25 important influence on the Senior Member, has published over government approved alpha sites and computing industry move toward 30 papers in journals and is currently licensable as a research component software, influential in conferences. He is a nationally product from TI. Wells developed leading to the Object Management recognized leader in object the open systems principles and Architecture (OMA) of the Object technology standards. He co- mechanisms that make Open OODB Management Group (OMG). organized the Application extensible by third parties. The Open Integration Architectures Workshop OODB meta architecture served as In 1995 Craig Thompson, David (1993) which convened principals one of the bases for the Object Wells, and Steve Ford founded from over 20 standards groups on Management Group’s (OMG) Object Services and Consulting, Inc. the theme of standards convergence Object Services Architecture, which (OBJS). Craig won and served as for enterprise computing. He also has been adopted by over 500 principal investigator on the DARPA co-organized the Joint W3C/OMG computer vendors and object- contract Scaling Object Services Workshop on Distributed Objects oriented software companies as the Architectures to the Internet (1995- and Mobile Code (1996) and basis for developing interoperable 1998) and Agility: Agent -Ility organized the OMG-DARPA systems. David is experienced in Architecture (1998-2002). Workshop on Compositional leading large research and Thompson provided architectural Software Architectures (1998). development projects, having been consulting and review for the PI of four research contracts with DARPA TRP National Industrial Craig received a B.A. in total funding of $10.7M. He holds Information Infrastructure Protocols Mathematics from Stanford three patents and two patent (NIIIP) Consortium (1995-1997), University in 1971. He received an applications in the areas of database the MCC Object Infrastructure M.A., and a Ph.D. in Computer software techniques and data Project (OIP) (1997), and the Science from The University of security, has authored five journal DARPA ISO Advanced Information Texas at Austin, in 1977 and 1984 articles, twelve conference Technology Services (AITS) respectively. proceedings, and twenty-four invited Architecture (1997-1998). He is David Wells is the OBJS Vice external presentations and panels. organizer and co-chair of the OMG President, and head of software He is reviewer for various journals Internet Platform Special Interest research. and conferences including the Group, chartered to merge the OMG International Conferences on Very OMA architecture with Internet and He was an Assistant Professor in the Large Databases, the VLDB Web standards to enable large-scale Computer Science Department at Journal, ACM Transactions on Internet-enabled object-based Southern Methodist University from Databases, IEEE Computer, IEEE distributed computing, and also the 1980 to 1986 where he conducted Transactions on Software OMG Agent Platform Special research in databases, computer Engineering, and SIGMOD Interest Group, chartered to meld security, and computer graphics. He Conferences. distributed object and multi-agent joined Texas Instruments in 1986 systems. and until 1995 was principal David received his D. Eng. degree in software architect and Co-Principal Computer Science from the Investigator for the DARPA Open University of Wisconsin-Milwaukee in 1980.

14 STN 4-4: Software Agents References [10] Horus, a Flexible Group [16] Architecture of an Open Communication System, Renesse, [1] Joint Vision 2020, US Object-Oriented Database R., Birman, K. and Maffeis, S., Department of Defense, Staff, J-7, Management System, Wells, D.L., Communications of the ACM, Joint Doctrine Division Support Blakeley, J., Thompson, C.W., April 1996. Group, [email protected], IEEE Computer, Vol. 25, No. 10, http://www.dtic.mil/jv2020/ [11] Advanced Logistics Project: October 1992. Program Briefing, Carrico, T., [2] White Paper: Application- Contact Information DARPA ISO, 1998, Driven Ultra*Log Robustness http://www.darpa.mil/iso2/alp/ Tom Bannon Metrics, Wells, D.L., OBJS, April, public_access/pr_brf_4398. Object Services and Consulting Inc. 2001, http://www.objs.com/msglog/. [12] Specifying and Measuring 5111 Purdue Ave. [3] Msg*Log Message Transport Quality of Service in Distributed Dallas, TX 75209 Policy, Thompson, C.W and Wells, Object Systems, Loyall, J.P., Phone: 214-902-8368 D.W., OBJS, April, 2001, Schantz, R.E., Zinky, J.A., Bakken E-mail: [email protected] http://www.objs.com/msglog/. D.E., Proceedings of the First [4] Estimating Service Failure, Steve Ford International Symposium on Object- Object Services and Consulting Inc. Wells, D.L. and Langworthy, D.E., Oriented Real-Time Distributed OBJS, May, 1999, 225 State Rd. Computing (ISORC ’98), 20-22 West Grove, PA 19390 http://www.objs.com/Survivability/ April 1998, Kyoto, Japan, EstimatingServiceFailure.html. http://www.dist-systems.bbn.com/ Phone: 610-345-0290 [5] Advanced Logistics Project papers/1998/ISORC. E-mail:[email protected] Homepage, Defense Advanced [13] Quality of Service Adaptation Dr. Craig Thompson Research Projects Agency, for Ultra*Log Core, Zinky, J., Object Services and Consulting Inc. http://www.darpa.mil/iso2/alp/. Ultra*Log Developers Meeting, 2725 Deep Valley Trail [6] Ultra*Log Homepage, Defense April 2001, http://www.ultralog.net/ Plano, TX 75023 Advanced Research Projects Agency, products/meetings/4April/ Phone: 972-612-6998 http://www.darpa.mil/tto/programs/ 4l_QoSUltraLogDev-Apr2001- E-mail:[email protected] ultraLog.html. 5.ppt. [7] Cougaar Homepage, [14] Msg*Log Metrics, Wells, D.W., David Wells http://www.cougaar.net/. OBJS, April, 2001, Object Services and Consulting Inc. http://www.objs.com/msglog/. 6111 Baywood Ave. [8] Msg*Log Homepage, Object Baltimore, MD 21209 Services and Consulting, [15] DARPA Control of Agent http://www.objs.com/ultralog/. Based Systems (CoABS) Program: Phone: 410-318-8938 [9] Routing in Telecommunications The CoABS Grid, Global InfoTek, E-mail: [email protected] Networks with “Smart” Ant-Like Inc., March, 2000, Agents, Bonabeau, E., Henaux, F., http://coabs.globalinfotek.com/ Gu’erin, S., Snyers, D, Kuntz, P. and coabs_public/coabs_pdf/ Object Services and Theraulaz, G., Santa Fe Institute, gridvision.pdf. Consulting Inc. 1998, http://www.santafe.edu/sfi/ publications/Abstracts/98-01- 003abs.html. http://www.objs.com/

Data & Analysis Center for Software (DACS) 15 The Need and Tools to Gain Precision in Electronic Commerce Gio Wiederhold, Stanford University

1. Introduction produced by these errors. In The major problem facing individual statistics these are characterized as decision-makers is the ubiquity and The web is moving our decision- false positives, or type 2 errors. In diversity of information. Even more making processes from an most business situations, a modest than the advertising section of a information-sparse setting into in fraction of missed opportunities daily newspaper the World-Wide information-rich setting. A major (type 1 errors) are acceptable. We Web contains more alternatives than problem facing individual users is will discuss the trade-offs and can be investigated in depth. When the ubiquity and diversity of present current and future tools to leafing through advertisements the information. The World-Wide Web enhance precision in electronic selection is based on the prominence contains more alternatives than can information gathering. of the advertisement, the be investigated in depth. The value convenience of getting to the While much progress in Information system itself is changing, whereas advertised merchandise in one’s Science is triggered by progress in traditionally information had value, neighborhood, the reputation of technology, when assessing the now it is the attention of the quality, personal or created by future we must focus on the purchaser that has value. Evidence is marketing, of the vendor, and consumers. In this paper we consider the focus on free availability of web features - suitability for a specific the needs of decision-makers, resources and the difficulties in need, and price. The dominating consumer, business, and professional maintaining copyrights. factor differs based on the merchandise. Similar factors apply to on-line purchasing of merchandise and services. Lacking the convenience of leafing through the newspaper, greater dependence for selection is based on selection tools. 1.1 Getting the Right Information Getting complete information is a question of breadth. In traditional measures completeness of coverage is termed recall. To achieve a high Figure 1: Limits on Human Processing recall rapidly all possibly relevant sources have to be accessed. Since Tools are needed to search through needs were also addressed in an complete access for every the mass of potential information. earlier source report [29]. Support of information request is not feasible, Traditional information retrieval decision-making must be done information systems depend on tools have focused on returning as expeditiously, with a very low rate of having indexes. Having an index much possible relevant information, error and modest human supervision. means that an actual information in the process lowering the precision, When excessive information must be request can start from a manageable since much irrelevant material is processed, efficiency drops, and what list, with points to locations and returned as well. However, for e- is worse, confusion ensues, making pages containing the actual commerce to be effective, we cannot decision-making error prone. information. afford to wade through excess data

16 STN 4-4: Software Agents The effort to index all publicly High quality indexes can help was a inducement to obtain more available information is immense. immensely. Input for indexes can be information. Having more Comprehensive indexing is limited produced by the information information was seen as being able due to the size of the web itself, and supplier, but those are likely to be to make better decisions, and reduce the rate of change of updates to the limited. Schemes requiring risks, save resources, and reduces information on the web. Some of cooperation of the sources have been losses. these problems can be, and are being proposed [8]. Since producing an Today we have access to an excess addressed by brute force, using index is a valued-added service, it is of information. The search engines heavyweight indexing engines and best handled by independent will typically retrieve more than a smart indexing engines. For instance, companies, who can distinguish requestor can afford to read. The sites that have been determined to themselves, by comprehensiveness metrics for information systems have change frequently will be visited by versus specialization, currency, been traditionally recall and the worms that collect data from the convenience of use, and cost. Those precision. Recall is defined as the sources more often, so that the companies can also use tools that ratio of relevant records retrieved to average information is as little out- break through access barriers in all relevant records in the database. of-date as feasible [17]. Of course, order to better serve their population. Its complement, the count of relevant sites change very frequently, say records not retrieved is termed a type more than once a day, cannot be 1.2 The Need for 1 error in statistics. Precision is effectively indexed by a broad-based Precision defined similarly as the ratio to search engine. We have summarized Our information environment has relevant records to irrelevant the approaches currently being used changed in recent years. In the past, records. The irrelevant records in [30]. Say ten years ago, most decision retrieved are categorized as type 2 Getting complete information makers operated in settings where errors. In practical systems these are typically reduces the fraction of information was scarce, and there related, as shown in Figure 2. While actual relevant material in the retrieved collection. It is here where it is crucial to make improvements, since we expect that the recall volume of possibly relevant retrieved information will grow as the web and retrieval capabilities grow. Selecting a workable quantity that is of greatest benefit to a customer requires additional work. This work can be aided by the sources, through better descriptive information or by intermediate services, that provide filtering. If it is not performed, the customer has a heavy burden in processing the overload, and is likely to give up.

Figure 2: Trading of Relevance versus Precision

Continued on page 18 Data & Analysis Center for Software (DACS) 17 The Need and Tools to Gain Precision in Electronic Commerce Continued from page 17 recall can be improved by retrieving aggregations of relative word The cost of an individual type 2 more records, the precision becomes frequencies in documents for the error is borne by the decision-maker, disproportionally worse. search terms and popularity of who has to decide that an erroneous, webpages, as indicated by access irrelevant supplier was selected, There are a number of problems counts or references from peer pages perhaps a maker of toy trucks when with these metrics: measuring [Google ref]. For e-commerce, real trucks were needed. The cost of relevance and precision, the relative where the catalog entries are short an individual rejection may be small, cost of the associated errors, and the and references harder to collect these but when we deal with large scale effect of very large collections. rankings do not apply directly. Other collections, the costs can become Relevance - Well recognized is that services, as MySimon, and Epinion substantial. We will argue that more the decision on relevance of [6] try to fill that void by letting automation is needed here, since documents is fluid. When the users vote. manual rejection inhibits resources, as on the web, are automation. immense, the designation of relevance itself can become irrelevant. Some documents add so little information that an actual decision-making process will not be materially affected. A duplicate document might be rated relevant, although it provides no new information. Most experiments are evaluated by using expert panels to rate the relevance of modest document collections, since assessing all documents in the collection is a tedious task. Figure 3: Costs of type 1 versus type 2 Errors Precision - The measurement of Cost - Not considered in most Scale. Perfection in retrieval is hard precision suffers from the same assessments of retrieval performance to achieve. In selected areas we find problem, although it does not require are relative costs to an actual user of now precision ratios of 94% [19]. that all documents in the collection the types of errors encountered. For While we don’t want to belittle such be assessed, only the ones that have instance, in a purchasing situation, achievements, having 6% type 2 actually be retrieved. Search the cost of not retrieving all the errors can still lead to very many engines, in order to assist the user, possible suppliers of an item may irrelevant instances, when such typically try to rank retrieved items cause paying more than necessary. techniques are applied to large in order of relevance. Most users However, once the number of collections, for instance, a 6% error will only look at the 10 top-ranked suppliers is such that a reasonable rate on a million potential items will items. The ranking computation choice exists, the chance that other generate 60 000 errors, way too differs by search engine, and account suppliers will offer significantly many to check manually. It is hard to for much of the differences among lower prices is small. The cost of be sure that no useful items have them. Two common techniques are type 1 errors is then low, as shown in been missed if one restricts oneself to Figure 3. the 10 top-ranked items.

18 STN 4-4: Software Agents 1.3 Errors impossibility of having wide counties, etc. are routinely encoded. agreements on the meaning of terms When such terms are used in a The reasons for having errors are among organizations that are database the values in a schema manifold. There are misspellings, independent of each other. We column are constrained, providing there is intentional manipulation of denote the set of terms and their another example of a structural webpages to make them rank high, relationships, following current relationship. There are thousands of there is useful information that has usage in Artificial Intelligence, as an such lists, often maintained by not been accessed recently by search ontology. In our work we define domain specialists. Other ontologies engines, and there are suppliers that ontologies in a grounded fashion, are being created now within DTD intentionally do not display their namely: definitions for the eXtended Markup wares on the web, because they want Language (XML) [5]. to be judged by other metrics, say Ontology: a set of terms and their quality, than the dominant metric relationships 2.1 Sources of when purchasing, namely price. All Term: a reference to real- Ontologies these sources of errors warrant world and abstract investigation, but we will focus here objects Although the term ontology is just on a specific problem, namely Relationship: a named and typed set now getting widespread acceptance, semantic inconsistency. of links between all of us have encountered ontologies objects in various forms. Often terms used in The importance of errors is also paper systems have been reused in domain-dependent. A database which Reference: a label that names computer-based systems: is perfectly adequate for one objects application may have an excessive Abstract a concept which Lexicon: collection of terms error rate when used for another Object: refers to other objects used in information purpose. For instance, a payroll systems Real-world an entity instance might have too many errors in the object: with a physical Taxonomy: categorization or a employee’s address field to be useful manifestation classification of terms for mailout. It’s primary purpose is Database attributes, ranges, not affected by such errors, since Grounding the definitions so that schemas: constraints most deposits are directly transferred they can refer to actual collections, Data guide to systems with to banks, and the address is mainly as represented in databases, allows dictionaries: multiple files, owners used to determine tax deduction validation of the research we are requirements for local and state undertaking [27]. Many precursors Object grouped attributes, governments. To assure adequate of ontologies have existed for a long libraries: inherit., methods precision of results when using data time. Schemas, as used in databases, Symbol terms bound to collected for another objective some are simple, consistent, intermediate- tables: implemented content quality analysis is needed level ontologies. Foreign keys programs prior to making commitments. relating table headings in database Domain interchange terms in schemas imply structural models: XML DTDs, schemas. 2. Semantic relationships. Included in more Inconsistency comprehensive ontologies are the The ordering in this list implies an values that variables can assume; of ongoing formalization of knowledge The semantic problem faced by particular significance are codes for about the data being referenced. systems using broad-based enumerated values used in data- collections of information is the processing. Names of states, Continued on page 20 Data & Analysis Center for Software (DACS) 19 The Need and Tools to Gain Precision in Electronic Commerce Continued from page 19

Database schemas are the primary interchanging information to widely Large Ontologies means used in automation to distributed participants. Examples A major effort, sponsored by the formalize ontological information, are found in diverse applications, as National Library of Medicine but they rarely record relationship petroleum trading and the analysis of (NLM), has integrated diverse information, nor define the Shakespeare’s plays. The ontologies used in healthcare into the permissible range for data attributes. participants in those enterprises have Unified Medical Language System Such information is often obtained shared knowledge for a long time, (UMLS) [11]. In large ontologies during design, but rarely kept and and a formal and processable collected from diverse sources or even less frequently maintained. encoding is of great benefit. constructed by multiple individuals Discovering the knowledge that is There is still a need in many of these over a long time some implicit in the web itself is a domains to maintain the ontologies. inconsistencies are bound to remain. challenging task [9]. In healthcare, for instance, the terms Maintenance of such ontologies is 2.2 Large versus Small needed for reporting patient’s required when sources change [21]. Ontologies diseases to receive financial It took several years for UMLS to reimbursement change periodically, adapt to an update in one of its Of concern is the breadth of as therapies evolve and split for sources, the disease registry ontologies. While having a alternate manifestations. At a finer mentioned earlier. Still. UMLS consistent, world-wide ontology over granularity, disease descriptors used fulfills is mission in broadening all the terms we use would cause the in research areas evolve even faster, searches and increasing recall, the problem of semantic inconsistency to as we learn about distinctions in main objective of bibliographic go away, we will argue that such a genotypes that affect susceptibility to systems. goal is not achievable, and, in fact, diseases. Large ontologies have also been not even desirable. The maintenance of these domain collected with the objective to assist Small Ontologies ontologies often evolves onto in common-sense reasoning (CyC) professional associations. Such [16]. CyC provides the concept of We have seen successes with small, associations have a membership that microtheories to circumscribe focused ontologies. Here we has an interest in sharing and contexts within its ontology. CyC consider groups of individuals, that cooperating. Ontology creation and has been used to articulate relevant cooperate with some shared maintenance is a natural outgrowth information from distinct sources objective, on a regular basis. of their function in dissemination of without constraints imposed by Databases within companies or information, and merges well with microtheories [4]. That approach interest groups have been effective the role they have in publication and provides valuable matches, and means of sharing information. Since organizing meetings. An example of improves recall, but does not they are finite, it is also possible for such a close relationship in improve precision. participants to inspect their contents Computer Science is the The inconsistency of semantics and validate that the individual classification of computer literature among sources is due to their expectations and the information [1], published by the ACM and autonomy. Each source develops its resources match. Once this semantic revised approximately every 5 years. terminology in its own context, and match is achieved, effective This document provides an effective uses terms and classifications that automatic processing of the high-level view to the literature in are natural to its creators and information can take place. Many of the scientific aspects of the domain, owners. The problem with the ongoing developments in although it does not provide a articulation by matching terms from defining XML DTD’s and schemas granularity suitable for, say, trading diverse sources is not just that of follow the same paradigm, while and purchasing of software.

20 STN 4-4: Software Agents synonyms - two words for the same up specialized terms and ontologies. Since a simple object, or one word for completely abbreviations, as 3D for a three- integration of small ontologies will different objects, as miter in penny sized nail, would be lead us directly into the problems carpentry and in religion. The inefficient — language in any faced by large ontologies, we must inconsistencies are much more domain is enhanced to provide learn to combine the small complex, and include overlapping effective communication within that ontologies as needed, specifically as classes, subsets, partial supersets, domain. The homeowner cannot needed for the applications that and the like. Examples of problems afford to learn the thousands of require the combined knowledge. abound. The term vehicle is used specialized terms needed to maintain However, inconsistent use of terms differently in the transportation code, one’s house, and the carpenter makes sharing of information from in police agencies, and in the cannot afford wasting time by multiple sources incomplete and building code, although over 90% of circumscribing each nail, screw, and imprecise. As shown above, forcing the instances are the same. tool with precise attributes. every category of customers to use The problems of maintaining The net effect of these problems, matching terms is inefficient. consistency in large ontologies is when extended over all the topics we Mismatches are rife when dealing recursive. Terms do not only refer to wish to communicate about is that it with geographic information, real-world objects, but also to is impossible to achieve a globally although localities are a prime abstract groupings. The term consistent ontology. Even if such a criterion for articulation [18]. ‘vehicle’ is different for architects, goal could be achieved, it could not Most ontologies have associated when designing garage space, versus be maintained, since definitions textual definitions, but those are its use in traffic regulation, dealing within the subdomains will, and rarely sufficiently precise to allow a with right-of-way rules at must continue to evolve. It would formal understanding without intersections. At the next higher also be inefficient, since the human interpretation. Although level, talking about transportation subdomains would be restricted in these definitions will help readers will have very different coverage for their use of terms. The benefits to the knowledgeable about the domain, the relevant government department common good, that we all could they cannot guarantee precise versus a global company shipping its communicate consistently will be automatic matching in a broader goods. outweighed by the costs incurred context, because the terms used in locally and the cost of the There are also differences in the definitions also come from their requirements that we all acquire granularity with domains. A vendor own source domains. The result is consistent global knowledge. site oriented towards carpenters will that inconsistencies will occur when use very specific terms, say sinkers Composition of Small terms for independent, but relatable and brads, to denote certain types of Ontologies domains are matched. nails, that will not be familiar to the These inconsistencies are a major general population. A site oriented to If we have proven here, albeit source for errors and imprecision. home owners will just use the informally, that large global We have all experienced web general categorical term nails, and ontologies cannot be achieved, even searches that retrieved entries that may then describe the diameter, though they are desirable to solve had identically spelled keywords, length, type of head, and material. broader problems than can be solved but were not all related to the For the homeowner to share the with small ontologies, we are faced domain we are addressing - type 2 ontologies of all the professions with one conclusion. It will be errors. When we augment the involved in construction would be necessary to address larger problems impossible. For the carpenter to give we interoperating with small Continued on page 22 Data & Analysis Center for Software (DACS) 21 The Need and Tools to Gain Precision in Electronic Commerce Continued from page 21 queries with possible synonyms, For instance, checking for a same granularity, and once matched, because we sense a high rate of relationship of automobile the articulation is easy. missing information, type 1 errors, purchasing and accidents requires Understanding such articulation the fraction of junk, type 2 errors, looking for the car owners in dealer points is a service implicitly provided typically increases disproportionably. records that list the buyers. now by experts, here travel agents. In The problems due to inconsistency any application where subtasks cross We define then the articulation to be are more of a hindrance to the boundaries of domain some the semantically meaningful automation than to browsing, where experts exist that help bridge the intersection of concepts that relate one deals with one instance at a time. semantic gaps. domains with respect to an 3. Articulation application. The instances should Often the matching rules become Since we cannot hope to achieve global consistency, but still must serve applications that span multiple domains, we must settle composition. The theme, that only focused, application oriented approaches will be maintainable, directs us to limit us to the concepts needed for interoperation, for which we will reuse the term articulation.

Once we have clear domain Figure 4: Articulation of Two Domains ontologies that are to be related within an application we must recognize their intersections, where match according to our definition of complex. In listings of the California concepts belong to multiple domains. an ontology, given in the Department of Motor Vehicles For clarity, we restrict ourselves to introduction to this section. (DMV) houseboats are included. To match vehicles correctly for, say, an intersections of two domains. More An articulation point hence defines a analysis of fuel consumption, the complex cases are certainly feasible, relevant semantic match, even if the articulation rule has to exclude those but we will address them using the actual terms and their representation houseboats. The attributes that define algebraic capabilities presented in do not match. For instance, for the classes now become part of the Section 5. We will deal in this section vacation travel planning a trip input needed for the execution of the with the binary case. segment matches the term flight articulation. Such differences in from the airline domain, and term scope are common, and yet often 3.1 Semantic Rules journey from the railroad domain. surprising, because the application Terms at a lower level of abstraction, An application requiring information designer has no reason to suspect that defining instances also have to made from two domains must be able to such differences exist. A good way to to match. For instance, to take a train join them semantically, so that there check correctness of matches is to to San Francisco Airport one must will be a semantic intersection process the underlying databases. between them. Such a match may not get off at the San Bruno Caltrain be found by lexical word matching. station. Here the terms are at the

22 STN 4-4: Software Agents The concept is not to force alignment When two sources come from the afford to spend human efforts to of entire base ontologies, but only same organization, we would expect correct semantic mismatches every present to the application consistent an easy match. i.e., a consistent time. terms in the limited overlapping ontology. However, we found that To summarize, articulations that are area. Typical applications that rely even in one company the payroll needed among domains are made on intersections are in purchasing department defined the term implicitly by smart people. goods and services from another employee differently from the Converting human expertise in domain, the example above cited definition used in personnel, so that dealing with domain intersections to journeys and lights. Terms only the intersection of their two permit automation will require a used in one domain need not be databases is smaller than either formalization of the domain aligned, as sleeping compartment source. Such aberrations can easily ontologies and their semantic and in-flight movie. be demonstrated, by computing the intersections. Such research will be differences of the membership from an important component of moving 3.2 Creating Articulations the respective databases, following to the semantic web [2] There are already people in all kinds an ontological grounding as we use of business settings who perform here. In large multi-national 3.3 An Algebra for such work. Any travel agent has to corporations and in enterprises that Ontologies be able to deal with the diversity of have grown through mergers, resources. However, when differences are bound to exist. These There will be many applications that interacting the phone or directly with can be dealt with if the problems are require more than a pair of diverse webpages on the Internet, the formally recognized and articulated, ontologies. For example, logistics, problems are not widely recognized. but often they are handled in an which must deal with shipping For automation they will need to be isolated fashion, and solved over and merchandise via a variety of carriers: solved formally. over in an ad-hoc fashion. truck, rail, ship, and air, requires interoperation among many diverse Such analyses are not feasible when Keeping the rules that define an domains, as well as multiple source information sources are articulation specific to narrow companies located in different world-wide, and contexts become application contexts simplifies their countries. To resolve these issues we unclear. Here no comprehensive creation and maintenance. Even are developing an ontology algebra, matching can be expected, so that within an application area multiple which further exploits the certain operation cannot be executed rule sets can exist, for instance one capabilities of rule-based reliably on-line, although many tasks might specific to logistics in drug articulation [20]. distribution. The logical can be carried out. These difficulties organization to be responsible for the are related to the applicability of the Once we define an intersection of rules which define such a specific closed-world-assumption (CWA) ontologies through articulation, we articulation ontology for, say, [22]. should also define union and difference operations over ontologies pharmaceutical drugs would be the It requires an effort to define [24]. We apply the same semantic National Drug Distributors articulations precisely. The matching rules we used for Association (NDDA) in the USA. investment pays off as it reduces the articulation to transform the There will be a need for tools to wasted effort in taking care of the traditional set operations to manage those rules, and these tools effects of errors that are now operations that are cognizant of can serve diverse applications, both avoided. The initial effort becomes inter-domain semantics. Assuring in creation and maintenance [13]. essential to support repetitive transactions, where one cannot Continued on page 24 Data & Analysis Center for Software (DACS) 23 The Need and Tools to Gain Precision in Electronic Commerce Continued from page 23 that soundness and consistency, 4. Architecture quality, or risk versus having-up-to- mirroring what we expect from date information must be relegated traditional set operations, is a We use the term architecture to refer to the decision maker, and not challenge. to the composition of modules of automated at lower levels in a information systems. Traditional system hierarchy. Having an algebra not only achieves information systems have depended disciplined scalability to an on human experts. Their elimination Domain-type mediators can integrate unlimited set of sources, but it also through the capability of providing domains as financial information, provides a means to enumerate direct linkages on the web has led to personnel management, travel, alternate composition strategies, disintermediation [23] . logistics, technology etc. [24]. assess their performance, and, if Within these domains will be further warranted, perform optimizations We see these services being replaced specialization, as in finance to [13]. We expect that the semantic by automated engines, positioned provide information about investing union operation will mainly be between the information clients and in precious metals, bonds, blue-chip employed to combine the results of the information resources. Within the stocks, utilities, and high tech. prior intersections, in order to mediators will be the intelligent There will be meta-services as well, increase the breadth of ontological functions that encode the required helping to locate those services and coverage of an application. expertise for semantic matching and filtering [15]. Composition of reporting on their quality. Mediators The semantic difference operation synergistic functions creates a encompass both experts and software will allow the owners of a domain mediator performing substantial to perform these functions, and ontology to distinguish the terms that service. Such a service is best sustain the services as functional the owners can change as their needs envisaged as a module within the requirements and underlying change. The excluded terms, by networks that link customers and ontologies evolve. definition, participate in some resources, as sketched in Figure 5. 4.1 Middleware and articulation, and changes made to Many customers can share mediator them will affect interoperation with services through their web portals Mediation related domains, and hence make [14]. Multiple mediators will often The need for middleware to connect some application less precise, or be needed, when measures used for clients to servers has been well even disable it. Informally, selection and valuation are nor established. although it has only difference allows ontology owners to commensurate. For instance, attracted a modest amount of assess the scope of their local tradeoffs involving cost versus academic interest [7]. However, autonomy. middleware products only enable communication, and deal with issues as establishing connectivity, reliability, transmission security, and resolution of differences in representation and timing. A mediator can exploit those technologies and avoid dealing with the problems that arise from

Figure 5. Mediator Architecture 24 STN 4-4: Software Agents having an excess of standards. updates, scheduled by a service, 5. Summary However, middleware never deals often hurt a customer, even though with true semantic differences and in the long run the improvement is Information presented to customers only rarely with integration of desired. To allow customers to or applications must have a value information, leaving these tasks to a schedule their adaptation to new that is greater than the of obtaining superior layer of software. capabilities when it is suitable for and managing it. A large fraction of them, mediator owners can keep the cost is dealing with erroneous There is today a small number of prior versions available. Since and irrelevant data, since such companies building mediators with mediators are of modest size and do processing requires human insight transformation and integration not hold voluminous data internally, and knowledge. More information is capability [28]. However, the keeping an earlier copy has a modest hence not better, and less may well available technology is not yet cost. be, if relevance per unit of suitable to be shrink-wrapped and information produced is increased requires substantial adaptation to The benefits of not forcing all individual settings. Many products customers to change interfaces at the The need for assistance in obtaining focus on specific increments. When same time are significant. First of all relevant information from the world- the added value is modest, then the customers can update at a time when wide-web was recognized early in benefit gained is likely outweighed they can do it best. A second benefit the web’s existence by the cost in performance incurred is that first only a few customers, [BowmanEa:94]. This field has seen when adding a layer into an namely those that need the new rapid advances, and yet the users information system architecture. In capabilities will be served. Any remain dissatisfied with the results. those cases, making incremental errors or problems in the new version Complaints about information improvements to the sources, as can be repaired then, in cooperation overload’ abound. Web searches providing object transforms, or in with those customers, and broader retrieve an excess of references, and applications, as providing multiple and more serious problem will be getting an actually relevant result, interfaces, seems to be preferable. avoided [25]. requires much subsequent effort. However, if the services placed into Since maintenance of long-lived In this article we focused on one an intermediate layer are artifacts, including software, is such aspect, namely precision, the comprehensive, sufficient added a large fraction of the lifetime cost it elimination of excess information.. value can be produced for the is crucial to plan for maintenance, so The main method we presented is applications that access the that maintenance can be carried out constrained and precise articulation mediators and the cost of transit expeditiously and economically. among domains, to avoid the errors through the additional layer will be Being able to be responsive to that occur when searches and offset. maintenance needs increases integration of retrieved data is based consumer value and reduces both on simply lexical matches. 4.2 Incremental consumer and provider cost. Where We refer to the services that replace Maintenance maintenance today often amounts to traditional human functions in 80% of lifetime cost, a 25% To deliver valuable services, information generation as mediators, reduction in those costs can double mediators will have to be updated as and place them architecturally the funds available for systems well. Some changes are bound to between the end-users, the human improvements, while a 25% increase affect the customers, as new professionals and their local client can inhibit all development and lead interfaces or changes in the software, and the resources, often to stasis. underlying ontologies. Unwanted

Continued on page 26 Data & Analysis Center for Software (DACS) 25 The Need and Tools to Gain Precision in Electronic Commerce Continued from page 25 legacy databases and inconsistently Dr. Wieferhold has authored nine [6] R.V. Guha: Check Before you structured web sources. Such novel books and over 300 technical reports Buy; Epinions.com, 2000. software will require more powerful and papers. He received his Ph.D. in [7] The Garther Group: Middleware; hardware, but we are confident that Medical Information Science from Gartner group report, 1995. hardware-oriented research and the University of California in 1976. [8] L. Gravano , H. Garcia-Molina, development is progressing, and will and A. Tomasic: “Precision and be able to supply the needed Contact Information Recall of GlOSS Estimators for infrastructure. The major reason for Gio Wiederhold Database Discovery’’; Parallel and slow acceptance of innovations is not Computer Science Department Distributed Information Systems, the technology itself, but the Gates Computer Science Bldg. 4A 1994. massiveness of the organizational Room 433 [9] J. Heflin, J. Hendler, and S. and human infrastructure. Stanford University Luke: “Reading Between the Lines: Stanford CA, 94305 Biography Using SHOE to Discover Implicit Phone: 650-725-8363 Knowledge from the Web”; in Dr. Wiederhold is currently a E-mail: [email protected] AAAI-98 Workshop on AI and Professor Emeritus at Stanford Information Integration, 1998. University in both the Departments References of Computer Science, and Medicine [10] Darrel Huff: How to Lie with and Electrical Engineering (by [1] Neal Coulter, et al: ACM Statistics; Norton, 1954. courtesy) where his research Computing Classification System, [11] Betsy Humphreys and Don includes; design and operation of 1999, http://www.acm.org/class/ Lindberg: “The UMLS project : large-scale software systems, [2] Tim Berners-Lee, Jim Hendler, Making the conceptual connection database design, knowledge bases, and Ora Lassila: “The Semantic between users and the information distributed systems, applications in Web”; Scientific American, they need’’; Bulletin of the Medical medicine, planning, and business, May 2001. Library Association, 1993, see also and privacy/security. [3] C. Mic Bowman, Peter B. http://www.lexical.com/. Dr. Wiederhold os also the CEO of Danzig, Darren R. Hardy, Udi [12] Jan Jannink, Prasenjit Mitra, Symmetric Security Technologies, a Manber and Michael F. Schwartz: Erich Neuhold, Srinivasan Pichai, software development and consulting “The HARVEST Information Rudi Studer, Gio Wiederhold: “An firm specializing in Internet security. Discovery and Access System’’; Algebra for Semantic Interoperation He is greatly concerned about secure Proceedings of the Second of Semistructured Data”; in 1999 information management. He has International World Wide Web IEEE Knowledge and Data been the principal investigator for Conference, Chicago, Illinois, Engineering Exchange Workshop NSF and DARPA-funded research October 1994, pp 763—771. (KDEX’99), Chicago, Nov. 1999. projects to develop security [4] C. Collet, M. Huhns, and W-M. [13] Jan Jannink: Towards Semantic mediation in the healthcare domain Shen: “Resource Integration Using Interoperation of Semistructured (Trusted Interoperation of a Large Knowledge Base in Data; Stanford University, Healthcare Information (TIHI)), for CARNOT’’; IEEE Computer, Computer Science Department PhD medical images (Trusted Image Vol.24 No.12, Dec.1991. thesis, March 2001, Dissemination (TID)), and CAD [5] Dan Connolly (ed.): XML: http://www-db.stanford.edu/~jan/. (Secure Access Wrapper (SAW)). Principles, Tools, and Techniques, O’Reilly, 1997.

26 STN 4-4: Software Agents [14] YuHui Jin, Stefan Decker, and [21] Diane E. Oliver and Yuval [27] Gio Wiederhold and Michael Gio Wiederhold: OntoWebber: Shahar: “Change Management of Genesereth: “The onceptual Basis Model-Driven Ontology-Based Web Shared and Local Health-Care for Mediation Services”; IEEE Site Management; in preparation, Terminologies”; IMIA Working Expert, Intelligent Systems and Stanford University CSD, Ontoweb Group 6 Meeting on Health Concept their Applications, Vol.12 No.5, project, May 2001. Representation and Natural Sep-Oct.1997. [15] Thomas Langer: “MeBro - A Language Processing, [28] Gio Wiederhold: “Weaving Framework for Metadata-Based Phoenix, AZ, 1999. Data into Information”; Database Information Mediation’’; First [22] R. Reiter,R.: “On Closed Programming and Design, Freeman International Workshop on Practical World Data Bases”; in Logic and pubs, Sept. 1998. Information Mediation and Data Bases, Gallaire and Minker [29] Gio Wiederhold: Trends in Brokering, and the Commerce of (eds) Plenum Press NY., 1978, Information Technology; report to Information on the Internet, Tokyo [23] Gio Wiederhold, Gio: JETRO.MITI, Japan, September 1998, http:// “Mediators in the Architecture of http://www-db.stanford.edu/pub/gio/ context.mit.edu/imediat98/paper2/. Future Information Systems”; IEEE 1999/miti.htm. [16] D. Lenat and R.V. Guha: Computer, March 1992, pages 38- [30] Wiederhold, Gio: “Precision in Building Large Knowledge-Based 49; reprinted in Huhns and Singh: Processing Data from Heterogeneous Systems, Addison-Wesley (Reading Readings in Agents; Morgan Resources”; in B.Lings and MA), 372 pages. Kaufmann, October, 1997, K.Jeffreys (eds.): Advances in [17] Clifford Lynch: “Searching the pp.185-196. Databases, Procedings of 17th Internet”; The Internet: Fulfilling [24] Wiederhold, Gio: “An Algebra British National Conference on the Promise; Scientific American; for Ontology Composition”; Databases, Exeter, UK, July 2000, March 1997. Proceedings of 1994 Monterey pages 1-18. Workshop on Formal Methods, Sept [18] David Mark et al.: Geographic Acknowledgments Information Science: Critical Issues 1994, U.S. Naval Postgraduate in an Emerging Cross-Disciplinary School, Monterey CA, pages 56-61. Background material for this article Research Domain; NCGIA, Feb. [25] Gio Wiederhold: “Modeling came from a study prepared for 1999, http://www.geog.buffalo.edu/ and System Maintenance”; in JETRO / MITI [29] , and in turn ncgia/workshop report.html. Papazoglou: OOER’95: Object- was based on many published [19] Tom Mitchell: “Machine Oriented and Entity Relationship resources as well as on discussions Learning and Data Mining”; Modelling; Springer Lecture Notes with wonderful people that I have Communications of the ACM, in Computer Science, Vol. 1021, encountered in my work and studies. Vol. 42, No. 11, November 1999. 1995, pp. 1-20. Work on mediation in general was [26] Gio Wiederhold: “Customer supported by DARPA for many [20] Prasenjit Mitra, Gio years. The research focusing on Wiederhold, and Martin Kerstens Models for Effective Presentation of Information”; Position Paper, articulation is being supported by “A Graph-Oriented Model for AFOSR under the New World Articulation of Ontology Flanagan, Huang, Jones, Kerf (eds): Human-Centered Systems: Visions initiative and by DARPA’s Interdependencies”; Proceedings of DAML project. Extending DataBase Technologies, Information, Interactivity, and (EDBT 2000), Konstanz, Intelligence, National Science Germany, Springer Verlag LNCS, Foundation, July 1997, pp.218-221. March 2000;

Data & Analysis Center for Software (DACS) 27 STN Vol. 4, No. 4 Article Reproduction STN Editorial Board Images and information presented in Lon R. Dean, STN Editor In This Issue these articles may be reproduced as ITT Industries, DACS long as the following message is Agent Based Computing Paul Engelhart, DACS COTR noted: Air Force Research Lab (IFTD) for Autonomous “This article was originally printed Intelligent Software ....2 Elaine Fedchak in the DOD Software Tech News, ITT Industries vol. 4, no. 4. Requests for copies of Dr. Jack Ferguson Msg*Log: E-mail-based the referenced newsletter may be ODUSD (S&T) Agent Messaging to submitted to the following address: Morton A. Hirschberg, STN Improve Robustness in Data & Analysis Center for Software Editorial Board Chairman a Distributed Logistics (DACS) Attn: Lon R. Dean, Editor US Army Research Lab (retired) Planner ...... 5 PO Box 1400 Philip King Rome, NY 13442-1400 ITT Industries, DACS The Need and Tools Toll free: 800-214-7921; Thomas McGibbon, to Gain Precision Fax: 315-334-4964 [email protected] DACS Director in Electronic ITT Industries, DACS An archive of past newsletters is Commerce...... 16 Dave Nicholls, available at www.dacs.dtic.mil/awareness/ DACS Deputy Director newsletters/.” ITT Industries, DACS Marshall Potter In addition to the copyright message, we ask that you send us three copies of Federal Aviation Administration any document that references any Nancy Sunderhaft article appearing in the DOD Software ITT Industries, DACS Tech News.

Data & Analysis Center for Software PRSRT STD U.S. Postage P.O. Box 1400 P A I D Permit #566 Rome, NY 13442-1400 UTICA, NY Return Service Requested