Command & Control Concepts for Maritime Area Air Defence (C3

'HIHQFH5HVHDUFKDQG 5HFKHUFKHHWGpYHORSSHPHQW 'HYHORSPHQW&DQDGD SRXUODGpIHQVH&DQDGD

Command & Control Concepts for Maritime Area Air Defence (C3-MAAD) Literature Survey

Tamara Keating, Tamara McLaughlin NRC Canada Institute for Scientific and Technical Information

The scientific or technical validity of this Contract Report is entirely the responsibility of the Contractor and the contents do not necessarily have the approval or endorsement of the Department of National Defence of Canada.

Defence Research and Development Canada – Valcartier Contract Report DRDC Valcartier CR 2011-219 May 2011

Command & Control Concepts for Maritime Area Air Defence (C3-MAAD) Literature Survey

Tamara Keating, Tamara McLaughlin NRC Canada Institute for Scientific and Technical Information

Prepared By: Tamara Keating and Tamara McLaughlin NRC Canada Institute for Scientific and Technical Information 1200 Montreal Rd., M-55 Ottawa, ON K1A 0R6 Contractor's Document Number: STI 7193, DRDC SDA 07-001-010 Contract Project Manager: T. Keating, (613) 949-8100 PWGSC Contract Number: SDA 07-001-010 CSA: Hengameh Irandoust, Defence Scientist, 418-844-4000 x 4193

Defence Research and Development Canada – Valcartier Contract Report DRDC Valcartier CR 2011-219 May 2011 IMPORTANT INFORMATIVE STATEMENTS

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2011 © Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2011 Abstract ……..

The following project was completed in two phases. The first phase sought to identify collaborative decision making tools currently on the market or being developed specifically for naval tactical C2 purposes. This was accomplished through a combination of database and Internet searches.

The second phase of this project was centered on identifying major organizations (academic, government and companies) currently conducting research in the areas of human-computer interaction and collaborative decision support tools, with a focus on Canadian organizations and laboratories. Information was sought from scientific and technical databases and some highly relevant conference proceedings. Bibliographic information was compiled into two master databases and then names of organizations were normalized and lists of the most prolific institutions in terms of numbers of publications were compiled.

DRDC Valcartier CR 2011-219 i Résumé ….....

/HSURMHWVXLYDQWDpWpPHQpHQGHX[SKDVHV'DQVOHFDGUHGHODSUHPLqUHO¶REMHFWLIFRQVLVWDLWj WURXYHUOHVRXWLOVFROODERUDWLIVGHSULVHGHGpFLVLRQFRQoXVVSpFLDOHPHQWSRXUOH&WDFWLTXH QDYDOTXLVRQWDFWXHOOHPHQWGLVSRQLEOHVVXUOHPDUFKpRXHQFRXUVGHGpYHORSSHPHQW'DQVFH EXWGHVUHFKHUFKHVRQWpWpHIIHFWXpHVGDQVGHVEDVHVGHGRQQpHVHWGDQV,QWHUQHW

6HXOHPHQWTXHOTXHVRXWLOVFROODERUDWLIVGHSULVHGHGpFLVLRQSRXUOHVRSpUDWLRQVWDFWLTXHVQDYDOHV RQWpWpWURXYpVVRLWOD&RRSHUDWLYH(QJDJHPHQW&DSDELOLW\ FDSDFLWpG¶HQJDJHPHQWHQ FRRSpUDWLRQRX&(& OD0XOWLSODWIRUP(QJDJHPHQW&DSDELOLW\ FDSDFLWpG¶HQJDJHPHQW PXOWLSODWHIRUPHRX03(& HWOHVV\VWqPHVGHJHVWLRQGHFRPEDWGH7KDOHV6HXOVGHV UHQVHLJQHPHQWVWHFKQLTXHVOLPLWpVVXUOHVV\VWqPHVGH7KDOHVHWGH5D\WKHRQpWDLHQWGLVSRQLEOHV PDLVOHVOLHQVPHQDQWjODGRFXPHQWDWLRQVXUOHXUSURGXLWRQWpWpIRXUQLV,OSRXUUDLWrWUH LPSRUWDQWGHVXUYHLOOHUWRXWQRXYHDXGpYHORSSHPHQWOLpjG¶DXWUHVSURMHWVHWSURJUDPPHVQRQ VSpFLDOHPHQWFRQoXVSRXUOD0DULQHVRLWOHSURJUDPPHGHFDSDFLWpHQUpVHDXGHO¶27$1 11(& OD1HWZRUN&HQWULF&ROODERUDWLYH7DUJHWLQJ FDSDFLWpGHFLEODJHFROODERUDWLI UpVHDXFHQWULTXHRX1&&7 GHODIRUFHDpULHQQHDPpULFDLQHHWO¶LQLWLDWLYH-RLQW,QWHJUDWHG$LUDQG 0LVVLOH'HIHQVH GpIHQVHDQWLDpULHQQHHWDQWLPLVVLOHLQWpJUpHFRQMRLQWHRX-,$0' GHVeWDWV 8QLV'HVDFWHXUVPDMHXUVGDQVFHGRPDLQHVRQW5D\WKHRQ7KDOHVHW,VUDHO$HURVSDFH,QGXVWULHV ,$, HQWUHDXWUHV%RQQRPEUHGHVFRQWUDWVGHGpYHORSSHPHQWGHVGHUQLqUHVDQQpHVRQWpWp FRQILpVjGHVFRQVRUWLXPVVRXYHQWGLULJpVSDU5D\WKHRQ7KDOHVRX/RFNKHHG0DUWLQ

/DGHX[LqPHSKDVHGHFHSURMHWYLVDLWHVVHQWLHOOHPHQWjLGHQWLILHUOHVJUDQGHVRUJDQLVDWLRQV XQLYHUVLWDLUHVJRXYHUQHPHQWDOHVHWSULYpHV PHQDQWDFWXHOOHPHQWGHVUHFKHUFKHVGDQVOHV GRPDLQHVGHO¶LQWHUDFWLRQSHUVRQQHPDFKLQHHWGHVRXWLOVFROODERUDWLIVG¶DLGHjODGpFLVLRQ SDUWLFXOLqUHPHQWOHVRUJDQLVDWLRQVHWOHVODERUDWRLUHVFDQDGLHQV/¶LQIRUPDWLRQDpWpWLUpHGH EDVHVGHGRQQpHVVFLHQWLILTXHVHWWHFKQLTXHVHWGHVWUDYDX[GHFHUWDLQHVFRQIpUHQFHVKDXWHPHQW SHUWLQHQWHV/HVUHQVHLJQHPHQWVELEOLRJUDSKLTXHVRQWpWpUHJURXSpVGDQVGHX[EDVHVGHGRQQpHV SULQFLSDOHVSXLVOHVQRPVGHVRUJDQLVDWLRQVRQWpWpVWDQGDUGLVpVHWGHVOLVWHVGHVRUJDQLVDWLRQV OHVSOXVSUROLILTXHVTXDQWDXQRPEUHGHSXEOLFDWLRQVRQWpWpFRPSLOpHV

ii DRDC Valcartier CR 2011-219 STI Assessment

Command & Control Concepts for Maritime Area Air Defence (C3-MAAD) Product / Title Literature Survey Number STI 7193, DRDC SDA 07-001-010 Date May 11, 2011 Prepared for Hengameh Irandoust Prepared by Tamara Keating, Tamara McLaughlin Contact [email protected], (613) 949-8100

NRC-CISTI employees make every effort to obtain information from reliable sources. However, we assume no responsibility or liability for any decisions based upon the information presented.

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Table of Contents 1 Summary ...... 2

2 Background ...... 3 2.1 Context ...... 3 2.2 Key Issues ...... 4 2.3 Key Questions ...... 4

3 Findings...... 5 3.1 Naval Tactical Decision Support Tools ...... 5 3.2 Other Tactical Decision Support Tools ...... 7 3.3 Human-Computer Interaction for Decision Making ...... 11 3.3.1 Major Players...... 11 3.3.2 Top Authors ...... 14 3.4 Decision Support Tools ...... 17 3.4.1 Major Players...... 17 3.4.2 Top Authors ...... 20 3.5 Conclusions ...... 24

4 References ...... 26

5 Appendices ...... 27 5.1 Methodology ...... 27 5.1.1 Searches ...... 27 5.1.2 Analysis ...... 28 5.1.3 Sources Consulted ...... 29 5.2 Major Players data ...... 31 5.2.1 HCI - Major Players ...... 31 5.2.2 Decision Support Tools – Major Players ...... 35 5.2.3 Canadian Laboratories ...... 43

Page 1 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

1 SUMMARY

Major players in this field include Raytheon, Thales, and Israel Aerospace Industries (IAI). Many of the development contracts in recent years have been awarded to consortia, often led by Raytheon, Thales or Lockheed Martin.

Human Computer Interaction (HCI) Few Canadian players were identified in a narrow search on HCI for collaborative decision making, but a broader search showed that the following Canadian institutions are the top five actively engaged in HCI research: x University of British Columbia, Vancouver, BC x Simon Fraser University, Burnaby, BC x University of Toronto, ON x University of Calgary, AB x Concordia University, Montreal, QC

Internationally, the top five players, based on numbers of publications are: x Carnegie Mellon University, Pittsburgh, PA, USA x Purdue University, West Lafayette, IN, United States x Massachusetts Institute of Technology (MIT), Cambridge, MA, USA x Pennsylvania State University, University Park, PA, USA x Delft University of Technology, Netherlands

Page 2 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Decision Support Tools This search was also very narrow in scope, looking only for decision support tools for groups of operators. While the results were significant, there were few Canadian players in the final dataset and so a broader supplemental search to retrieve more Canadian author affiliations was conducted.

From this set, the top five Canadian players in collaborative decision making tools are: x University of Waterloo, ON x University of British Columbia, Vancouver, BC x University of Toronto, ON x Université Laval, Québec, QC x University of Calgary, AB

Internationally, the top five players, based on numbers of publications are: x Naval Postgraduate School, Monterey, CA, USA x BP Global, UK x Nanyang Technological University, Singapore x Massachusetts Institute of Technology (MIT), Cambridge, MA, USA x US Air Force Research Laboratory (AFRL), Wright-Patterson AFB, OH, USA

2 BACKGROUND 2.1 Context The Canadian Forces are considering littoral regions as an operational maneuver space from which a Task Force can influence situations, decisions and events, as part of a joint (national) and/or combined (coalition) mission. A Task Force (joint or combined) is a group of platforms/units formed to accomplish common mission objectives. The conduct of operations as a Task Force, as opposed to a single platform, introduces additional challenges to the Command and Control (C2) processes. Task Force operations are network-centric, as opposed to single platform operations, which are platform-centric (only concerned with self-defense). The Task Force is embedded within a network that links sensors, shooters, and C2 nodes. Achieving C2 tasks efficiently, in a network-centric context, introduces new requirements with regard to interoperability, communication, coordination, and information sharing among the participating units and the decision-makers.

The overwhelming environment, the spectrum of potential threats, and the diversity of the adversarial tactics and manoeuvres compounded with the inherent complexity associated with the joint/combined force render the effective execution of naval C2 functions a complex task.

Objectives of the project

The aim of the C3-MAAD technology development project (TDP) is to develop and demonstrate technologies and concepts to support the future Canadian Naval Task Group command teams in the conduct of Area Air Defence (AAD) and Force Anti-Ship Missile Defence (FASMD). The project will demonstrate a prototype capability that will enable: (i) collaborative exploitation of information and situation analysis;

Page 3 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

(ii) collaboration in threat recognition and evaluation; and (iii) coordination and optimization of force-wide response planning and execution.

The project has several objectives: (i) Develop knowledge to guide and support the development of AAD C2 requirements specifications for the Canadian Surface Combatant project (ii) Develop an advanced naval force C2 M&S capability that will be compatible with and of interest to the Navy (CFMWC and DMRS) and help evaluate industry responses to CSC RFP and support tactical developments for future systems and threats by CFMWC. The distributed architecture of the M&S facility will use open standards and offer the ability to connect to the ship and/or run simulations based on live/replayed data. (iii) Develop AAD C2 automation algorithms and solutions that provide FASMD coordination measures and procedures in order to address the battle space geometry of the dispersed operating environment. The focus will be on force R&I, threat evaluation, engageability assessment, and combat power management processes. (iv) Design command decision aids to provide the force command team with cognitive support for decision making during AAD operations (v) Develop comprehensive evaluation methods and metrics that permit the validation and assessment of new technologies and concepts of operations (vi) Demonstrate and validate the performance and potential operational effectiveness of key automation algorithms, decision aids and architecture

The work requested under this mandate covers two main topics: 1. The application domain: Area Air Defence and Force Anti-Ship Missile Defence 2. Decision support concepts and tools.

2.2 Key Issues DRDC researchers would like to complement their current knowledge of existing systems in the Naval C2 tactical domain and identify experts and potential research partners or industrial collaborators, with a preference for Canadian organizations and individuals.

2.3 Key Questions

1. What collaborative decision making tools are currently on the market or are being developed specifically for naval tactical C2 purposes? Who are the major players? 2. Who is working on what in human-computer interaction for decision making, including companies, academic institutions, government organizations and individual researchers? Who are the key Canadian players? 3. Who is working on what for tools being developed that enable a group of operators to engage in collaborative decision making and planning, including companies, academic institutions, government organizations and individual researchers? Who are the key Canadian players?

Page 4 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

3 FINDINGS 3.1 Naval Tactical Decision Support Tools Very few systems specifically designed for Navy applications were identified by our search. The U.S. Cooperative Engagement Capability (CEC) system and the European Multi-Platform Engagement Capability (MPEC) system were already known to be systems that come closest to incorporating all processes of picture compilation, threat evaluation, engageability assessment and combat power management. In a 2006 NATO Parliamentary Committee report, only the CEC and MPEC are mentioned as important ISTAR (Intelligence, Surveillance, Target Acquisition and Reconnaissance) programmes related to targeting and shooting, along with the US Air Force Advanced Tactical Targeting Technology (AT3) system (Nolin 2006).

The following table provides a summary of the naval tactical systems we identified – these classifications are based on a review of product information available on the company’s websites and not on detailed technical specifications. Further details and discussion on these systems follow below.

Table 1. Naval Tactical Systems Picture Threat Engageability Combat Power System Name Compilation Evaluation Assessment Management DCN (France) - Multi- Platform Engagement X X X X Capability (MPEC) Elta Maritime Centric Operation Network EL/I- X X 4001NC EMCO–NET Raytheon – Cooperative Engagement Capability X X X X (CEC) Thales Naval Electronic X X Warfare Systems Thales Naval Combat Management System - X X TACTICOS

Cooperative Engagement Capability (CEC) The concept of a Cooperative Engagement Capability (CEC) was first introduced by Johns Hopkins University’s Applied Physics Laboratory (APL) in the mid 1980’s (Walsh, 2005) but the development was handed over to Raytheon in the 1990’s and has been improved over the last 15 years (Acevedo 2006). Raytheon faced some competition from Lockheed Martin and a small company called Solipsys for the development of CEC Block II in 2002, however Raytheon responded by buying out Solipsys and partnering with Lockheed for the competition for the Block II contract (O'Rourke 2005). Raytheon has so far installed 53 CEC systems on US Navy Ships and in April 2010 they were awarded a $25.5 million production contract and a $13.7 million design agent and engineering services contract from the U.S. Navy (Raytheon 2010). US Congressional budget documents (http://www.dtic.mil/descriptivesum/Y2012/Navy/0603658N_4_PB_2012.pdf) show an ongoing interest in CEC, as they indicate between $44 million and $80 million dollar annual budgets from 2010 to 2016, for a total of

Page 5 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

$419.749 million USD on the CEC for the Navy over nine years (U.S. Navy 2011). This budgt document is also useful for its descriptions of the system and the list of contractors and test facilities.

CEC is described as “a sensor netting system that allows many ships to pool their radar and sensor information together … more consistent than any one ship could generate on its own. The data is then shared among all ships and participating systems in the air and on the ground, using secure frequencies” (Defense Industry Daily, http://www.defenseindustrydaily.com/cec-cooperative-engagement-for-fleet-defense-updated-03120). Acedevo further describes the system as one where raw radar data of multiple ships and/or aircraft is combined into a single network, creating a composite target track with the added advantage that a ship can join or leave the CEC network without compromising link integrity or bringing down the network (Acevedo 2006).

It follows that the concept and the technologies of CEC could extend to all joint battlespaces, including air forces and ground forces as well as the Navy, but O’Neil pointed out in 2007 that this is “neither technically feasible nor affordable at present” (O'Neil 2007). Nevertheless, the Department of Defense has followed through with efforts to develop a joint system, originally under the Single Integrated Air Picture (SIAP) Program Executive Office, which has been replaced by the Joint Integrated Air and Missile Defense (JIAMD) initiative. Efforts under this project include Multi-Service Systems Engineering (MSSE), Joint Track Manager Capability (JTMC) demonstrations and Joint Operational Requirements definition (U.S. Air Force 2010) – this programme was awarded an $18.9 million budget for 2011.

Multi-Platform Engagement Capability (MPEC) – DCN and Thales (France) MPEC (also know as Tenue de Situation Multi Plates-Formes Capacite d’Engagement Multi Plates-Formes or TSMPF/CEMP) is the equivalent to CEC for France. The French DGA (Defense Procurement Agency) awarded a €21 million (25 million USD) developmental contract to DCN in 2004 (http://www.deagel.com/Ship-Protection- Systems/CEMP_a001406001.aspx ). Detailed information on this system is much more difficult to find but a 2006 article states that the programme is “intended to demonstrate the technologies required for co-operative situational awareness in which participating platforms share tactical situation data and optimize the use of their respective sensors. This enables force-wide threat evaluation and resource allocation (using the weapons and countermeasures of all participating platforms).” Testing was scheduled to take place in 2006 and at that time it was predicted that the capability could be ready to enter service around 2015 (Scott 2006).

Israel Aerospace Industries (IAL) ELTA Systems - EL/I-4001NC EMCO–NET http://www.iai.co.il/34467-36672-en/Groups_ELTA_SystemsApp_GroundBased.aspx?btl=1 EMCO-NET is a C4ISR network that enables communication among airborne, surface, sub-surface and onshore systems and forces. According to their product literature, it “utilizes advanced multi-platform multi-sensor data fusion and multi-dimensional situation awareness processes to build a common, unique, accurate and real-time Maritime Domain Awareness Picture (MDAP).”This system appears to meet the criteria for classification, identification and threat assessment.

Thales - Combat Management Systems http://www.thalesgroup.com/Markets/Defence/What_we_do/Naval_forces/Above_water_warfare/Combat_m anagement_system/ Thales’ TACTICOS system (http://www.thalesgroup.com/tacticos/?pid=1203) incorporates features for picture compilation, threat evaluation, manual and automatic sensor and weapon assignment, and kill assessment. According to their product literature, it achieves improved situation awareness using multi-sensor data fusion, automatic recognition and identification capabilities and all tactical data-links. Information from the Recognised

Page 6 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Maritime Picture can be fused with the local area picture. Decision support and coordination is enabled at the force level and for own ship threat assessment. TACTICOS is operational in more than 15 Navies world-wide, on more than 150 naval vessels.

Thales - Naval Electronic Warfare Systems http://www.thalesgroup.com/Markets/Defence/What_we_do/Naval_forces/Above_water_warfare/Electronic_ Warfare_Systems/ This system is primarily a solution for the detection and identification of threats in a littoral environment followed by jamming and decoying as countermeasures.

3.2 Other Tactical Decision Support Tools Since so few naval systems were identified by our search, other tactical systems were considered. Even among these however, none of them seem to incorporate all four features. A summary is provided in Table 2 and further details, with Internet links, are provided below.

Table 2. Other Tactical Decision Support Tools System Name Picture Threat Engageability Combat Power Compilation Evaluation Assessment Management Bulle Opérationnelle X ? Aéroterrestre (BOA) NATO Network Enabled Capability (NNEC) - in X X X X proposal stage only ThalesRaytheon - Battle X Control System (BCS) ThalesRaytheon - Hizam X ? Al Taawun (HAT II) ThalesRaytheon - SCCOA Air Operations X X Command and Control System US Air Force - Advanced Tactical Targeting ? Technology (AT3) US Air Force - Network- Centric Collaborative X X Targeting (NCCT)

Bulle Opérationnelle Aéroterrestre (BOA) (France) http://www.thalesgroup.com/Case_Studies/LandJoint_CaseStudy_BOA/ This is France’s central network-centric programme. A €129 million contract has been awarded to a Thales-led consortium for TACTIC3 network enabled architecture for close combat in the air-land theatre. The most interesting objective of this future system is to “realize the LTO (Laboratoire Technico-Opérationnel or Battle-

Page 7 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

lab) containing collaborative tools dedicated to the air-ground combat”. LTO will be used to analyze land forces missions, to capitalize on feedback and to support design and modelling.

NATO Network Enabled Capability (NNEC) NATO Network-Enabled Capability (NNEC) is being developed by NATO agencies both in Norfolk, VA (ACT Information Superiority & NATO Network-Enabled Capabilities Integrated Capability Team - IS&NNEC) and Brussels (NATO Command Control and Communications Agency - NC3A). The ACT team is preparing a strategic framework and a road map that will modernize joint Alliance capabilities and enable NATO to create a truly networked force, while NC3A is striving to create technical standards and templates for new architectures. http://www.nato-pa.int/default.asp?SHORTCUT=1004

The programme is still under development, but according to a 2010 FAQ document, a significant milestone (#3) aims to complete the development of decision support tools (Domingo and Angel Rico 2010). Exact dates for these milestones have not been set. Focus areas of the system include picture compilation, threat evaluation, engageability assessment and combat power management.

Updates on the programme can be obtained by subscribing to the NNEC Information Portal: https://transnet.act.nato.int/WISE/Informatio/index_html

Network-Centric Collaborative Targeting (NCCT) Program (US Air Force) Descriptions of NCCT sound remarkably similar to CEC, but for an aerial environment. A 2008 article in C4ISR Journal describes it as follows:

NCCT directs and combines the “take” from a variety of sensors on separate airborne platforms (“the constellation”) that are collecting in a specific area. The different languages used by each platform to relay sensor data are converted into a common Internet Protocol (IP) message set, so that they can be communicated within the constellation to all the network controllers. Using common algorithms and building a common database, data from one platform is sorted and cued to others, so that they can focus on the same, time-sensitive target. In this fashion, the chances of detecting, identifying, fixing, tracking and eliminating fleeting emitters such as mobile Surface-to-Air (SAM) missile batteries or terrorist convoys increase exponentially (Pocock 2008).

L-3 ComCept is the primary contractor for NCCT. The following description is copied directly from the L-3 NCCT website http://www.comceptinc.com/L3comcept/NCCT.htm:

NCCT involves automated cross-cueing between platforms to find, fix, track, engage, and assess short-up time emitters and other time-sensitive targets. It allows correlation to quickly fuse sensor data from individual command, control, intelligence, surveillance, and reconnaissance platforms to build target folders in a common, shared database. Single collaborative NCCT tracks report to targeting decision- makers in minutes with greater accuracy than single platform operations. These networks are scalable and can be tailored to geography, warfare domains, or other criteria. Network participants can receive a correlated picture with pedigree data on existing displays and workstations. The goal is to provide a single target/threat picture for all participants that are interoperable via direct machine-to-machine networking and/or service-oriented architecture. NCCT provides collaborative multi-intelligence identification and geo-location on high-interest events to all participants and command and control elements in near real-time.

Page 8 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Advance Tactical Targeting Technology (AT3) (US Air Force) Information on this programme more recent than 2006 could not be found, but the programme is included here because it was mentioned in the same paragraph as CEC and MPEC in a NATO Assembly Brief in 2006 (Nolin 2006), even though the system does not appear to employ the types of decision making tools of interest to this project. According to Jane’s, AT3 is a US Air Force Materiel Command Laboratory and US Defense Advanced Research Projects Agency (DARPA) project “intended to produce the US Air Force's next-generation lethal Suppression of Enemy Air Defenses (SEAD) system. Its aim is to produce an RF targeting system capable of reducing the missile battery targeting time from minutes to seconds. Under this concept, multiple aircraft fitted with ESM receivers will be networked to provide real-time targeting of hostile emitters within a CEP (circular error probable) of 15 to 50 m, thus enabling use of standoff GPS-guided weapons, such as the Joint StandOff Weapon” (Advanced Tactical Targeting Technology (AT3) (US Air Force Materiel Command, Wright Laboratory), Military CNs, FMs, data and threat management 2005). Jane’s also mentions that Boeing, teamed with Litton Advanced Systems Division, Raytheon Electronic Systems and Lockheed Martin Federal Systems responded to the DARPA solicitation, but does not mention who was awarded the R&D contract. According to a 2005 budget document, the technology demonstration project ended in 2005 and the technology was transitioned to the Air Force and Navy in that same year. A technical overview can be obtained from this 1999 DARPA presentation: http://archive.darpa.mil/darpatech99/Presentations/spopdf/spoat3final.pdf (Kaspar 1999).

ThalesRaytheonSystems This joint venture has several ongoing programmes related to C2, though the nature of the decision support tools is not clear. Most of them appear to use at the very least picture compilation technologies and threat assessment. The following descriptions are copied directly from the cited websites, unless otherwise mentioned.

The Battle Control System (BCS) http://www.thalesraytheon.com/programs/battle-control-system-bcs.html is the primary air defense/battle management system for North American Air Defense (NORAD) and the U.S. Pacific Command (PACOM). The interoperable, open architecture air defense and command and control platform supports the U.S. and Canadian Homeland Defense and drug interdiction missions...... BCS processes, integrates, displays and distributes data from multiple sensors, data links and other C2 agencies to maintain situational awareness, decision support and combat identification for the United States and Canada.

ThalesRaytheonSystems is providing the French Air Force with key components of the SCCOA programme (http://www.thalesraytheon.com/programs/sccoa-air-operations-command-and-control-system.html). This programme provides highly automated global management capability for air operations, both within mainland France and in overseas operational theatres, based on a unified air operations command centre with high speed data links and a high degree of interoperability with French and foreign armed forces.

The Gulf Cooperation Council (GCC) contracted with ThalesRaytheonSystems in 2001 to build Hizam Al Taawun (HAT II) a system that provides automated interfaces between Member States for the coordination of multilevel and multinational air defense (http://www.thalesraytheon.com/programs/gcc-hat-ii.html). The system has "real time" requirements to track hundreds of aircraft simultaneously as well as complex tools, maps, and databases in Arabic and English to facilitate military cooperation. HAT is linked with the national air defense systems of each participating nation and exchanges information via high speed encrypted data links (http://www.saudia- online.com/press/press3.shtml).

Page 9 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Other Battle Management Systems Here follows a quick list of other systems for consideration. As with the above, none of them seem to meet all four criteria. These systems tend to meet the picture compilation criteria and often include elements of threat evaluation and engageability assessment as well.

x Australia - LAND 75 system http://www.defenseindustrydaily.com/Australia-Turns-to-Elbit-for-its- Battle-Management-System-06247/ x Canadian Forces’- Land Command Support System (LCSS) http://www.defenseindustrydaily.com/Canada-Signs-Contracts-to-Support-its-LCSS-Battlefield- Command-System-05331/ x Germany - FuInfoSys Heer (FuInfoSys H) http://www.defenseindustrydaily.com/germanys-fuinfosys-c4i- system-02899/ x Germany - Faust http://defense-update.com/products/f/Faust.htm x Italy - Forza NEC http://www.defenseindustrydaily.com/Italys-Forza-NEC-Battlefield-Command-System- 06432/ x US Army’s Blue Force Tracker http://www.defenseindustrydaily.com/1341m-for-blue-force-tracker- global-services-0427/ x UK - ASTOR Airborne Stand-Off Reconnaissance http://www.raytheon.com/capabilities/products/astor/

Page 10 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

3.3 Human-Computer Interaction for Decision Making One important technology stream of the C3-MAAD project is Human-Computer Interaction (HCI) mechanisms that enable an operator and the system to engage in problem solving and decision making in a collaborative, adaptive and mixed-initiative manner.

In the context of C3-MAAD, collaborative means that the operator and the system are engaged in a joint activity and endeavour to find a solution together. Adaptive means that the system is aware of the context in which the operator is performing his task and can adapt its interaction accordingly (this context includes the human- system communication context, the user and the task characteristics, the parameters of the operational context, etc.). Finally, mixed-initiative refers to a flexible interaction strategy where each agent (operator or system) can contribute to the task that it does best. The roles are opportunistically negotiated between the agents as the problem is being solved. It is also important that user preferences be taken into account since all the operators will not require the same type of information for performing their task.

Based on these definitions and context, a search strategy was derived to identify current scientific and technical research. Further details on the search terms used are provided in Appendix 5.1.1. The literature on HCI and on computer-supported collaborative work is quite extensive and so limits had to be applied - the concept of decision-making was added to the strategy in order to make the results very precise to our clients’ line of inquiry and to identify experts applying HCI technologies for decision-making.

3.3.1 Major Players When considering the affiliations of the authors in our dataset, we can see that the majority of players are from academic institutions (74%), government organizations (15%), corporations (10%) and some hospitals (1%). Figure 1 illustrates the distribution of numbers of publications by the types of organizations. The appearance of hospitals in our dataset is likely related to the application of HCI research in the field of clinical decision making. In the list of corporations, we see many high-tech software and hardware companies, such as IBM, Google, Mitsubishi, Siemens and Samsung; telecommunications companies such as France Telecom, NTT Corp., and Nokia; and companies that are active in defense markets, such as Lockheed-Martin, Boeing, General Dynamics, MITRE Corporation and Northrop Grumman.

Page 11 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

10%

15% Academic Government Corporate Hospital 74%

Figure 1. Human-Computer Interaction – Percentage of publications by type of organization

Figure 2 shows those organizations with six or more publications. Further details on these organizations, and the next top 10, including co-authoring institutions, top authors and areas of expertise are provided in appendix 5.2.1. Not showing in this short list are a number of military organizations, such as: US Air Force Research Laboratory (AFRL), Wright-Patterson AFB, OH; Defence Science Technology Organisation (DSTO), Australia; DRDC Valcartier; the Royal Netherlands Navy, and others.

Carnegie Mellon University, Pittsburgh, PA, USA

Purdue University, West Lafayette, IN, United…

Pennsylvania State University, University Park,…

Massachusetts Institute of Technology (MIT),…

Delft University of Technology, Netherlands

University of Patras, Rio Patras, Greece

TNO (Netherlands Organization for Applied…

Korea Advanced Institute of Science and…

IBM Corp., United States

Georgia Institute of Technology, Atlanta, GA,…

Chinese Academy of Sciences, Beijing, China

024681012 Number of publications

Figure 2. HCI – Organizations with 6 or more publications

Page 12 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Canadian organizations were of particular importance for this project; however there were very few Canadian organizations in the dataset. This does not mean that Canadians are not active in the field, but that in the narrow field of our inquiry, Canadians do not publish in as significant numbers as other countries. Our initial search identified only 14 institutions, or 2.4% of all the institutions listed (14/589), that are Canadian. These Canadian organizations contributed to 20 papers, or 2.9% of the total papers (20/670). Because of these limited numbers, supplemental searches were conducted that were somewhat broader, but only in the Scopus database, due to time limitations. Details on this second strategy are found in appendix 5.1.1.

More significant results for Canadian organizations were retrieved with the second search and we were able to see that Canadians are indeed active in this field. We can also see that these Canadian institutions tend to collaborate internationally to a high degree. While our search specified that the author affiliation should be Canadian, institutions from other countries are also seen because the co-authors of these Canadian papers were also retrieved. So while there are 103 Canadian institutions listed, there are an additional 194 institutions listed that are not Canadian. In this field, authors from Canada are most likely to collaborate with authors from the United States, United Kingdom, China, France and Germany.

Canadian institutions with six or more publications are shown in Figure 3. All of the top 20 organizations are academic, but there are some government labs on the list with fewer than six publications (DRDC, NRC) as well as a few corporations, such as Lockheed Martin, Autodesk Research, InfoBright Inc., and Oculus Info Inc. A complete list of all the companies found in our dataset is provided in the attachment to this report (filename: STI 7193 C3-MAAD Canadian players.xls). In the same file, a detailed list with areas of expertise and links to websites (where available) are provided for the most prolific institutions, or others that were selected because of their relevance. In addition, links to Canadian laboratories are found in appendix 5.2.3. This list is selective rather than exhaustive - only those organizations that appear to have the most relevance to this project have been retained since it was not possible within the timeframe given to gather details on all 103 institutions identified by the search.

Page 13 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

University of British Columbia, Vancouver, BC,… Simon Fraser University, Burnaby, BC, Canada University of Toronto, ON, Canada University of Calgary, AB, Canada Concordia University, Montreal, QC, Canada University of Waterloo, ON, Canada University of Ottawa, Ottawa, ON, Canada University of Alberta, Edmonton, AB, Canada University of Saskatchewan, Saskatoon, SK, Canada University of Victoria, BC, Canada Carleton University, Ottawa, ON, Canada University of Regina, SK, Canada McMaster University, Hamilton, ON, Canada York University, Toronto, ON, Canada University of Manitoba, Winnipeg, MB, Canada Queens University, Kingston, ON, Canada University of Guelph, ON, Canada Université de Montréal, QC, Canada University of Western Ontario, London, ON, Canada McGill University, Montreal, QC, Canada

0 5 10 15 20 25 30 35 40 Number of publications

Figure 3. HCI – Canadian organizations with 6 or more publications

3.3.2 Top Authors Over 1,900 individual authors were found in our dataset, however only a small percentage (0.8% or 16/1942) of these authored three or more publications. Interestingly, the most prolific authors were not necessarily from the most prolific institutions, the top two being from The University of Texas at Houston and CSIRO, Australia, respectively. Table 3 lists all authors with three or more publications and the author affiliations listed on their papers. It should be noted that as all author affiliations are listed in the database records, some of the affiliations in the list may be those of co-authors. The first organization listed is most likely to be the one where the named author works.

The top authors from the Canadian-only dataset are listed in Table 4. For this dataset, authors with five or more publications are shown.

Page 14 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Table 3. Human-computer interaction – Authors with 3 or more publications Author Name Affiliation(s) of Author Zhang, J.[5] University of Texas at Houston, TX, USA [2]; Boeing Company, USA [1]; Chinese Academy of Sciences, Beijing, China [1]; Columbia University, New York, NY, United States [1]; Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland [1]; Inha Univ., Incheon, South Korea [1]

Jianxin Li[4] Commonwealth Scientific and Industrial Research Organization (CSIRO), NSW, Australia [2]; Columbia University, New York, NY, United States [1]; IBM Corp., United States [1]; North Dakota State University, Fargo, ND, United States [1]

Pu, P.[4] Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland [3]; Hong Kong Baptist University, Hong Kong, China [1]

Stasko, J.[4] Georgia Institute of Technology, Atlanta, GA, United States [4] Baloian, N.[3] University of Chile, Santiago, Chile [3] Breazeal, C.[3] Massachusetts Institute of Technology (MIT), Cambridge, MA, USA [3] Chen, L.[3] Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland [2]; Hong Kong Baptist University, Hong Kong, China [1]

Chi EH[3] Palo Alto Research Center, Palo Alto, CA, USA [3] Jinquan Wang[3] Centrum voor Wiskunde en Informatica (CWI), Amsterdam, Netherlands [1]; Delft University of Technology, Netherlands [1]; Microsoft Research Asia, Beijing, China [1]; Nanjing University, China [1]; Tsinghua University, Beijing, China [1]

Jung Hyun Kim[3] Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea [1]; Sungkyunkwan University, Suwon, South Korea [1]; Universitat zu Koln. Albertus-Magnus-Platz, Koln, Germany [1]

Pirolli P.[3] Palo Alto Research Center, Palo Alto, CA, USA [2]; University of Miami, FL, USA [1]

Ricci, F.[3] Free University of Bozen-Bolzano, Italy [2]; Bell ID, Rotterdam, Netherlands [1]; University of Haifa, Israel [1]

Sanchez J.[3] University of Chile, Santiago, Chile [2]; John Deere Technol. Center, Moline, IL USA [1]

Xiaocong Fan[3] Pennsylvania State University, University Park, PA, USA [2]; US Army Research Laboratory (ARL), Aberdeen Proving Ground, MD, USA [1]

Yang, J.[3] Carnegie Mellon University, Pittsburgh, PA, USA [1]; East China Normal University (ICA-ECNU), Shanghai, China [1]; Zhejiang University, Hangzhou, China [1]

Yen, J.[3] Pennsylvania State University, University Park, PA, USA [2]; US Army Research Laboratory (ARL), Aberdeen Proving Ground, MD, USA [1]

Page 15 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Table 4. HCI – Canadian Authors with 5 or more publications Author Name Affiliation(s) of Author Seffah, A.[8] Concordia University, Montreal, QC, Canada [8]; University of Rostock, Germany [1]; Université du Québec à Montréal (UQAM), Montreal, QC, Canada [1]; VeriSign Inc., Mountain View, CA, United States [1] Carpendale, S.[7] University of Calgary, AB, Canada [7]; INRIA, Orsay, France [1]; Massachussetts Institute of Technology (MIT), Cambridge, MA, USA [1]; Microsoft Research Ltd., Redmond, WA, USA [1]; University of Magdeburg, Magdeburg, Germany [1]; École Centrale Paris, Paris, France [1] Shirmohammadi, S.[7] University of Ottawa, Ottawa, ON, Canada [7]; National Laboratory for Scientific Computing, Petrópolis, Brazil [1]; Sharif University of Technology, Tehran, Iran [1]

Dill, J.[6] Simon Fraser University, Burnaby, BC, Canada [6]; City University, London, United Kingdom [1]; Trinity College, Hartford, CT, United States [1]; Yale University, New Haven, CT, United States [1]

Kushniruk, A.[6] University of Victoria, BC, Canada [5]; Aalborg University, Copenhagen, Denmark [1]; Centre for Addiction and Mental Health, Toronto, ON, Canada [1]; HealthLink BC, Ministry of Health Services, BC, Canada [1]; Northern Ontario School of Medicine, ON, Canada [1]; Registered Nurses' Association of Ontario, ON, Canada [1]

Subramanian, S.[6] University of Saskatchewan, Saskatoon, SK, Canada [4]; Osaka University, Japan [3]; University of Bristol, United Kingdom [2]; Baycrest (Health Centre), Toronto, ON, Canada [1]; Fallon Clinic Foundation, Worcester, MA, United States [1]; Philips Research, Eindhoven, Netherlands [1] Conati, C.[5] University of British Columbia, Vancouver, BC, Canada [5]; University of Trento, Italy [1]; University of Washington, Seattle, WA, United States [1]

Fisher, B.[5] Simon Fraser University, Burnaby, BC, Canada [5]; University of British Columbia, Vancouver, BC, Canada [2]; Purdue University, West Lafayette, IN, United States [1]; University of Chicago, United States [1]

Ho, K.[5] University of British Columbia, Vancouver, BC, Canada [3]; Simon Fraser University, Burnaby, BC, Canada [2]; Arthritis Research Centre of Canada, Vancouver, Canada [1]; Centre for Digital Media, Vancouver, Canada [1]; University of Alberta, Edmonton, AB, Canada [1]; University of Ottawa, Ottawa, ON, Canada [1]

Page 16 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Author Name Affiliation(s) of Author Lindgaard, G.[5] Carleton University, Ottawa, ON, Canada [5]; Berlin University of Technology, Germany [1]; Communications Research Centre (CRC), Ottawa, ON, Canada [1]; DDD SYSTEMS, Dorset, United Kingdom [1]; Kingston University, Surrey, United Kingdom [1]

Stuerzlinger, W.[5] York University, Toronto, ON, Canada [5]; Bauhaus-University, Weimar, Germany [1]; INRIA, Orsay, France [1]; Osaka University, Japan [1]; University of Arizona, Tucson, AZ, United States [1]; University of Tokyo, Japan [1]

3.4 Decision Support Tools

For this project, the clients were interested in collaborative tools and concepts that can enable a group of operators to engage in collaborative sensemaking, decision making and planning, whether it is in the context of a national or multinational (coalition) Task Group/Force. For the search, we did not specify a military context but did attempt to search for articles related only to collaborative decision making, groups of operators and distributed teams. The search strategy is described in appendix 5.1.1.

3.4.1 Major Players The literature retrieved for this section covered a variety of applications and industries, most notably medical clinical decision making tools, the oil drilling industry, military decision support, logistics and online gaming. The majority of players are from academic institutions, government organizations (especially military), corporations and some hospitals. Figure 4 illustrates the distribution of numbers of publications by the types of organizations. There are significantly fewer publications attributed to academic institutions in this dataset than in the HCI dataset. This suggests that decision support tools are much more mature commercially and that the research in this area tends to be more applied than theoretical in comparison to the HCI dataset.

Page 17 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

18%

Academic Corporate Government 19% 61% Hospital

Figure 4. Decision Support Tools – Percentage of publications by type of organization

The hospitals represented in this dataset, and many of the corporations, are publishing papers on clinical decision making tools, while many of the other companies are from the petroleum industry (for example, BP Global, Saudi Aramco, Schlumberger), or they are typically companies that serve the military and aerospace markets, such as Boeing, Lockheed Martin, BAE Systems, and MITRE Corporation. A large portion of the government players are military departments, primarily from the United States (for example, US Air Force Research Laboratory, NASA and the Defense Information Systems Agency), but we also see many of the military colleges, such as the US Naval Postgraduate School, and many large government labs, such as Lawrence Berkeley and Sandia National Laboratories, as well as Defence R&D Canada. Figure 5 shows those organizations with five or more publications. Further details on these organizations, and the next top 10, including co-authoring institutions, top authors and areas of expertise are provided in appendix 5.2.2.

Page 18 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Naval Postgraduate School, Monterey, CA, USA

BP Global, UK

Nanyang Technological University, Singapore

US Air Force Research Laboratory (AFRL), Wright-…

Massachusetts Institute of Technology (MIT), Cambridge,…

University of Patras, Rio Patras, Greece

Shanghai Jiao Tong University, Shanghai, China

Schlumberger Corp.,

Lockheed-Martin Corporation, United States

Hong Kong Polytechnic University, Hong Kong

Drexel University, Philadelphia, PA, United States

Delft University of Technology, Netherlands

0246810121416 Number of publications

Figure 5. Decision Support Tools – Top Organizations

Canadian organizations were of particular importance to this project; however there were very few Canadian organizations in the dataset. Our initial search identified 34 institutions, or 4.5% of all the institutions listed (34/746), that are Canadian. These Canadian organizations contributed to 35 papers, or 5% of the total papers (35/700). Because of these limited numbers, supplemental searches were conducted that were somewhat broader, but only in the Scopus database, due to time limitations. Details on this second strategy are found in appendix 5.1.1.

Results of the second search were much more telling, though the results had to be manually weeded to remove papers that were not relevant, particularly those that referred to decision making processes or techniques (especially clinical decision making) rather than software tools or particular types of displays. Canadian institutions with five or more publications are shown in Figure 6. With the exception of DRDC Valcartier and the National Research Council, they are all academic institutions. There are some corporations in the list as well, but they had fewer publications, for example: Oculus Info Inc., Rolls-Royce Canada, Gallium Visual Services Inc., Lansdowne Technologies and others. A detailed list with areas of expertise and links to websites (where available) are provided in an attachment to this report (filename: STI 7193 C3-MAAD Canadian players.xls). In addition, appendix 5.2.3 provides links to Canadian laboratories. This list is selective rather than exhaustive - only those organizations that appear to have the most relevance to this project have been retained since it was not possible within the timeframe of this project to gather details on all 110 institutions identified by the search. All the companies in the dataset are listed in the Excel file but details and web links are not provided.

Page 19 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

University of Waterloo, ON, Canada University of British Columbia, Vancouver, BC, Canada University of Toronto, ON, Canada Université Laval, Québec, QC, Canada University of Calgary, AB, Canada University of Ottawa, ON, Canada University of Western Ontario, London, ON, Canada Simon Fraser University, Burnaby, BC, Canada University of Manitoba, Winnipeg, MB, Canada University of Alberta, Edmonton, AB, Canada McMaster University, Hamilton, ON, Canada Concordia University, Montreal, QC, Canada University of New Brunswick, Fredericton, NB, Canada DRDC Valcartier, Quebec, QC, Canada Wilfrid Laurier University, Waterloo, ON, Canada Ryerson University, Toronto, ON, Canada McGill University, Montreal, QC, Canada Ecole Polytechnique de Montreal, QC, Canada Université de Montréal, QC, Canada National Research Council Canada, London, ON, Canada Centre Hospitalier Universitaire de Québec (CHUQ),…

0 5 10 15 20 25 Number of publications

Figure 6. Decision Support Tools – Canadian Players

3.4.2 Top Authors Table 5 below lists all those authors in our dataset with four or more publications. Probably of most interest in this list is Robert S. Bolia of the US Air Force Research Laboratory. His publications can primarily be found in the presentations of the International Command and Control Research and Technology Symposium (ICCRTS) (available from: http://www.dodccrp.org/html4/events_past.html ).

Table 6 lists all the Canadian authors with four or more publications.

Page 20 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Table 5. Decision Support tools – Authors with 4 or more publications Author Name Affiliation(s) of Author Avouris, N.[5] University of Patras, Rio Patras, Greece [5]; University of Freiburg, Germany [1]

Lauche, K.[5] Delft University of Technology, Netherlands [5]; BP Global, UK [4]; CJSC Elvary Neftegaz [1]; People Factor Consultant Ltd. [1]; University of Aberdeen, Scotland, UK [1]

Sawaryn, S. J.[5] BP Global, UK [5]; Delft University of Technology, Netherlands [4]; CJSC Elvary Neftegaz [1]; People Factor Consultant Ltd. [1]; University of Aberdeen, Scotland, UK [1]

Bolia, Robert S.[4] US Air Force Research Laboratory (AFRL), Wright-Patterson AFB, OH, USA [3]; Boeing Company, USA [1]; Defence Science and Technology Laboratory (DSTL), Farnborough, UK [1]; General Dynamics Corp., USA [1]; Human Performance Architects, Orlando, FL [1]; Massachusetts Institute of Technology (MIT), Cambridge, MA, USA [1]

Jianxin Li[4] Beihang University, Beijing, China [1]; Beijing Academy of Science and Technology, China [1]; Edith Cowan Univ., Perth, WA, Australia [1]; Shanghai Jiao Tong University, Shanghai, China [1]; University of Delaware, Newark, DE, USA [1]; University of Science Malaysia, Penang, Malaysia [1]

Jie Lu[4] University of Technology, Sydney (UTS), NSW, Australia [4]; Belgian Nuclear Research Centre (SCK.CEN), Boeretang, Belgium [3]; Brussels EU Chapter, Club of Rome (CoR-EU), Belgium [1]; Ecole Nationale Supérieure des Arts et Industries Textiles, France [1]; Flemish Institute for Technological Research (VITO), Belgium [1]; University of Leuven (KULeuven), Heverlee, Belgium [1] Kapur, M.[4] Nanyang Technological University, Singapore [4]; Columbia University, New York, NY, United States [3]

Rommetveit, R.[4] eDrilling Solutions AS, Narvik, Norway [2]; Bouvet AS, Narvik, Norway [1]; ConocoPhillips Norge AS, Norway [1]; eDrilling Solutions, Australia [1]; Edrilling Solutions, United States [1]; Narvik University College, Narvik, Norway [1]

Tien, J. M.[4] University of Miami, Coral Gables, FL, United States [3]; Rensselaer Polytechnic Institute, Troy, NY, United States [1]

Zhang, G.[4] University of Technology, Sydney (UTS), NSW, Australia [4]; Belgian Nuclear Research Centre (SCK.CEN), Boeretang, Belgium [3]; Brussels EU Chapter, Club of Rome (CoR-EU), Belgium [1]; Ecole Nationale Supérieure des Arts et Industries Textiles, France [1]; Flemish Institute for Technological Research (VITO), Belgium [1]; University of Leuven (KULeuven), Heverlee, Belgium [1]

Page 21 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Table 6. Decision Support Tools – Canadian Authors with 4 or more publications Author Name Affiliation(s) of Author Hipel, K. W.[7] University of Waterloo, ON, Canada [7]; Nanjing University of Aeronautics and Astronautics (NUAA), China [3]; Wilfrid Laurier University, Waterloo, ON, Canada [3]; Tokyo Institute of Technology, Tokyo, Japan [2]; Ryerson University, Toronto, ON, Canada [1]; Sheffield Hallam University, United Kingdom [1]

Légaré, F.[6] Centre Hospitalier Universitaire de Québec (CHUQ), Quebec, QC, Canada [5]; Université Laval, Québec, QC, Canada [5]; University of Ottawa, ON, Canada [4]; Cardiff University, Cardiff, Wales, United Kingdom [3]; Centre de Santé et de Services Sociaux de la Vieille-Capitale, Québec, QC, Canada [2]; Maastricht University, Maastricht, Netherlands [2]

Martel, J.-M.[6] Université Laval, Québec, QC, Canada [6]; DRDC Valcartier, Quebec, QC, Canada [2]; Institut Supérieur de Commerce et de Comptabilité de Bizerte (ISCCB), Tunisia [2]; GIAD, Faculté des Sciences Economiques et de Gestion de Sfax, Tunisia [1]; LOGIQ, Institut Supérieur de Gestion Industrielle de Sfax, Tunisia [1]; University of Economic Sciences and Management, Sfax, Tunisia [1]

Cowan, D. D.[5] University of Waterloo, ON, Canada [5]; Pontifícia Universidade do Rio Grande do Sul (PUCRS), Brazil [1]; University of Texas at Dallas, Richardson, TX, United States [1]

Stacey, D.[5] University of Ottawa, ON, Canada [5]; Centre Hospitalier Universitaire de Québec (CHUQ), Quebec, QC, Canada [3]; Cardiff University, Cardiff, Wales, United Kingdom [2]; Maastricht University, Maastricht, Netherlands [2]; Maine Medical Center, Portland, ME, United States [2]; Michigan State University, East Lansing, MI, United States [2]; Oregon Health Sciences University, Portland, OR, United States [2]; University of Lyon, Lyon, France [2]; Université Laval, Québec, QC, Canada [2]

Wang, D.[5] University of Manitoba, Winnipeg, MB, Canada [5]; University of Ontario Institute of Technology, Oshawa, ON, Canada [4]

AbouRizk, S. M.[4] University of Alberta, Edmonton, AB, Canada [4]; City of Edmonton Asset Management and Public Works, Drainage Services, Design and Construction, AB, Canada [2]; Columbia University, New York, NY, United States [1]; SMA Consulting Ltd.,Edmonton, AB, Canada [1]; Univ. of Illinois at Urbana-Champaign, Urbana, IL, United States [1]

Elwyn, G.[4] Cardiff University, Cardiff, Wales, United Kingdom [4]; Maastricht University, Maastricht, Netherlands [3]; University of Ottawa, ON, Canada [3]; Centre Hospitalier Universitaire de Québec (CHUQ), Quebec, QC, Canada [2]; University of Newcastle, Framlington Place, Newcastle upon Tyne, UK [2]; Université Laval, Québec, QC, Canada [2]

Page 22 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Author Name Affiliation(s) of Author Jabeur, K.[4] Université Laval, Québec, QC, Canada [4]; DRDC Valcartier, Quebec, QC, Canada [2]; Institut Supérieur de Commerce et de Comptabilité de Bizerte (ISCCB), Tunisia [2]

Mendonca, M.[4] University of Waterloo, ON, Canada [4]; Pontifícia Universidade do Rio Grande do Sul (PUCRS), Brazil [1]

Naterer, G. F.[4] University of Manitoba, Winnipeg, MB, Canada [4]; University of Ontario Institute of Technology, Oshawa, ON, Canada [4]

Shen, W.[4] National Research Council Canada, London, ON, Canada [4]; University of Western Ontario, London, ON, Canada [2]; Concordia University, Montreal, QC, Canada [1]; Zhejiang Normal University, China [1]

Sheppard, S. R. J.[4] University of British Columbia, Vancouver, BC, Canada [4]; Arizona State University, United States [2]; Metro Vancouver, Burnaby, BC, Canada [2]; Environment Canada, Vancouver, BC, Canada [1]; University of Oxford, Oxford, United Kingdom [1]

Wang, G. G.[4] University of Manitoba, Winnipeg, MB, Canada [4]; University of Ontario Institute of Technology, Oshawa, ON, Canada [4]

Wang, L.[4] China University of Geosciences, China [1]; National Research Council Canada, London, ON, Canada [1]; Ryerson University, Toronto, ON, Canada [1]; University of British Columbia, Vancouver, BC, Canada [1]; University of Skövde, Sweden [1]; University of Waterloo, ON, Canada [1]

Page 23 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

3.5 Conclusions Naval Tactical Systems Only a few collaborative decision making tools specifically for naval tactical operations were identified, particularly the Cooperative Engagement Capability (CEC), the Multiplatform Engagement Capability (MPEC) and Thales’ Combat Management Systems. Only limited technical information on Thales’ and Raytheon’s systems was available but links to product information have been provided. Other projects and programmes not specifically for the Navy are worth watching for new developments, namely: NATO’s Network Enabled Capability (NNEC) program, the US Air Force Network-Centric Collaborative Targeting (NCCT) and the US Joint Integrated Air and Missile Defense (JIAMD) initiative, which is a joint system purported to be similar to the CEC.

Human Computer Interaction (HCI) This portion of our research was concentrated on identifying major players. Few Canadian players were identified in a narrow search on HCI for collaborative decision making, but a broader search showed that the following Canadian institutions are the top five actively engaged in HCI research: x University of British Columbia, Vancouver, BC x Simon Fraser University, Burnaby, BC x University of Toronto, ON x University of Calgary, AB x Concordia University, Montreal, QC

From this set, the top five Canadian players in collaborative decision making tools research are: x University of Waterloo, ON x University of British Columbia, Vancouver, BC x University of Toronto, ON x Université Laval, Québec, QC x University of Calgary, AB

Page 24 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Areas for further study The data gathered for this study will remain useful for studies on technology trends in the domain. Subject- based analysis and emerging trends could easily be extracted from the data collected and crossed with the major players to see more clearly who is working on what. A second mandate could be drafted to address these issues.

The significant amounts of money that are being invested in CEC and the JIAMD may also mean that many new technological developments can be expected in the coming years. It would be worthwhile for DRDC researchers to continue to monitor developments related to CEC and NATO’s NNEC program to stay on top of new technologies in the area. Using the search strategies developed for this mandate, literature alerts could be established by DRDC’s Information Centre to assist with this activity.

Page 25 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

4 REFERENCES

Acevedo, Rafael A. 2006. Valued Information at the Right-Time (VIRT) and the Navy's Cooperative Engagement Capability (CEC) - A Win/Win Proposition. Master's Thesis, Department of Information Sciences, Naval Postgraduate School, Monterey, CA.

Advanced Tactical Targeting Technology (AT3) (US Air Force Materiel Command, Wright Laboratory), Military CNs, FMs, data and threat management. 2005. Jane's Avionics.

Domingo, Alberto, and Miguel Angel Rico. 2010. NATO Network-Enabled Capability (NNEC) Frequently Asked Questions (FAQ), Interim Document ACT-NNEC-FAQ-001/01. Norfolk, VA: North Atlantic Treaty Organization (NATO).

Kaspar, Beth. 1999. Advanced Tactical Targeting Technology Presentation DARPATECH 99, http://archive.darpa.mil/darpatech99/Presentations/spopdf/spoat3final.pdf

Nolin, Pierre Claude. 2006. Execution Phase: Targeting and Shooting. In 177 STC 06 E - Interoperability: The Need for Transatlantic Harmonisation Brussels: NATO Parliamentary Assembly. http://www.nato- pa.int/default.asp?SHORTCUT=1004 (accessed 3 May 2011).

O'Neil, William D. 2007. The Cooperative Engagement Capability (CEC): Transforming Naval Anti-air Warfare, Case Studies in National Security Transformation, No. 11. Washington, DC: National Defense Universtiy, Center for Technology and National Security Policy.

O'Rourke, Ronald. 2005. Navy Network-Centric Warfare Concept: Key Programs and Issues for Congress. http://www.fas.org/sgp/crs/natsec/RS20557.pdf (accessed 3 May 2011).

Pocock, Chris 2008. Locating trouble: Network-Centric Collaborative Targeting system goes operational C4ISR Journal, http://www.c4isrjournal.com/story.php?F=3312805.

Raytheon. 2010. Raytheon Awarded Cooperative Engagement Capability Contracts. http://www.raytheon.com/newsroom/technology/rtn10_nvylg/news7/.

Scott, Richard. 2006. France to demostrate multi-platform network. Jane's Defence Weekly 25 Jan 2006:6.

U.S. Air Force. 2010. Exhibit R-2, RDT&E Budget Item Justification: PB 2011 Air Force. PE 0606323F: Multi-Service Systems Engineering. In Research, Development, Test & Evaluation, Air Force BA 6: RDT&E Management Support. Washington, DC. http://www.dtic.mil/descriptivesum/Y2011/AirForce/0606323F_PB_2011.pdf (accessed 3 May 2011).

U.S. Navy. 2011. Exhibit R-2, RDT&E Budget Item Justification: PB 2012 Navy. PE 0603658N: Cooperative Engagement. In Research, Development, Test & Evaluation, Navy. BA 4: Advanced Component Development & Prototypes (ACD&P). Washington, DC: US Department of Defense. http://www.dtic.mil/descriptivesum/Y2012/Navy/0603658N_4_PB_2012.pdf (accessed 10 May 2011).

Page 26 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

5 APPENDICES 5.1 Methodology 5.1.1 Searches Several searches were conducted in various databases, particularly INSPEC, Ei-Compendex, Scopus, NTIS and NATO Scientific Publications. Results were limited to the last 5 years (2006-2011). Additional manual searches were performed in the sources listed in section 5.1.3 below.

Conference proceedings of the annual International Command and Control Research and Technology Symposium (ICCRTS) 2006-2010 were scanned for relevant articles and manually added to the database (http://www.dodccrp.org/html4/events_past.html).

The table below shows groups of concepts, which were combined in multiple variations using database-specific syntax to obtain relevant references.

Part 1 - Human-Computer Interaction for decision making

Search concepts: 1: HCI and Interfaces 2: Decision support 3: Adaptive/collaborative

Human computer interaction Decision support Adaptive system HCI Decision making Strategic interaction User interfaces Decision theory Interactive system GUI Decision aids Cognitive system Man machine systems Problem solving Cognitive support User-computer interfaces Solve problems Augmented cognition Human-automation Sensemaking Context sensitive interaction Sense making Dynamic information Operator-machine interface User preferences Multimodal interfaces User-defined Visual analysis Visual analytics Collaborative Visual displays collaboration Multimedia interfaces

The original search combined these sets as follows: 1 and 2 and 3.

For the supplemental searches for Canadian players, these sets were combined in two different ways ((1 and 2) OR (1 and 3)) and limited to author affiliations in Canada only.

All results were limited by date for publication years 2006-2011.

Total results for HCI international set: 670 records Total results for the HCI Canada-only set: 356 records.

Page 27 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Part 2- Collaborative Decision-Making Tools

Search concepts: 1: Decision support 2:Collaborative/groups 3: Real-time 4: Software/systems 5: Military (optional)

Decision support Collaborative Realtime Software Tactical Decision making Collaboration Real-time Tool Military Decision aids Cooperative work Live Systems Defense Decision tools Co-operative work Synchronous Environments Defence Problem solving Joint cognition Simultaneous Combat Solve problems Joint cognitive Concurrent Warfighter Sensemaking Task force Battle* Sense making Multinational Air Force Coalition Navy Groupware Naval Group decision Army Distributed teams Warfare Distributed work teams Soldier Remote team Armed Forces Group of operators Command and control C2 C4ISR

For this search, two combinations of sets were executed: 1 and 2 and 3 and 4 1 and 2 and 3 and 5

For the supplemental searches for Canadian players, results were limited to author affiliations in Canada and these sets were combined in two different ways: (1 and 2 and 3) OR (1 and 2 and 4).

All results were limited by date for publication years 2006-2011. In addition, the results were manually scanned by reading abstracts and titles to eliminate non-relevant publications.

Total records for the Decision Support tools, international dataset: 726 records. Total records for the Decision Support Tools, Canada-only set: 234 records.

5.1.2 Analysis All references were downloaded into VantagePoint software for analysis. VantagePoint allows us to create various groupings, matrices, graphs, cross-correlations and statistical analyses to analyze the data and draw conclusions about topics and subtopics and to profile the activities of the major players.

Author names and author affiliations were cleaned to harmonize variant forms and spellings and group together departments from the same institutions.

Keywords, identifiers (akin to author-supplied keywords), descriptors, subject headings and phrases and words from titles were merged together to facilitate subject analysis, resulting in over 5,800 and 7,800 terms for HCI

Page 28 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011 and Decision Support, respectively. These terms were cleaned and edited to harmonize variant spellings, acronyms and similar meanings.

5.1.3 Sources Consulted

Scientific & Technical Literature: x Scopus (accessed via CISTI license) x INSPEC (accessed via CISTI license) x EiCompendex (accessed via CISTI license) x NTIS (accessed via DRDC license) x NATO Research & Technology Organisation - Scientific Publications http://www.rta.nato.int/abstracts.aspx

Market and Trade Literature: x Frost & Sullivan (accessed via CISTI license) x Marketresearch.com x Frost and Sullivan x Strategic Business Insights x IDC x Defense Industry Daily http://www.defenseindustrydaily.com/ x Defense Update.com http://defense-update.com/ x Global Security.org http://www.globalsecurity.org/ x NNEC Information Portal https://transnet.act.nato.int/WISE/Informatio/index_html

Additional sources: x NATO http://www.nato-pa.int/ x Command & Control Centre of Excellence http://www.c2coe.org/ x Command and Control Research Program http://www.dodccrp.org/ x Johns Hopkins University Applied Physics Laboratory http://www.jhuapl.edu/ x C2Pedia http://www.c2coe.org/c2pedia/index.php?title=Main_Page x NACMA http://www.nacma.nato.int/ x Federation of American Scientists http://www.fas.org/ x Tactical Report http://www.tacticalreport.com/ x US Army Program Executive Office for Command, Control and Communications-Tactical http://peoc3t.monmouth.army.mil/c3t/ x Joint Air Power Competence Centre http://www.japcc.de/c4istar.html x Australian Department of Defence http://www.defence.gov.au/ x Jane’s Guide http://articles.janes.com/articles/Janes-Military-Communications/ x AL Defaiya http://www.defaiya.com/defaiyaonline/ x DARPA http://archive.darpa.mil x Rafael Advanced Defence Systems http://www.rafael.co.il/

Page 29 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

x Northrop Grumman http://www.as.northropgrumman.com/ x BAE Systems http://www.baesystems.com/ x Comcept Inc http://www.comceptinc.com/ x Israel Aerospace Industries http://www.iai.co.il/ x Thales Group http://www.thalesgroup.com/ x Thales Raytheon http://www.thalesraytheon.com/ x Cassidian http://www.cassidian.com/cassidian/int/en/ x Raytheon http://www.raytheon.com/ x General Dynamics Canada - http://www.gdcanada.com/ x Finmeccanica http://www.finmeccanica.it/EN/

The following review article may also be particularly useful: Seymour, George E., and Michael Cowen. 2006. A Review of Team Collaboration tools for Crisis Response in the Military and Government. In 2006 Command and Control Research and Technology Symposium. San Diego, CA. Available: http://www.dodccrp.org/events/2006_CCRTS/html/papers/037.pdf

Page 30 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

5.2 Major Players data 5.2.1 HCI - Major Players Table 6. Human Computer Interaction – Organizations with 5 or more publications Organization Name Top Authors Co-authoring institutions Top subject terms Carnegie Mellon Nourbakhsh I. [2]; DeepLocal, Pittsburgh, PA, Decision Making [5]; University, Pittsburgh, PA, Chai, J. [1]; United States [1]; User interfaces [5]; USA[11] Chang C. -Y. [1]; George Mason University, Problem Solving [4]; Chen, D. [1]; Fairfax, VA, USA [1]; computer-supported cooperative work DiSalvo, C. [1]; Michigan State University, East [2]; Garlan, D. [1] Lansing, MI, United States [1]; Constraint theory [2]; National Chengchi University, Mobile phones [2]; Taipei, Taiwan [1]; Robotics [2]; National Taiwan Normal Adaptive Systems [1]; University, Taipei, Taiwan [1] affective behavior modeling [1]; Animation [1] Purdue University, West Collins TF [2]; California Polytechnic State Decision Making [4]; Lafayette, IN, United Ebert DS [2]; University, San Luis Obispo, CA, Problem Solving [4]; States[10] Yun Jang [2]; United States [1]; data visualization [3]; Ault A. [1]; Colorado School of Mines, Human-computer interaction (HCI) [3]; Babbar Sebens M [1]; Colorado, CO, USA [1]; Visual analytics [3]; Bue B. [1] George Mason University, Command And Control Systems [2]; Fairfax, VA, USA [1]; emergency services [2]; Illinois Institute of Technology, Interactive systems [2]; Chicago, IL, United States [1]; Mathematical Models [2]; Indiana University. Bloomington, Mobile devices [2]; IN, United States [1] Situational awareness (SA) [2]; Teaching [2]; User interfaces [2] Massachusetts Institute of Breazeal, C. [3]; Arizona State University, Cognitive systems [4]; Technology (MIT), Picard, R. [2]; Phoenix, AZ, United States [1]; Human-computer interaction (HCI) [4]; Cambridge, MA, USA[9] Wang, A. [2]; Univ. of Pavia, via Ferrata 1, human-robot interaction [4]; A. S. Clare [1]; 27100 Pavia, Italy [1]; Learning systems [3]; Abu-Hanna, A. [1]; University of Amsterdam, Artificial Intelligence [2]; Ahn H. -I. [1] Netherlands [1]; Computational agents [2]; University of Arizona, Tucson, AZ, Graphical user interfaces (GUI) [2]; United States [1]; Indirect collaboration [2]; University of Konstanz. Germany Indirect human computer interaction [1] [2]; Learning [2]; man-machine systems [2]; Robotics [2]; User interfaces [2]; User studies [2]

Page 31 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-authoring institutions Top subject terms Pennsylvania State Xiaocong Fan [2]; Huazhong University of Science Human-computer interaction (HCI) [4]; University, University Yen, J. [2]; and Technology, Wuhan, China Problem Solving [4]; Park, PA, USA[9] Cai, S. [1]; [1]; Cognitive systems [3]; Carroll, J. M. [1]; Rutgers University, New Decision Support Systems (DSS) [2]; Farooq, U. [1]; Brunswick, NJ, United States [1]; Geographic information systems (GIS) Ganoe, C. H. [1] University of Illinois at Urbana- [2]; Champaign, Urbana, IL, United Human-centered teamwork [2]; States [1]; Software [2]; University of Missouri, Columbia, User interfaces [2]; MO, United States [1] Visual analytics [2]; 3D collaborative filtering [1]

Delft University of Ali, W. [1]; TNO (Netherlands Organization Decision Making [4]; Technology, Badke-Schaub, P. [1]; for Applied Scientific Research), Human-computer interaction (HCI) [4]; Netherlands[8] De Vries, A. P. [1]; Soesterberg, Netherlands [2]; User interfaces [4]; Grootjen, M. [1]; Centrum voor Wiskunde en Artificial Intelligence [3]; Hindriks, K. V. [1]; Informatica (CWI), Amsterdam, Cognitive Engineering (CE) [3]; Jalote-Parmar, A. [1] Netherlands [1]; Cognitive systems [3]; ErgoS Eng. & Ergonomics, Human engineering [3]; Enschede Netherlands [1]; Task Performance and Analysis [3]; Imperial College London, United Automation [2]; Kingdom [1]; Cognition [2]; Royal Netherlands Navy, The computer interfaces [2]; Hague, Netherlands [1] Control systems [2]; Decision Support [2]; Decision Support Systems (DSS) [2]; Equipment Design [2]; Information Systems [2]; Netherlands [2]; Problem Solving [2]; Task performance [2] Chinese Academy of Dai, G. [1]; Inha Univ., Incheon, South Korea Problem Solving [4]; Sciences, Beijing, China[6] Du, Y. [1]; [1]; User interfaces [3]; Gong, J. [1]; New Jersey Institute of Human-computer interaction (HCI) [2]; Han, Y. [1]; Technology, Newark, NJ, United Adaptive Systems [1]; Joobong Song [1]; States [1]; adaptive user interests modeling [1]; Liu, W. [1] Southwest Jiaotong University, Artificial Intelligence [1]; Chengdu, China [1]; cognition interactions [1]; University of Technology, Cognitive model [1]; Sydney, Australia [1]; Cognitive science [1]; University of Toledo, Toledo, OH, Cognitive systems [1] United States [1] Georgia Institute of Stasko, J. [4]; data visualization [2]; Technology, Atlanta, GA, Catrambone R. [2]; Decision Making [2]; United States[6] Cohen S [1]; Domain Experts [2]; Gorg C [1]; Interactive systems [2]; Hunter L [1]; interactive visualization [2]; M. L. Bink [1] investigative analysis [2]; Army Training [1]; Battle Management [1]; collaborative process [1]

Page 32 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-authoring institutions Top subject terms IBM Corp., United Aggarwal V. [1]; Columbia University, New York, User interfaces [3]; States[6] Behal A [1]; NY, United States [1]; Decision Support Systems (DSS) [2]; Borlawsky, T. [1]; IBM India Research Laboratory. Problem Solving [2]; Chandra, S. [1]; New Delhi India [1] advanced notification [1]; Christensen, J. E. [1]; Analysis process [1]; Gotz D. [1] analytic knowledge [1]; clinical decision support systems [1]; Clinical event monitor [1]; collaborative reasoning [1];

Korea Advanced Institute Yoon, W. C. [2]; National University of Singapore, User interfaces [4]; of Science and Technology Yoon, Y. S. [2]; Singapore [1] Cognitive systems [3]; (KAIST), Daejeon, South Cho, A. [1]; Mathematical Models [3]; Korea[6] Han B. -K. [1]; Decision Making [2]; Jung Hyun Kim [1]; Human-computer interaction (HCI) [2]; Jung, J. [1] Usability engineering [2]; 3D [1]; Abstract design principles [1]; feature extraction [1]; Adaptive control systems [1] TNO (Netherlands Neerincx, M. [2]; Delft University of Technology, Decision Support Systems (DSS) [4]; Organization for Applied van Maanen P P [2]; Netherlands [2]; Adaptive Systems [3]; Scientific Research), Grootjen, M. [1]; ErgoS Eng. & Ergonomics, Cognition [3]; Soesterberg, Janssen, W. [1]; Enschede Netherlands [1]; Decision Making [3]; Netherlands[6] Klos T [1]; Royal Netherlands Navy, The Augmented Cognition [2]; Lenior, D. [1] Hague, Netherlands [1] Automation [2]; Cognitive Engineering (CE) [2]; Cognitive systems [2]; Control systems [2]; Human factors [2]; Human-computer interaction (HCI) [2]; Task Performance and Analysis [2]; User interfaces [2] University of Patras, Rio Avouris, N. [2]; Aristotle University of User interfaces [5]; Patras, Greece[6] Adamides, E. [1]; Thessaloniki, Greece [1]; Collaborative problem solving [3]; Bouras, C. [1]; Research Academic Computer Groupware [3]; Evangelou, C. [1]; Technology Institute (CTI), Algorithms [2]; Gortzis LG [1]; Greece [1]; computer-supported cooperative work Kahrimanis, G. [1] TEI of Messolonghi, Nea Ktiria, [2]; Messolonghi, Greece [1] Human-computer interaction (HCI) [2]; Learning systems [2]; Problem Solving [2]; argumentation-enabling mechanism [1]; associated structured dialogue scheme [1]

Page 33 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-authoring institutions Top subject terms Aristotle University of Manolopoulos, Y. [2]; Research Academic Computer Problem Solving [4]; Thessaloniki, Greece[5] Nanopoulos, A. [2]; Technology Institute (CTI), User interfaces [4]; Symeonidis, P. [2]; Greece [1]; Collaborative Filtering (CF) [2]; Bamidis, P. D. [1]; University of Patras, Rio Patras, computer-supported cooperative work Bouras, C. [1]; Greece [1] [2]; Bratsas, C. [1] Human-computer interaction (HCI) [2]; Recommender systems [2]; Semantics [2]; Academic parameters [1]; access to information [1]; Algorithms [1]

Ecole Polytechnique Pu, P. [3]; User interfaces [4]; Fédérale de Lausanne Chen, L. [2]; Cognitive systems [2]; (EPFL), Switzerland[5] Dillenbourg, P. [2]; collaborative learning [2]; Jermann, P. [2]; Computer aided instruction [2]; Cuendet, S. [1]; Decision Support [2]; Do-Lenh, S. [1] Decision Support Systems (DSS) [2]; Electronic commerce [2]; Groupware [2]; Human-computer interaction (HCI) [2]; Problem Solving [2]; Recommender systems [2] Palo Alto Research Center, Chi EH [3]; Science Applications design [2]; Palo Alto, CA, USA[5] Convertino G [2]; International Corporation (SAIC), activity awareness [1]; Lichan Hong [2]; McLean, VA, United States [1] advanced Web tools [1]; Nelson L [2]; analytics environments [1]; Pirolli P. [2]; application [1]; Back, M. [1] Architectural design [1]; Argumentation marshalling [1]; chemistry [1]; Collaboration [1]; collective intelligence [1] University of California, Alpine, P. M. [1]; University of Minnesota, Distributed computer systems [2]; Irvine, CA, USA[5] Baumer, E. [1]; Minneapolis, MN, United States Problem Solving [2]; Canales, L. [1]; [1] User interfaces [2]; Correa, A. [1]; Visualization [2]; DiGioia P. [1]; Adaptive interface agents [1]; Ding X. [1] Adaptive interfaces [1]; Adaptive user interface design [1]; Agents [1]; Autonomous agents [1]; University of Chile, Baloian, N. [3]; Universidad del Cauca, Colombia Computer aided instruction [2]; Santiago, Chile[5] Sanchez J. [2]; [1]; Computer technology [2]; Baytelman, F. [1]; Universidad Castilla-La Mancha, Human-computer interaction (HCI) [2]; Bravo, C. [1]; Spain [1] Knowledge Management [2]; Collazos, C. A. [1]; Problem Solving [2]; Guerrero, L. A. [1] Ad hoc networks [1]; Awareness [1]; blind people [1]; building designers [1]

Page 34 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

5.2.2 Decision Support Tools – Major Players Table 7. Decision Support tools – Organizations with 4 or more publications Organization Name Top Authors Co-Authoring Institutions Top subjects Naval Postgraduate Nissen, Mark E. [3]; Evidence Based Research Inc. [1]; Decision Making [6]; School, Monterey, CA, Gallup, Shelley P. [2]; Loyola College in Maryland, Collaboration [5]; USA[15] MacKinnon, Douglas J. [2]; Baltimore, MD, United States [1]; Situational awareness (SA) [5]; Zhao, Ying [2]; Parity Communications Inc. [1]; Information Exchange [4]; Zhou, Charles [2]; Quantum Intelligence Inc. [1]; Collaborative Techniques [3]; A. Bordetsky [1] US Army Engineer Research and Command and Control Systems [3]; Development Center (ERDC), Distributed [3]; Vicksburg, MS, USA [1] Experimental Laboratory for Investigating Information-sharing Collaboration and Trust (ELICIT) [3]; Knowledge Management [3]; Networks [3] BP Global, UK[13] Sawaryn, S. J. [5]; Delft University of Technology, Decision Making [9]; Lauche, K. [4]; Netherlands [4]; Drilling operations [5]; Bayerl, P. S. [2]; Accenture National Security real-time systems [5]; Thorogood, J. L. [2]; Services, Camden, NJ, USA [2]; Well drilling [5]; Badke-Schaub, P. [1]; Schlumberger Corp., [2]; Collaborative environments [4]; Branch, D. [1] CJSC Elvary Neftegaz [1]; Problem Solving [4]; ExxonMobil Corp., [1] real-time data [4]; Administrative data processing [3]; computer-supported cooperative work [3]; Data Acquisition [3]; data visualization [3]; Engines [3]; Fossil fuels [3]; information management [3]; Intelligent Energy [3]; Petroleum prospecting [3]; Work process [3] Nanyang Technological Kapur, M. [4]; Columbia University, New York, Problem Solving [5]; University, Singapore[10] Kinzer, C. K. [3]; NY, United States [3]; Algorithms [4]; Binh Ta, D. N. [2]; Hong Kong Baptist University, Client assignment [2]; Zhou, S. [2]; Kowloon, Hong Kong [1] computer-supported cooperative work Du H [1]; [2]; Jiao, R. J. [1] Decision Making [2]; Decision Support Systems (DSS) [2]; Distributed virtual environments [2]; Ill-structured problem solving [2]; Participation inequity [2]; Servers [2]; synchronous computer-supported collaborative learning CSCL [2]; Virtual Reality [2]; Well-structured problem solving [2]

Page 35 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-Authoring Institutions Top subjects Massachusetts Institute of Cummings, M. L. [2]; Boeing Company, USA [1]; Decision Making [3]; Technology (MIT), Pentland, A. [2]; Cisco Systems Inc [1]; Collaboration [2]; Cambridge, MA, USA[8] A. S. Clare [1]; Glaivestone Software [1]; Human engineering [2]; Bolia, Robert S. [1]; NASA Ames Research Center, Automated Planner [1]; Bran, C. A. [1]; Moffett Field, CA USA [1] Autonomous Navigation [1]; Burton, J. [1] Break down [1]; business competition [1]; Change management [1]; Command and control (C2) [1] US Air Force Research Bolia, Robert S. [3]; Evidence Based Research Inc. [2]; Command and control (C2) [2]; Laboratory (AFRL), Wright- Havig, Paul [2]; Boeing Company, USA [1]; Knowledge Management [2]; Patterson AFB, OH, USA[8] Leedom, D.K. [2]; Defence Science and Technology Network centric operations (NCO) Nelson, W. Todd [2]; Laboratory (DSTL), Farnborough, [2];adversary intent [1]; Aleva, Denise [1]; UK [1]; Air Battle Management [1]; Arnold, R. D. [1] DSO National Laboratories, anticipatory understanding [1]; Singapore [1]; battlefield visualization [1]; Human Performance Architects, battlespace [1]; Orlando, FL [1] cognitive issues [1]; Collaboration [1] Schlumberger Corp.,[7] Gomez, J. [2]; BP Global, UK [2]; Decision Making [5]; Gorgone, I. [2]; Shell [1]; Drilling optimization [3]; Uddenberg, G. [2]; Statoil ASA, Norway [1] real-time systems [3]; Brown, N. M. [1]; Communication systems [2]; Chatterjee, D. [1]; Drilling operations [2]; Fleury, S. G. [1] Emerging technologies [2]; Intelligent Energy [2]; Measurement-while-drilling [2]; Monitoring and control [2]; Operation support [2]; Optimization [2]; Problem Solving [2]; project management [2]; real-time data [2]; real-time decision making [2]; Rig operations [2]; Work process [2] Shanghai Jiao Tong Jinwei Cao [2]; Shanghai Key Lab. of Advanced Decision Making [3]; University, Shanghai, Zhang, P. [2]; Manufacturing Environment, group decision making [3]; China[7] Chu, Xuening [1]; China [1]; concept space [2]; Deng Yong [1]; University of Calgary, AB, Canada Customer satisfaction [2]; Fan F. -Y. [1]; [1]; Decision Support Systems (DSS) [2]; Fan Xiu min [1] University of Delaware, Newark, Fuzzy Logic [2]; DE, USA [1] fuzzy set theory [2]; group productivity [2]; Group Support Systems (GSS) [2]; Product development [2]

Page 36 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-Authoring Institutions Top subjects University of Patras, Rio Avouris, N. [5]; University of Freiburg, Germany Groupware [5]; Patras, Greece[7] Chounta, I.-A. [3]; [1] Problem Solving [4]; Kahrimanis, G. [3]; computer-supported collaborative Baltogiannis C [1]; learning (CSCL) [3]; Dimopoulos KG [1]; collaboration quality [2]; Economou G PK [1] Collaborative problem solving [2]; Computer Aided Instruction [2]; interaction analysis [2]; User interfaces [2]; Adaptation [1]; Analysis method [1] Lockheed-Martin Banasiak, M. [1]; General Electric (GE) Corp., USA Decision Support Systems (DSS) [3]; Corporation, United Czajkowski, M. [1]; [1]; air traffic control [2]; States[6] DiIenno, T. [1]; University of Colorado, Boulder, Automation [2]; Hofmann, M. O. [1]; CO, USA [1] Multi-agent systems (MAS) [2]; Iyer, N. [1]; Service oriented architecture (SOA) [2]; J. Roberts [1] Accident prevention [1]; Ad-hoc dynamic service composition [1]; Adaptive Systems [1]; Air navigation [1] Delft University of Lauche, K. [5]; BP Global, UK [4];CJSC Elvary Drilling operations [4]; Technology, Sawaryn, S. J. [4]; Neftegaz [1];People Factor Data transfer [3]; Netherlands[5] Bayerl, P. S. [2]; Consultant Ltd. [1];University of Decision Making [3]; Thorogood, J. L. [2]; Aberdeen, Scotland, UK [1] real-time data [3]; Badke-Schaub, P. [1]; Collaborative environments [2]; Crichton, M. [1] Human factors [2]; Offshore drilling [2]; Onshore Operation Centres (OOC) [2]; real-time systems [2]; Remote operations [2]; Sensory information [2]; Work process [2] Drexel University, Stahl, G. [3]; Decision Making [2]; Philadelphia, PA, United Dugan, C. [1]; group cognition [2]; States[5] Modi, P. J. [1]; Small groups [2]; Palisano, R. J. [1]; text chat [2]; Perit ÇakIr, M. [1]; Ad hoc networks [1]; Peysakhov, M. [1] Algorithms [1]; Bandwidth [1]; Hong Kong Polytechnic Choy, K. L. [2]; Decision Making [3]; University, Hong Kong[5] Chan, S. C. F. [1]; computer-supported cooperative work Kwong CK [1]; [2]; Kwong, C. K. [1]; Decision Support Systems (DSS) [2]; Lee BLP [1]; Internet [2]; Leung YK [1] mould manufacturing [2]; Process planning [2]; production scheduling [2]; Radio frequency identification (RFID) [2]; real-time systems [2]; Virtual Reality [2]

Page 37 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-Authoring Institutions Top subjects Accenture National Adkins, Mark [2]; BP Global, UK [2]; Decision Making [3]; Security Services, Camden, Kruse, John [2]; MITRE Corporation, Hampton, Administrative data processing [2]; NJ, USA[4] Branch, D. [1]; VA, USA [1] Situational awareness (SA) [2]; Castro, A. [1]; ad-hoc collaboration [1]; Fanty, S. [1]; Advanced collaborative environments G. Grosse [1] [1]; Agile Development Methodology [1]; Air Force Operations [1]; Asset management [1] Boeing Company, USA[4] Bolia, Robert S. [1]; Massachusetts Institute of Air Traffic [2]; Comitz, P. [1]; Technology (MIT), Cambridge, NEXTGEN [2]; Cummings, M. L. [1]; MA, USA [1]; Abstract model [1]; Graeber, David A. [1]; Raytheon Company, Advanced weather interactive Greenlaw, C. [1]; Marlborough, MA, USA [1]; processing systems [1]; Lee, M. E. M. J. [1] US Air Force Research Laboratory air traffic control [1]; (AFRL), Wright-Patterson AFB, Air Traffic Systems [1]; OH, USA [1] Air transportation systems [1]; George Mason University, Blackmond Laskey, Saab Corp. [1]; Web services [2]; Fairfax, VA, USA[4] Kathryn [2]; US Army Engineer Research and Bayesian networks [1]; Hieb, Michael R. [2]; Development Center (ERDC), C2 Grammar [1]; Adelman, Leonard [1]; Alexandria, VA, USA [1]; Coalition Collaboration [1]; Altenau, Michael [1]; Viecore FSD, Eatontown, NJ, USA Coalition Operations [1]; Braswell, Kenneth [1]; [1] Collaboration [1]; Chang, KC [1] collaboration support [1]; Collective Endeavors [1]; Command and control (C2) [1] Georgia Institute of Allen, J. K. [2]; Decision Making [4]; Technology, Atlanta, GA, Mistree, F. [2]; concurrent engineering [2]; United States[4] Abowd, G. D. [1]; game theory [2]; Fernandez MG [1]; Information Exchange [2]; Grinter, R. E. [1]; Army Training [1]; Hayes, G. R. [1] Battle Management [1]; Capture and access [1]; Collaboration [1]; collaborative CAD [1] Johns Hopkins University, Bressler, N. B. [1]; Ryerson University, Toronto, ON, bibliographic database [1]; Baltimore, MD, United Cantu, Osbaldo [1]; Canada [1]; cataracts [1]; States[4] Casparis, H. [1]; St. Michael's Hospital, Toronto, clinical decision making [1]; Cost, R. Scott [1]; Canada [1]; Collaboration [1]; Firestone, M. [1]; Jules Gonin Eye Hospital, Command and control (C2) [1] Holder, Robert [1] Lausanne, Switzerland [1]; University of Toronto, ON, Canada [1] MITRE Corporation, Beaton E [1]; Decision Making [3]; Bedford, MA, USA[4] Boiney L [1]; Visualization [2]; Bonaceto, Craig [1]; Architectures and design of Burns, Kevin [1]; collaborative systems [1]; Drury JL [1]; collaborative systems [1]; Duncan MO [1] Computer Networks [1]; Cost effectiveness [1]; crisis management teams [1]

Page 38 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-Authoring Institutions Top subjects NASA Ames Research Ambrosia VG [1]; Massachusetts Institute of Decision Support Systems (DSS) [3]; Center, Moffett Field, CA Bell, D. [1]; Technology (MIT), Cambridge, real-time systems [2]; USA[4] Brummett, R. [1]; MA, USA [1]; ad hoc support [1]; Cummings, M. L. [1]; aerospace robotics [1]; Gawdiak, Y. [1]; Asset management [1]; Gurram, M. [1] Automation systems [1]; autonomous rovers [1]; climatic impact [1]; collaborative decision systems project [1]; Commercial Off The Shelf (COTS) tools [1]

National Chiao Tung Chen Sheng Wang [2]; Kainan University, Taiwan [1]; Decision Making [4]; University, Hsin-Chu, Gwo Hshiung Tzeng [2]; National Taiwan Ocean business process execution language Taiwan[4] Min Jen Tsai [2]; University, Keelung, Taiwan [1]; [2]; Shih Chang Wang [2]; National Taiwan digital watermark [2]; Jih Jeng Huang [1]; University,Taipei, Taiwan [1] Enterprise computing [2]; Ming Shin Kuo [1] Filter banks [2]; fuzzy group decision making [2]; fuzzy set theory [2]; Fuzzy sets [2]; group decision making [2]; Problem Solving [2]; Web services [2] Northwestern Chang Z. Y. [1]; Shanghai Univ., China [1] Problem Solving [2]; Polytechnical University Fan Q. M. [1]; (e ,3e) process [1]; (NWPU), Xi'an, China[4] H. Xue [1]; active control tactics [1]; Li W. -J. [1]; Aircraft [1]; Liu H. G. [1]; Aircraft conceptual design [1]; Mingjun Xin [1] Analytic Hierarchy Process (AHP) [1]; Collaborative allocation [1]; collaborative management [1] Ohio State University, Bakshi B [1]; Yonsei University, Seoul, South Distributed work [2]; Columbus, OH, United Billings, C. E. [1]; Korea [1] air traffic management [1]; States[4] Fiksel J [1]; Analysis and synthesis [1]; Glassman, M. [1]; asset employment [1]; Grossman, J. B. [1]; Cognitive task analysis [1]; Kang, M. J. [1] Computer Networks [1]; computer-supported cooperative work [1]; cooperative learning [1] Saudi Aramco, Saudi Al Meshabi, O. O. [1]; Actenum Corporation [1]; Decision Making [3]; Arabia[4] Al-Harbi, W. [1]; SAS Institute, United States [1] Optimization [3]; Al-Mushirfi, O. [1]; Asset management [2]; Guzman, R. P. [1]; business process [2]; Husain, K. [1]; decision making process [2]; Irgens, M. [1] Engines [2]; Management [2]; Petroleum reservoir evaluation [2]; Reservoir management [2]

Page 39 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-Authoring Institutions Top subjects Stanford University, Fruchter, R. [2]; Decision Making [2]; Stanford, CA, USA[4] Bastea-Forte M. [1]; A/E/C global teamwork [1]; Bhutani, V. K. [1]; Artificial Intelligence [1]; Boraiah, M. [1]; artificial ventilation [1]; Ioannidou, D. [1]; Brainstorming [1]; Swaminathan, S. [1] business losses [1]; Civil engineering [1] Tongji University, Zhang Ming [2]; Decision Making [3]; Shanghai, China[4] Guofeng Qin [1]; real-time systems [3]; Qiyan Li [1]; Communication mechanism [2]; Sheng Yao [1]; emergency management system [2]; Wang Zhiqiang [1]; Geographic information systems (GIS) Wang, Z. Q. [1] [2]; information management [2]; safety [2]; urban rail transit system [2]; Workflow [2]; accident disposal [1] Tsinghua University, Chen, B. [1]; Beijing University of Posts and Concurrency control [2]; Beijing, China[4] Chen, G. [1]; Telecommunications, China [1]; Abnormal detection [1]; Feng Xiang [1]; East China University of Science Action plan [1]; Junfei Huang [1]; and Technology, Shanghai, China adaptation rules [1]; Ma, B. [1]; [1] adaptive performance testing [1]; Mao Ye [1] Adaptive testing [1]; Administrative data processing [1]; autonomous agents [1]; autonomous decision making [1] University of Calgary, AB, Carpendale, S. [1]; Shanghai Jiao Tong University, Analytic network process (ANP) [1]; Canada[4] Chu, Xuening [1]; Shanghai, China [1]; Buffer framework [1]; De Alwis, B. [1]; Sichuan University, Chengdu, business competition [1]; Geng, X. [1]; China [1]; Collaborative design environments [1]; Greenberg, S. [1]; University of Saskatchewan, Competitive strategy [1]; Gutwin, C. [1] Saskatoon, SK, Canada [1] Computer systems [1]; conceptual design [1]; concurrent engineering [1]; Critical parameter [1] University of California, Booher, D. E. California State University, stakeholders [2]; Berkeley, CA, United [1];Goldstein, N. C. Sacramento, USA [1];Lawrence Adaptive management [1]; States[4] [1];Innes, J. E. [1];Kearns, Livermore National Laboratory, Algorithms [1]; F. R. [1];L. El Ghaoui Livermore, CA, USA [1] apparel industry [1]; [1];M. I. Jordan [1] Collaborative governance [1]; Collaborative Techniques [1]; complex adaptive network [1]

Page 40 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-Authoring Institutions Top subjects University of Maryland Adler, R. F. [1]; Emory University, Atlanta, GA, Cooperative Behavior [2]; Baltimore County, Cooper, D. [1]; United States [2]; nomenclature [2]; Baltimore, MD, USA[4] Dutton, R. P. [1]; Center for Integration of Alarm systems [1]; Faraj, S. [1]; Medicine and Innovative anticoagulant therapy [1]; Hemphill III, J. C. [1]; Technology, Boston, MA, USA [1]; antithrombocytic agent [1]; Holcomb, J. B. [1] Englewood Hospital, Englewood, aprotinin [1]; NJ, United States [1]; Artifacts [1]; Yale-New Haven Hospital, New Audiovisual Aids [1]; Haven, CT, United States [1]; Duke University, Durham, NC, United States [1] University of Maryland, Croninger, R. G. [1]; University of Alabama, Acquisition [1]; College Park, MD, United Day, R. W. [1]; Birmingham, AL, United States Artifacts [1]; States[4] Faraj, S. [1]; [1]; Artificial Intelligence [1]; Mackenzie, C. F. [1]; University of Connecticut, Storrs, Assignment problems [1]; Moss, J. [1]; CT, United States [1]; Audiovisual Aids [1]; Raghavan, S. [1] University of Maryland Baltimore Bidding languages [1]; County, Baltimore, MD, USA [1] Briefing Charts [1]; Cautionary notes [1] University of Allessio, D. A. [1]; University of Oklahoma, Boyd, Decision Making [2]; Massachusetts, Amherst, Boit, R. J. [1]; OK, USA [1]; Adaptive Systems [1]; MA, United States[4] Brotzge, J. A. [1]; University of Akron, OH, United adaptive time adjusting algorithm [1]; D. Corkill [1]; States [1] Analysts [1]; Deschamps, A. D. [1]; Army Personnel [1]; Droegemeier, K. [1] atmosphere [1]; Brigade Combat Teams (BCTS) [1]; Chat [1]; Collaboration [1] University of Pittsburgh, Chang, H. [1]; Centers for Disease Control in Problem Solving [2]; PA, United States[4] Chang, K. C.-M. [1]; Taiwan, Department of Health, Analytic Hierarchy Process (AHP) [1]; Chiu, C.-H. [1]; Taiwan [1]; Benefits and costs [1]; Chou Jr., H. [1]; NTU Hospital, Taipei City, Taiwan biological warfare [1]; Chu, Y.-T. [1]; [1]; classification [1]; Claypool, E. [1] Google [1]; Collaboration [1]; National Health Institute of Collaborative approach [1]; Research, Taiwan [1] Collaborative Information Behavior [1]; Collaborative Techniques [1] University of Stavanger, Bratvold, R. B. [2]; Computas AS, Norway [2]; Decision Support Systems (DSS) [4]; Norway[4] Fjellheim, R. A. [2]; ConocoPhillips Norge AS, Norway Decision Theory [4]; Herbert, M. C. [2]; [2]; Integrated Operations [4]; Arild, Ø. [1]; University of Oslo, Norway [2]; Decision Making [3]; Bislo, R. [1]; Institute of Energy Technology, decision support [3]; Giese, M. [1] Norway [1]; Collaboration [2]; Norwegian University of Science Collaborative decision making (CDM) and Technology (NTNU), Norway [2]; [1] Decision modeling [2]; Drilling operations [2]; Influence diagram [2]; Offshore oil wells [2];

Page 41 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Top Authors Co-Authoring Institutions Top subjects University of Technology, Jie Lu [4]; Belgian Nuclear Research Centre Decision Making [2]; Sydney (UTS), NSW, Zhang, G. [4]; (SCK.CEN), Boeretang, Belgium Evaluation model [2]; Australia[4] Laes, E. [3]; [3]; fuzzy numbers [2]; Ruan, D. [3]; Brussels EU Chapter, Club of Group decision support systems (GDSS) Jun Ma [2]; Rome (CoR-EU), Belgium [1]; [2]; Meskens, G. [2]; Ecole Nationale Supérieure des Multi-criteria decision making (MCDM) Wu, F. [2] Arts et Industries Textiles, [2]; Roubaix, France [1]; Administrative data processing [1]; Flemish Institute for Artificial Intelligence [1]; Technological Research (VITO), Bionics [1] Boeretang , Belgium [1]; University of Leuven, Heverlee, Belgium [1]

Page 42 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

5.2.3 Canadian Laboratories Table 8 provides links to relevant Canadian HCI labs and academic groups researching decision support tools. This is list is selective rather than exhaustive and inclusion is based on numbers of publications or preceived importance. More complete details with top authors, co-authoring institutions, and top subject terms are provided in the attachment to this report: STI 7193 C3-MAAD Canadian Players.xls.

Table 8. Canadian Laboratories and Research Groups Organization Name Departments and Laboratories Notes

Carleton University, Ottawa, ON Human Oriented Technology Lab (HOT Lab) http://www2.carleton.ca/hot/

Concordia University, Montreal, Human Centered Software Engineering Group QC http://www.cs.concordia.ca/research/researchgroups/hcse/

Concordia Institute for Information Systems Engineering http://www.ciise.concordia.ca/

Ecole Polytechnique de Montreal, Dept. Génie informatique et génie logiciel See also Dr. Michel Demarais: Montreal, QC http://www.polymtl.ca/gigl/recherche/ http://www.professeurs.polymtl.ca/michel.desmarais/desm arais_michel_c-fr.html Groupe d'études et de recherche en analyse des décisions (GERAD) http://www.gerad.ca/fr/index.php

McGill University, Montreal, QC Multimodal Interaction Laboratory Dr. Raja Sengupta, research interests include Spatial Decision http://mil.mcgill.ca/ Support Systems and Agent-Based Modelling http://www.geog.mcgill.ca/faculty/sengupta/ Centre for Intelligent Machines http://www.cim.mcgill.ca/

Shared Reality Lab http://www.cim.mcgill.ca/sre/

Page 43 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Departments and Laboratories Notes

McMaster University, Hamilton, No particular department or lab for clinical decision support ON tools could be identified, but professor Mark Morreale lists it as one of his interests: http://fhs.mcmaster.ca/ceb/faculty_member_morreale.htm.

See also Dr. Milena Head: http://www.business.mcmaster.ca/IS/head/

National Research Council Canada, Human-Computer Interaction Program http://www.nrc- Institute for Information cnrc.gc.ca/eng/programs/iit/human-computer.html Technology, Fredricton, NB Learning and Collaborative Technologies http://www.nrc- cnrc.gc.ca/eng/programs/iit/collaborative-technologies.html

National Research Council Canada, Centre for Computer-assisted Construction Technologies London, ON http://www.nrc-cnrc.gc.ca/eng/ibp/irc/ccct/index.html Nipissing University, North Bay, Computer Science and Mathematics Nipissing is a mostly undergraduate university, however, Dr. ON http://www.nipissingu.ca/compsciandmath/ H. Zhu of the computer science department has published extensively on adaptive collaboration and collaborative tools: http://www.nipissingu.ca/faculty/haibinz/ Queens University, Kingston, ON Human Media Lab http://www.hml.queensu.ca/

Multimedia Coding and Communications Laboratory (Mc2L) http://www.ece.queensu.ca/Research/Labs/Mc2L/index.html

Ryerson University, Toronto, ON Centre for Interdisciplinary Human Factors Research Ryerson offers degrees and certificates in GIS Spatial http://www.ryerson.ca/cihfr/ Analysis, which is often related to decision support. http://www.ryerson.ca/geography/programs/ Department of Geography http://www.ryerson.ca/geography/

Page 44 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Departments and Laboratories Notes

Simon Fraser University, Burnaby, Social Cognition and Interactive Expertise in Natural and SFU Surrey houses several labs that may be of interest: BC Computational Environments (SCIENCE) Lab http://tirpitz.iat.sfu.ca/labs/index.php http://interaction-science.iat.sfu.ca/

Vision and Media Lab http://www.cs.sfu.ca/research/groups/VML/

School of Interactive Arts and Technology http://www.siat.sfu.ca/

Université de Montréal, QC Laboratoire de Recherche en Communication Multimédia (LRCM) http://www.com.umontreal.ca/recherche/lrcm.htm

Laboratoire des Usages et du Design des Technologies d’Information et de Communication (LUDTIC) no website found, contact: Carole Groleau et Lorna Heaton 514- 343-6111 poste 5446

Université Laval, Québec, QC Département des opérations et systèmes de décision Faculté des sciences de l'administration http://www4.fsa.ulaval.ca/cms/accueil/faculte/departementsec ole/osd University of Alberta, Edmonton, Advanced Man-Machine Interfaces (AMMI) Laboratory AB (Department of Computer Science) https://www.cs.ualberta.ca/research/research-areas/advanced- man-machine-interfaces

Page 45 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Departments and Laboratories Notes

University of British Columbia, Department of Electrical and Computer Engineering Most of the UBC articles are related to clinical decision Vancouver, BC https://137.82.61.1/ making tools or processes. Papers are often cross- departmental. Department of Anesthesiology, Pharmacology and Therapeutics http://www.apt.ubc.ca/UBC_Anesthesiology__Pharmacology_a MAGIC includes sub-specialities in HCI and visual analytics nd_Therapeutics.htm

Sauder School of Business http://www.sauder.ubc.ca/

MAGIC - Multimedia and Graphics Interdisciplinary Centre http://www.magic.ubc.ca/pmwiki.php?n=Main.HomePage

University of Calgary, AB Innovations in Visualization Laboratory Several laboratories under the Electrical and Computer http://innovis.cpsc.ucalgary.ca/GroupLabhttp://grouplab.cpsc.u Engineering department may also be of interest: calgary.ca/ Intelligent Software Systems Research Laboratory Visualization Research Laboratory Software Engineering Decision Support Laboratory Human-Computer Interaction Research Laboratory http://www.seng-decisionsupport.ucalgary.ca/

University of Guelph, ON Department of Computing and Information Science The CIS department lists several labs of interest: http://www.cis.uoguelph.ca/research.php Intelligent Decision Support Systems Laboratory (IDSS Lab) Interactive Cognitive Computing Laboratory (ICC Lab)

University of Manitoba, Winnipeg, Human-Computer Interaction Lab This lab may also be of interest, as some of the authors cited MB http://hci.cs.umanitoba.ca/Main/HomePage here come from it, though it is not entirely clear from their website if it is relevant: Autonomous Agents Laboratory http://aalab.cs.umanitoba.ca/

Page 46 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Departments and Laboratories Notes

University of New Brunswick, Faculty of Computer Science URLs for individual labs could not be found, however the Fredericton, NB http://www.unb.ca/fredericton/cs/about/research/index.html Faculty of Computer Science lists several that are probably of interest, including the HCI Lab and the Intelligent and Adaptive Systems Research Group. UNB is a frequent partner with the NRC-Institute for Information Technology in Fredricton. See separate entries for NRC.

University of Ottawa, ON Multimedia Communications Research Laboratory http://www.mcrlab.uottawa.ca/

Distributed & Collaborative Virtual Environments Research Laboratory (DISCOVER) http://www.discover.uottawa.ca/

University of Regina, SK Department of Computer Science For details on which faculty specialize in HCI, see this list: http://www.cs.uregina.ca/ http://www.cs.uregina.ca/Research/majorareas.html

University of Saskatchewan, Human-Computer Interaction Lab See also the page of Dr. Carl Gutwin, Canada Research Chair Saskatoon, SK http://hci.usask.ca/ in Next Generation Groupware Advanced Engineering Design Laboratory http://www.usask.ca/research/research_services/crc/profile http://www.engr.usask.ca/departments/mee/facilities/aedl.php s/gutwin.php

Page 47 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Departments and Laboratories Notes

University of Toronto, ON Department of Computer Science, Dynamic Graphics Project Dynamic Graphics Project (dgp) is an interdisciplinary http://www.dgp.toronto.edu/ research laboratory whose mission is advanced research and graduate instruction in human-computer interaction and Department of Health Policy, Management and Evaluation computer graphics. http://www.hpme.utoronto.ca/about/research.htm The Remote Sensing, Modelling and GIS laboratory has done Department of Geography and Program in Planning some work previously on augmented GIS mapping for http://www.geog.utoronto.ca/ decision making, it is not clear if this work is still ongoing. http://www.geog.utoronto.ca/resources/labs Cognitive Engineering Laboratory http://cel.mie.utoronto.ca/ The Autonomous Systems and Biomechatronics Lab does research in human-robot interaction for search and rescue and emergency services. http://asblab.mie.utoronto.ca/

University of Victoria, BC Visual Interaction Design (VisID) research group http://visid.cs.uvic.ca/index.html

Computer Human Interaction Software Engineering Lab (CHISEL) http://www.thechiselgroup.org/

University of Waterloo, ON Department of Systems Design Engineering (SYDE) SYDE is the most relevant department at Waterloo, http://www.systems.uwaterloo.ca/research/index.html particularly the Advanced Interface Design Lab: http://www.aidl.uwaterloo.ca/. Collaborative Systems Laboratory (under SYDE) http://collaborativesystemslab.igloocommunities.com/ Other areas of interest include the Intelligent Human Machine Systems Lab, and research programs in David R. Cheriton School of Computer Science optimization and decision making. http://www.cs.uwaterloo.ca/ Dr. Stacey Scott is the director of the Collaborative Systems Human Computer Interaction Laboratory Laboratory and has already worked with DRDC Atlantic and http://hci.uwaterloo.ca/ Gallium Visual Systems (Ottawa): http://www.eng.uwaterloo.ca/~s9scott/wiki/pmwiki.php

Page 48 of 53

DRDC Valcartier CR 2011-219 C3 MAAD Literature Survey May 2011

Organization Name Departments and Laboratories Notes

University of Western Ontario, Facility for Intelligent Decision Support Dr. Kamran Sedig has research interests in HCI: London, ON http://www.eng.uwo.ca/research/iclr/fids/default.htm http://www.csd.uwo.ca/People/sedig.shtml Human-Computer Interaction Lab http://hci.fims.uwo.ca/

Wilfrid Laurier University, Conflict Analysis Group, Department of Systems Design Check out their Decision Support System - GMCR II Waterloo, ON Engineering http://www.systems.uwaterloo.ca/Research/CAG/gmcrII.ht http://www.systems.uwaterloo.ca/Research/CAG/index.html ml

York University, Toronto, ON Center for Vision Research http://www.cvr.yorku.ca/home/

Elder Laboratory: Human and Computer Vision http://elderlab.yorku.ca/

Interactive Systems Research Group (ISRG) http://www.cse.yorku.ca/~isrg/

Page 49 of 53

DRDC Valcartier CR 2011-219 DOCUMENT CONTROL DATA (Security markings for the title, abstract and indexing annotation must be entered when the document is Classified or Designated) 1. ORIGINATOR (The name and address of the organization preparing the document. 2a. SECURITY MARKING Organizations for whom the document was prepared, e.g. Centre sponsoring a (Overall security marking of the document including contractor's report, or tasking agency, are entered in section 8.) special supplemental markings if applicable.) Tamara Keating and Tamara McLaughlin UNCLASSIFIED NRC Canada Institute for Scientific and Technical Information 1200 Montreal Rd., M-55 2b. CONTROLLED GOODS Ottawa, ON (NON-CONTROLLED GOODS) K1A 0R6 DMC A REVIEW: GCEC APRIL 2011

3. TITLE (The complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U) in parentheses after the title.) Command & Control Concepts for Maritime Area Air Defence (C3-MAAD) : Literature Survey

4. AUTHORS (last name, followed by initials – ranks, titles, etc. not to be used) Keating, T.K.; McLaughlin, T.M.

5. DATE OF PUBLICATION 6a. NO. OF PAGES 6b. NO. OF REFS (Month and year of publication of document.) (Total containing information, (Total cited in document.) including Annexes, Appendices, etc.) May 2011

7. DESCRIPTIVE NOTES (The category of the document, e.g. technical report, technical note or memorandum. If appropriate, enter the type of report, e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.) Contract Report

8. SPONSORING ACTIVITY (The name of the department project office or laboratory sponsoring the research and development – include address.) Defence Research and Development Canada – Valcartier 2459 Pie-XI Blvd North Quebec (Quebec) G3J 1X5 Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicable research 9b. CONTRACT NO. (If appropriate, the applicable number under and development project or grant number under which the document which the document was written.) was written. Please specify whether project or grant.) C3-MAAD-11bi SDA-07-001-010

10a. ORIGINATOR'S DOCUMENT NUMBER (The official document 10b. OTHER DOCUMENT NO(s). (Any other numbers which may be number by which the document is identified by the originating assigned this document either by the originator or by the sponsor.) activity. This number must be unique to this document.) STI 7193, DRDC SDA 07-001-010 DRDC Valcartier CR 2011-219

11. DOCUMENT AVAILABILITY (Any limitations on further dissemination of the document, other than those imposed by security classification.) Unlimited

12. DOCUMENT ANNOUNCEMENT (Any limitation to the bibliographic announcement of this document. This will normally correspond to the Document Availability (11). However, where further distribution (beyond the audience specified in (11) is possible, a wider announcement audience may be selected.)) Unlimited 13. ABSTRACT The following project was completed in two phases. The first phase sought to identify collaborative decision making tools currently on the market or being developed specifically for naval tactical C2 purposes. This was accomplished through a combination of database and Internet searches.

Only a few collaborative decision making tools specifically for naval tactical operations were identified, particularly the Cooperative Engagement Capability (CEC), the Multiplatform Engagement Capability (MPEC) and Thales’ Combat Management Systems. Only limited technical information on Thales’ and Raytheon’s systems was available but links to their product literature have been provided. Other projects and programmes not specifically for the Navy may be important to watch for new developments, namely: NATO’s Network Enabled Capability (NNEC) program, the US Air Force Network-Centric Collaborative Targeting (NCCT) and the US Joint Integrated Air and Missile Defense )JIAMD) initiative. Major players in this field include Raytheon, Thales. And Israel Aerospace Industries (IAI). Many of the development contracts in recent years have been awarded to consortia, often led by Raytheon, Thales or Lockheed 0artin. The second phase of this project was centered on identifying major organizations (academic, government and companies) currently conducting research in the areas of human-computer interaction and collaborative decision support tools, with a focus on Canadian organizations and laboratories. Information was sought from scientific and technical databases and some highly relevant conference proceedings. Bibliographic information was compiled into two master databases and then names of organizations were normalized and lists of the most prolific institutions in terms of numbers of publications were compiled.

/HSURMHWVXLYDQWDpWpPHQpHQGHX[SKDVHV'DQVOHFDGUHGHODSUHPLqUHO¶REMHFWLIFRQVLVWDLWjWURXYHU OHV RXWLOV FROODERUDWLIV GH SULVH GH GpFLVLRQ FRQoXV VSpFLDOHPHQW SRXU OH & WDFWLTXH QDYDO TXL VRQW DFWXHOOHPHQWGLVSRQLEOHVVXUOHPDUFKpRXHQFRXUVGHGpYHORSSHPHQW'DQVFHEXWGHVUHFKHUFKHVRQWpWp HIIHFWXpHVGDQVGHVEDVHVGHGRQQpHVHWGDQV,QWHUQHW

6HXOHPHQWTXHOTXHVRXWLOVFROODERUDWLIVGHSULVHGHGpFLVLRQSRXUOHVRSpUDWLRQVWDFWLTXHVQDYDOHVRQWpWp WURXYpVVRLWOD&RRSHUDWLYH(QJDJHPHQW&DSDELOLW\ FDSDFLWpG¶HQJDJHPHQWHQFRRSpUDWLRQRX&(& OD 0XOWLSODWIRUP(QJDJHPHQW&DSDELOLW\ FDSDFLWpG¶HQJDJHPHQWPXOWLSODWHIRUPHRX03(& HWOHVV\VWqPHV GHJHVWLRQGHFRPEDWGH7KDOHV6HXOVGHVUHQVHLJQHPHQWVWHFKQLTXHVOLPLWpVVXUOHVV\VWqPHVGH7KDOHVHW GH5D\WKHRQpWDLHQWGLVSRQLEOHVPDLVOHVOLHQVPHQDQWjODGRFXPHQWDWLRQVXUOHXUSURGXLWRQWpWpIRXUQLV ,OSRXUUDLWrWUHLPSRUWDQWGHVXUYHLOOHUWRXWQRXYHDXGpYHORSSHPHQWOLpjG¶DXWUHVSURMHWVHWSURJUDPPHV QRQVSpFLDOHPHQWFRQoXVSRXUOD0DULQHVRLWOHSURJUDPPHGHFDSDFLWpHQUpVHDXGHO¶27$1 11(& OD 1HWZRUN&HQWULF&ROODERUDWLYH7DUJHWLQJ FDSDFLWpGHFLEODJHFROODERUDWLIUpVHDXFHQWULTXHRX1&&7 GH ODIRUFHDpULHQQHDPpULFDLQHHWO¶LQLWLDWLYH-RLQW,QWHJUDWHG$LUDQG0LVVLOH'HIHQVH GpIHQVHDQWLDpULHQQH HWDQWLPLVVLOHLQWpJUpHFRQMRLQWHRX-,$0' GHVeWDWV8QLV'HVDFWHXUVPDMHXUVGDQVFHGRPDLQHVRQW 5D\WKHRQ 7KDOHV HW ,VUDHO $HURVSDFH ,QGXVWULHV ,$, HQWUH DXWUHV %RQ QRPEUH GHV FRQWUDWV GH GpYHORSSHPHQW GHV GHUQLqUHV DQQpHV RQW pWp FRQILpV j GHV FRQVRUWLXPV VRXYHQW GLULJpV SDU 5D\WKHRQ 7KDOHVRX/RFNKHHG0DUWLQ

/DGHX[LqPHSKDVHGHFHSURMHWYLVDLWHVVHQWLHOOHPHQWjLGHQWLILHUOHVJUDQGHVRUJDQLVDWLRQV XQLYHUVLWDLUHV JRXYHUQHPHQWDOHV HW SULYpHV PHQDQW DFWXHOOHPHQW GHV UHFKHUFKHV GDQV OHV GRPDLQHV GH O¶LQWHUDFWLRQ SHUVRQQHPDFKLQHHWGHVRXWLOVFROODERUDWLIVG¶DLGHjODGpFLVLRQSDUWLFXOLqUHPHQWOHVRUJDQLVDWLRQVHWOHV ODERUDWRLUHVFDQDGLHQV/¶LQIRUPDWLRQDpWpWLUpHGHEDVHVGHGRQQpHVVFLHQWLILTXHVHWWHFKQLTXHVHWGHV WUDYDX[ GH FHUWDLQHV FRQIpUHQFHV KDXWHPHQW SHUWLQHQWHV /HV UHQVHLJQHPHQWV ELEOLRJUDSKLTXHV RQW pWp UHJURXSpVGDQVGHX[EDVHVGHGRQQpHVSULQFLSDOHVSXLVOHVQRPVGHVRUJDQLVDWLRQVRQWpWpVWDQGDUGLVpVHW GHVOLVWHVGHVRUJDQLVDWLRQVOHVSOXVSUROLILTXHVTXDQWDXQRPEUHGHSXEOLFDWLRQVRQWpWpFRPSLOpHV

14. KEYWORDS, DESCRIPTORS or IDENTIFIERS

naval tactic decision support tools; human-computer interaction; decision support tools; decision making

Defence R&D Canada R & D pour la défense Canada

Canada's Leader in Defence Chef de file au Canada en matière and National Security de science et de technologie pour Science and Technology la défense et la sécurité nationale