Operational Requirements for Soldier-Robot Teaming

Home , Autonomous logistics, Loitering munition, Master corporal, Platoon (automobile)

CAN UNCLASSIFIED

Simon Banbury Kevin Heffner Hugh Liu Serge Pelletier Calian Ltd.

Prepared by: Calian Ltd. 770 Palladium Drive Ottawa, Canada K2V 1C8

Contractor Document Number: DND-1144.1.1-01 PSPC Contract Number: W7719-185397/001/TOR Technical Authority: Ming Hou, DRDC – Toronto Research Centre Contractor's date of publication: August 2020

The body of this CAN UNCLASSIFIED document does not contain the required security banners according to DND security standards. However, it must be treated as CAN UNCLASSIFIED and protected appropriately based on the terms and conditions specified on the covering page.

Defence Research and Development Canada Contract Report DRDC-RDDC-2020-C172 November 2020

CAN UNCLASSIFIED

IMPORTANT INFORMATIVE STATEMENTS

This document was reviewed for Controlled Goods by Defence Research and Development Canada using the Schedule to the Defence Production Act.

Disclaimer: This document is not published by the Editorial Office of Defence Research and Development Canada, an agency of the Department of National Defence of Canada but is to be catalogued in the Canadian Defence Information System (CANDIS), the national repository for Defence S&T documents. Her Majesty the Queen in Right of Canada (Department of National Defence) makes no representations or warranties, expressed or implied, of any kind whatsoever, and assumes no liability for the accuracy, reliability, completeness, currency or usefulness of any information, product, process or material included in this document. Nothing in this document should be interpreted as an endorsement for the specific use of any tool, technique or process examined in it. Any reliance on, or use of, any information, product, process or material included in this document is at the sole risk of the person so using it or relying on it. Canada does not assume any liability in respect of any damages or losses arising out of or in connection with the use of, or reliance on, any information, product, process or material included in this document.

Template in use: EO Publishing App for CR-EL Eng 2019-01-03-v1.dotm

CAN UNCLASSIFIED

Operational Requirements for Soldier- Robot Teaming

Task Authorization W7719-185397/001/TOR Human Autonomy Interaction Task #1

Calian Report DND-1144.1.1-01 Version 3.0 10 August 2020

Presented to Scientific Authority: Dr. Ming Hou Senior Defence Scientist Human Effectiveness Section

Defence Research and Development Canada – (DRDC) Toronto Research Centre 1133 Sheppard Ave. W Toronto, ON, M3K 2C9

Prepared by:

Calian Ltd. Engineering and Technical Services (ETS) 770 Palladium Drive Ottawa, Ontario Canada, K2V 1C8

Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

QUALITY ASSURANCE AND VERSION TRACKING

Authorization

Title Operational Requirements for Soldier-Robot Teaming

Document Number DND-1144.1.1-01

Ver. Developed By Reviewed By Approved By Date Dr. Simon Banbury Damon Gamble Dr. Kevin Heffner 1.0 Damon Gamble Gord Youngson 31-Mar-20 Dr. Hugh Liu (Corporate release) Mr. Serge Pelletier Dr. Simon Banbury Damon Gamble Dr. Kevin Heffner 2.0 Damon Gamble Gord Youngson 24-Jun-20 Dr. Hugh Liu (Corporate release) Mr. Serge Pelletier Dr. Simon Banbury Damon Gamble Dr. Kevin Heffner 3.0 Damon Gamble Gord Youngson 08-Jul-20 Dr. Hugh Liu (Corporate release) Mr. Serge Pelletier

© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2020 © Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2020

The scientific or technical validity of this Contract Report is entirely the responsibility of the Contractor and the contents do not necessarily have the approval or endorsement of the Department of National Defence of Canada.

Release Tracking

Ver. Action By Date

1.0 Release to Client 31-Mar-20

G. Youngson 2.0 Update based on feedback from the client.

24-Jun-20

G. Youngson

Page i Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.0 Finalized based on feedback from the client.

08-Jul-20

G. Youngson

Page ii Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

TABLE OF CONTENTS 1. INTRODUCTION ...... 5

1.1 BACKGROUND ...... 5 1.2 PURPOSE ...... 5 1.3 TECHNICAL APPROACH ...... 6 1.4 DOCUMENT STRUCTURE ...... 7 2. GENERAL DESCRIPTION OF OPERATIONAL CAPABILITY ...... 9

2.1 CAPABILITY GAP ...... 12 2.1.1 Introduction ...... 12 2.1.2 Methodology ...... 12 2.1.3 Results ...... 13 2.1.4 Summary...... 17 2.2 OVERALL MISSION AREA DESCRIPTION ...... 17 2.2.1 Dismounted Infantry Platoon Organisational Structure...... 17 2.2.2 Dismounted Infantry Platoon Generic Missions ...... 19 2.3 DESCRIPTION OF THE PROPOSED PRODUCT OR SYSTEM ...... 20 2.4 SUPPORTING ANALYSIS...... 22 2.4.1 Integrated Soldier System ...... 22 2.4.2 SRT Use Cases ...... 23 2.5 MISSIONS THAT THE PROPOSED SYSTEM WILL ACCOMPLISH ...... 27 2.5.1 Mule ...... 29 2.5.2 Wingman ...... 32 2.5.3 Artemis ...... 35 2.5.4 RoboMedic ...... 38 2.5.5 Amazon ...... 41 2.5.6 Summary...... 45 2.6 OPERATIONAL AND SUPPORT CONCEPT ...... 45 2.6.1 Concept of Operations ...... 45 2.6.2 Existing System Shortfalls ...... 46 2.6.3 Threat ...... 47 2.6.4 Support Concept ...... 48

3. CAPABILITIES REQUIRED ...... 49

3.1 OPERATIONAL PERFORMANCE PARAMETERS ...... 49 3.2 KEY PERFORMANCE PARAMETERS (KPPS) ...... 51 3.3 SYSTEM PERFORMANCE & EMPLOYMENT SCENARIOS ...... 51 3.3.1 Methodology ...... 52 3.3.2 Results ...... 53 3.3.3 Summary...... 80 3.4 OPERATIONAL REQUIREMENTS SUMMARY ...... 81 3.4.1 Command and Control Information System Requirements ...... 82 3.4.2 ISS Requirements ...... 83 3.4.3 Robotic System Requirements ...... 91

4. FUTURE WORK ...... 96

Page iii Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

4.1 NEXT STEPS FOR REQUIREMENTS DEVELOPMENT ...... 96 4.2 SYSTEM SUPPORT ...... 96 4.3 FORCE STRUCTURE FOR EXPECTED EMPLOYMENT ...... 97 4.4 SCHEDULE ...... 98 4.5 SYSTEM AFFORDABILITY ...... 98 5. CONCLUSIONS ...... 100 ANNEX A. INFORMATION REQUIREMENTS ...... 103 ANNEX B. ANALAYSIS ON TECHNICAL CHALLENGES OF SRT OPERATIONS ...... 106 ANNEX C. SRT HUMAN FACTORS ENGINEERING EXPERIMENTATION PLAN ...... 130

LIST OF FIGURES

Figure 1: Integrated Soldier System Overview w/Tactical User Interface (Tactical UI or TUI) ...... 10 Figure 2: Integrated Soldier System with Soldier Robot Teaming Capability ...... 11 Figure 3: Canadian Army Dismounted Infantry Platoon Structure and Personnel Composition (including NATO symbols) ...... 18 Figure 4: Generic Mission Profile of a Canadian Army Dismounted Infantry Platoon ...... 19 Figure 5: Future ISS System Overview ...... 20 Figure 6: Human Supervisory Control ...... 21 Figure 7: Contribution of each Stakeholder to the SRT Use Cases ...... 28 Figure 8: ISS Solder-Robot Teaming CONOPS ...... 46 Figure 9: Functional and Task Decomposition – ‘Mule’ ...... 54 Figure 10: Functional and Task Decomposition – ‘Artemis’ ...... 58 Figure 11: Functional and Task Decomposition – ‘Wingman’ ...... 63 Figure 12: Functional and Task Decomposition – ‘RoboMedic’ ...... 67 Figure 13: Functional and Task Decomposition – ‘Amazon’ ...... 74 Figure 14: Summary of SRT Features Identified from Use Case Task Analyses ...... 80 Figure 15: Summary of Platoon Level Unmanned Assets ...... 99 Figure 16: Overview of SRT HFE Evaluation Planning Process (including applicable sections of the report) ...... 138 Figure 17: Mapping SRT Use Cases against Generic Platoon Mission Profile ...... 142 Figure 18: Decision Tree for Selecting an Analysis Technique for Suitable for Identifying Cognitive Measures of Performance ...... 147 Figure 19: TIGER Facility – DRDC Toronto Research Centre ...... 165

LIST OF TABLES

Table 1: Summary of SRT Stakeholder Interviews ...... 13 Table 2: Identification of SRT Use Cases from Stakeholder Interviews ...... 28 Table 3: SRT Use Case Storyboard – ‘Mule’ ...... 29

Page iv Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 4: SRT Use Case Storyboard – ‘Wingman’ ...... 32 Table 5: SRT Use Case Storyboard – ‘Artemis’ ...... 35 Table 6: SRT Use Case Storyboard – ‘RoboMedic’ ...... 38 Table 7: SRT Use Case Storyboard – ‘Amazon’ ...... 41 Table 8: Task Analysis – ‘Mule’ ...... 55 Table 9: Task Analysis – ‘Artemis’ ...... 59 Table 10: Task Analysis – ‘Wingman’ ...... 64 Table 11: Task Analysis – ‘RoboMedic’ ...... 68 Table 12: Task Analysis – ‘Amazon’ ...... 75 Table 13: Command and Control Information System Features ...... 83 Table 14: ISS Combat Support Features ...... 83 Table 15: ISS Logistics Features ...... 84 Table 16: ISS Medical Support Features ...... 85 Table 17: ISS UVS Navigation Control Features ...... 87 Table 18: ISS Reporting Features ...... 87 Table 19: ISS Display Features ...... 89 Table 20: ISS Sensor Features ...... 91 Table 21: Robotic System Features...... 91 Table 22: Technical Development Supporting SRT Use Cases (X: Relevant, !: Critical) ...... 119 Table 23: Mapping SRT Use Cases against SRT HFE Experimental Themes...... 136 Table 24: Mapping SRT Use Cases against DRDC HSP Project Scenarios ...... 143 Table 25: Example MOEs and MOPs Relating to SRT Evaluations ...... 148 Table 26: Mapping SRT Measures of Effectiveness and Performance against SRT HFE Experimentation Themes ...... 164

Page v Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

EXECUTIVE SUMMARY

This work was conducted under Task 1 ‘Concept of Operations (CONOPS) for Manned- Unmanned Teaming and Human Factors Engineering and Experimentation Plan’ for Defence Research and Development Canada – Toronto Research Centre (DRDC TRC) by the Calian Team (including C3 Human Factors Consulting, Pegasus Research and Technologies, and the University of Toronto) as part of the larger Human Autonomy Interaction (HAI) Task Authorization Contract (TAC; Contract #: W7719-185397/001/TOR). The objective of HAI TAC is to design, develop, implement, and evaluate Soldier-Robot Teaming (SRT) concepts, methodologies, and technologies to improve overall Human-Machine System (HMS) performance for the Canadian Armed Forces (CAF).

The concept of SRT has been identified by the CAF as a way of improving mission effectiveness by leveraging enabling technologies such as robotics and autonomous systems. To support this endeavour, DRDC TRC will conduct a series of Human Factors Engineering (HFE) and Experimentation activities in the next three years. The following work items were conducted and reported in this document:

1. SRT Stakeholder Analysis. The first step was to engage with key stakeholders within the Department of National Defence (DND) and DRDC to identify future SRT concepts for further development under the auspices of the HAI TAC. The project team met with several DND and DRDC stakeholders to identify future SRT concepts for the CAF across a broad range of missions and operational environments. 2. SRT Use Case Development. Based on the results of the stakeholder analysis, (as well as a brief review of state-of-the-art SRT concepts from allied partners, industry, and academic networks), five storyboard use cases were developed to capture the intended Concept of Operation, SRT interactions, operational contexts, and expected mission performance. 3. SRT Technical Challenges Review. A review of the state-of-the-art of SRT concepts and best practices from industry, academia, and government organizations was conducted to reflect the technical challenges of integrating manned and autonomous systems. 4. Preliminary SRT Mission and Task Analysis. Based on the SRT use cases identified by the stakeholders, a preliminary Mission Function Task Analysis was conducted to identify key SRT capability ‘features’ that would need to be developed to support the envisioned improvements to operational effectiveness. In turn, SRT operational requirements were then developed to inform future DRDC/DND research and development activities. 5. SRT HFE Experimentation Plan. Based on the four work items, an HFE experimentation plan to guide future SRT experimentation conducted under the auspices of the HAI TAC. The work items described above are consistent with the Concept Development and Experimentation (CD&E) approach to developing military capabilities to fill a capability gap where the focus is on defining and validating hypotheses through analyses and trials. As per the CD&E methodology, the stakeholder engagement activities included interviews and workshops

Page vi Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

which provided the data that subsequent analyses were based (e.g., use case analysis, task analysis, and requirements analysis).

Toward the goal of creating an overarching document that includes the five work items described above, this report follows the United States (US) Department of Homeland Security template for an Operational Requirements Document (ORD). The ORD template is typically used for capturing the description of an operational capability associated with Capstone projects such as the CAF Close Engagement Operating Concept and related Adaptive Dispersed Operations Capstone Concept. The ORD template provides guidance for identifying the capability gap that the concept is supposed to address and organizing the supporting analysis in the operational context of the missions for which the concept or system is intended to operate, including whether the system is intended as a countermeasure to a threat. In addition to a description of the CONOPS, the template provides guidance for describing the shortfalls of existing systems or concepts. Finally, the ORD provides a structure for presenting the operational requirements and features associated with the required capabilities. The template addresses system support issues, the force structure, as well as scheduling and cost constraints.

Although all sections of the template were not completed for the purposes of the current study, it provided a useful format to organize and structure the work items produced in this study. The report concludes by identifying areas of future work including follow-on studies that will provide information that could be added to sections of the template that were not completed in this study. For example, a more detailed set of requirements could provide the basis for determining system support needs in terms of maintenance, supply, equipment support, and training. In addition, further details will provide the basis for addressing system affordability and schedule constraints.

Page vii Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

LIST OF ACRONYMS

ACA Artificial Cognitive Agent ACU Artificial Cognitive Units ADO Adaptive Dispersed Operations AFRL Air Force Research Laboratory AI Artificial Intelligence AOO Area of Operations AOR Area of Responsibility AR Anticipation Ratio ATTRACTOR Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability BARS Behaviorally Anchored Rating Scales BDA Battle Damage Assessment BG Battle Group BMS Battle Management System BN Bayesian Network C2 Command and Control C2IS Command and Control Information Systems CA Canadian Army CAF Canadian Armed Forces CALWC Canadian Army Land Warfare Centre CARS Crew Awareness Rating Scale CASEVAC Casualty Evacuation CAST Coordinated Awareness of Situation by Teams CBRN Chemical, Biological, Radiological, Nuclear CBP Complex Battle Position CD&E Concept Development & Experimentation CE Close Engagement CGF Computer Generated Forces CNT Cognitive Network Tracing COGMON Cognition Monitor CONOPS Concept of Operations CSAR Combat Search and Rescue CSS Combat Service Support DARPA Defense Advanced Research Projects Agency DLR Directorate of Land Requirements DMCA Dual-mode Cognitive Automation DND Department of National Defence DRA Defence Research Agency DRAWS DRA Workload Scales DRDC Defence Research and Development Canada EMAV Expeditionary Modular Autonomous Vehicle ETA Estimated Time of Arrival

Page 1 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

EW Electronic Warfare FLOE Future Land Operating Environment FOB Forward Operating Base FVEYS Five Eyes GNC Guidance Navigation and Control GPMG General Purpose Machine Gun HAI Human Autonomy Interaction HAT Human Autonomy Teaming HFE Human Factors Engineering HITL Human In The Loop HOTL Human On the Loop HMI Human Machine Interface HMS Human Machine System HQ Headquarters HSC Human Supervisory Control IDEaS Innovation for Defence Excellence and Security ISR Intelligence Surveillance and Reconnaissance ISS Integrated Soldier System IUI Intelligent User Interfaces IWS Integrated Workload Scale JIMP Joint Interagency Multinational and Public KPP Key Performance Parameters LCSS Land Command Support System LMG Light Machine Gun LSA Local Situation Awareness MALE Medium Altitude Long Endurance MEDEVAC Medical Evacuation MFTA Mission Function and Task Analysis MGRS Military Grid Reference System MIPLA Mixed Initiative Planning Assistant MISTAT Mechanism Injury Symptoms Treatment Age Time MOE Measure of Effectiveness MOP Measure of Performance MR Mission Route MUM-T Manned Unmanned Teaming MULE Multifunction Utility / Logistics and Equipment NBC Nuclear, Biological, Chemical OPFOR Opposing Force OPP Operational Performance Parameters ORD Operational Requirements Document PAN Personal Area Network PIC Pilot In Command PM Perception Management

Page 2 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

POCA Perception-Oriented Cooperation Agent QUASA Quantitative Analysis of Situational Awareness RCAF Royal Canadian Air Force RCIC Royal Canadian Infantry Corps RCAMC Royal Canadian Army Medical Corps RCN Royal Canadian Navy RCR Royal Canadian Regiment RDV Rendez-Vous RECCE Reconnaissance REORG Reorganization RMS Root Mean Square S&R Sense and Respond SA Situation Awareness SAGAT Situation Awareness Global Assessment Technique SAR Search and Rescue SART Situation Awareness Rating Technique SASHA Situation Awareness for SHAPE SAW-IM Situation Awareness and Workload In-Flight Measure SDT Signal Detection Theory SE Systems Engineering SHAPE Solutions for Human Automation Partnerships in European Air Traffic Management SITREP Situation Report SM Sensor Management SME Subject Matter Expert SMEP Squad Multipurpose Equipment Transport SNA Social Network Analysis SOF Show of Force SOW Statement of Work SPAM Situation Present Assessment Method SPMS Sensor and Perception Management System SRT Soldier Robot Teaming SSA Shared Situation Awareness SSTRM Soldier System Technology Roadmap SUS System Usability Scale SWaP Size Weight and Power SWAT Subjective Workload Assessment Technique SWORD Subjective Workload Dominance TA Target Acquisition TAC Task Authorization Contract TE Target Engagement TIGER Testbed for Integrated Ground Control Station Experimentation and Rehearsal TRC Toronto Research Centre

Page 3 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

TLX Task Load Index TMS Transactive Memory System TUI Tactical User Interface UAS Unmanned Aircraft System UAV Unmanned Aerial Vehicle UCAV Unmanned Combat Aerial Vehicle UGV Unmanned Ground Vehicle US United States USE Usefulness, Satisfaction, and Ease of Use USV Unmanned Surface Vehicle UTIAS University of Toronto Institute for Aerospace Studies UUV Unmanned Underwater Vehicle UVS Unmanned Vehicle System VPA Verbal Protocol Analysis W/INDEX Workload Index

Page 4 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

1. INTRODUCTION

This work was conducted under Task 1 ‘Concept of Operations (CONOPS) for Manned- Unmanned Teaming and Human Factors Engineering and Experimentation Plan’ for Defence Research and Development Canada – Toronto Research Centre (DRDC TRC) by Calian, C3 Human Factors Consulting, Pegasus Research and Technologies, and the University of Toronto as part of the larger Human Autonomy Interaction (HAI) Task Authorization Contract (TAC; Contract #: W7719-185397/001/TOR). The objective of HAI TAC is to design, develop, implement, and to evaluate Soldier-Robot Teaming (SRT) concepts, methodologies, and technologies to improve overall Human-Machine System (HMS) performance for the Canadian Armed Forces (CAF).

1.1 Background

The concept of SRT has been identified by the CAF as a way of improving mission effectiveness by leveraging enabling technologies such as robotics and autonomous systems (Hou et al, 2015a; Choi, 2019). For these technologies to be successfully deployed in the future, several research questions will need to be addressed by the HAI TAC:

1. What CAF land operations would most benefit from the deployment of SRT concepts, and what improvements to mission performance should be expected? 2. What are the state-of-the-art enabling technologies (e.g., autonomous systems) that can deliver the future capabilities needed to improve mission performance? 3. What are the critical roles and responsibilities of the soldier and autonomous system for a successful SRT partnership? 4. What are the critical organizational factors (e.g., level, size, structures) that underpin effective SRT partnerships? 5. What is the right mix of autonomous assets at the section, platoon, and company levels? Does the asset mix this scale up to teams of platforms (or ‘swarms’)? 6. How can SRT be effectively integrated within the soldiers’ tasks and operations without burdening them physically and cognitively?

1.2 Purpose

To answer these questions and to understand the needs of the Canadian Army (CA), DRDC TRC will conduct a series of Human Factors Engineering (HFE) and Experimentation activities in the next three years. The following work items were conducted and reported in this document:

Page 5 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

several DND and DRDC stakeholders to identify future SRT concepts for the CA across a broad range of missions and operational environments. 2. SRT Use Case Development. Based on the results of the stakeholder analysis, as well as a brief review of state-of-the-art SRT concepts from Five Eyes (FVEYS) allied partners, industry, and academic networks, five storyboard use cases were developed to capture the intended CONOPS, SRT interactions, operational contexts, and expected mission performance. 3. SRT Technical Challenges Review. A review of the state-of-the-art of SRT concepts and best practices from industry, academia, and government organizations was conducted to reflect the technical challenges of integrating manned and autonomous systems. 4. Preliminary SRT Mission and Task Analysis. Based on the SRT use cases identified by the stakeholders, a preliminary Mission Function Task Analysis (MFTA) was conducted to identify key SRT capability ‘features’ that would need to be developed to support the envisioned improvements to operational effectiveness. In turn, SRT operational requirements were then developed to inform future DRDC/DND research and development (R&D) activities. 5. SRT HFE Experimentation Plan. Based on the four work items, an HFE experimentation plan to guide future SRT experimentation was conducted under the auspices of the HAI TAC.

1.3 Technical Approach

The work items described above are consistent with the Concept Development and Experimentation (CD&E) approach to developing military capabilities to fill a capability gap where the focus is on defining and validating hypotheses through analyses and trials (Labbe et al, 2006). As per the CD&E methodology, the stakeholder engagement activities included interviews and workshops which provided the data that subsequent analyses were based (e.g., use case analysis, task analysis, and requirements analysis). When operational concepts are validated and sufficiently vetted, it is possible to employ a Systems Engineering (SE) approach to capability development that is focused more on requirements management and system development (MITRE, 2014). The use of the CD&E approach is described further in Heffner (2020), which considers the overall capabilities based planning context that places CD&E in a SE framework that also includes operational requirements development as part of an interaction- centered design methodology.

Toward the goal of creating an overarching document that includes the five work items described in section 1.2, this report follows the United States Department of Homeland Security template for an Operational Requirements Document (ORD)1. The ORD template is typically used for capturing the description of an operational capability associated with Capstone projects such as the CAF Close Engagement Operating Concept and related Adaptive Dispersed

1 https://www.dhs.gov/xlibrary/assets/ORD_Template.pdf

Page 6 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Operations Capstone Concept (DND, 2011a; Aral, 2019). The ORD template provides guidance for identifying the capability gap that the concept is supposed to address and organizing the supporting analysis in the operational context of the missions for which the concept or system is intended to operate, including whether the system is intended as a countermeasure to a threat. In addition to a description of the CONOPS, the template provides guidance for describing the shortfalls of existing systems or concepts. Finally, the ORD provides a structure for presenting the operational requirements and features associated with the required capabilities. The template addresses system support issues, the force structure, as well as scheduling and cost constraints.

Although all sections of the template were not completed for the purposes of the current study, it provided a useful format to organize and structure the work items produced in this study. Section 4 identifies areas of future work and this includes performing follow-on studies that will provide information that could be added to sections of the template that were not completed in this study. For example, a more detailed set of requirements could provide the basis for determining system support needs in terms of maintenance, supply, equipment support, and training. In addition, additional details will provide the basis for addressing system affordability and schedule constraints.

1.4 Document Structure

The document is structured as follows:

1. Section 1 – Introduction. This section provides an overview of work conducted to support the identification of SRT concepts for the CA and the scope of purpose of the work presented in this document. 2. Section 2 – General Description of Operational Capability. This section is the first of two parts of the Operational Requirements documentation. It describes the capability gap of which the future system is intended to fulfill, a description of the overall CA SRT mission, a description of the proposed future system, and a description of the mission use cases that the proposed future system will accomplish 3. Section 3 – Capabilities Required. This section is the second of two parts of the Operational Requirements documentation. It describes the operational performance and key performance parameters of the future system, system performance and employment scenarios, and a summary of the operational requirements identified from the mission use cases. 4. Section 4 – Future Work. This section describes the future work required to complete the Operational Requirements documentation. 5. Section 5 – Conclusions. This section presents the conclusions of the study. 6. Section 6 – References. This section lists the references cited by the report. 7. Annex A – Information Requirements. This annex presents the information requirements

Page 7 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

pertaining to CA reporting. 8. Annex B – Analysis of Technical Challenges of SRT Operations. This annex presents the results of a literature on the technical challenges of implementing SRT technologies within CA operations. 9. Annex C – SRT HFE Experimentation Plan. This annex presents the SRT HFE Experimentation Plan.

Page 8 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2. GENERAL DESCRIPTION OF OPERATIONAL CAPABILITY

In 2011, the government of Canada published the Soldier System Technology Roadmap (SSTRM), that included an action plan that identified the need to develop technologies required to satisfy requirements for soldier system capability areas such as: survivability, sustainability, mobility, lethality and C4I (Command, Control, Communication, Computer and Intelligence) (DND, 2011a). At the time of the publication of the SSTRM, there was much focus on developing systems with low requirements for Size, Weight and Power (SWaP), i.e. electronic devices that were small and light with low energy consumptions combined with lightweight, compact energy storage solutions. Toward the goal of minimizing physical burden, load-carrying technologies were identified, including dermoskeletons, exoskeletons and robotic multifunction utility/logistics and equipment ‘mule’ platforms. One major focus area of the SSTRM was that of human systems integration, including the consideration of technical domain specific drivers, including:

• Human-System interface integration (input design, visual displays and auditory outputs); • Dependency on information; • The concept of a Personal Area Network (PAN); and • Interoperability between soldier systems and other systems, (e.g. vehicles and weapons).

The SSTRM identified the main strength of using load-carrying technologies as increasing the soldier’s autonomy while gaining vantage points without being exposed to typical risks, while the weakness was identified as the soldier’s autonomy and responsiveness became dependent on system reliability. The critical barriers that were identified were:

• Ensuring that robotic mules were able to traverse rough terrain; • Increasing the level of autonomy of existing platforms; and • Reducing the system noise.

At the time of the publication of the SSTRM, the priority of robotic mules (2) was lower than the priority on developing low SWaP devices (1), out of a scale of 1 (high) to 3 (low). With the increasing maturity of artificial intelligence and robotics technologies, there has been an increased interest in recent years on developing SRT capabilities to satisfy the requirements needed for SSTRM capabilities.

In 2019, the Canadian Army Land Warfare Centre (CALWC) published the Canadian Land Operations Capstone Operating Concept for Close Engagement (CE); a conceptual model that dictates how the CA should be configured, equipped and trained, from about 2030 onward (Aral, 2019). CE is “the ability to conduct both lethal and non-lethal activities at the tactical level to

Page 9 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

create effects that have influence across the physical, moral and cognitive planes within the operating environment” (Aral, 2019).

CE may involve one or more of the following operational functions:

• Act. Act is the operational function that integrates maneuver, firepower and information operations to achieve the desired effects. • Sense. Sense is the operational function that provides the commander with the knowledge needed to understand the environment. This function incorporates all capabilities that collect and process data. • Shield. Shield is the operational function that protects a force, its capabilities and its freedom of action. • Sustain. Sustain is the operational function that regenerates and maintains capabilities in support of operations. • Command. “Command is the operational function that thrives through the use of information that comes from previous operational functions. It integrates all the operational functions into a single, comprehensive strategic, operational or tactical-level concept.” (Aral, 2019)

Figure 1: Integrated Soldier System Overview w/Tactical User Interface (Tactical UI or TUI)

Page 10 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

To perform these functions, soldiers must maintain Situation Awareness (SA) that includes both individual SA and Shared SA (SSA)2. Soldiers, crews and team members are required to communicate to coordinate task execution within their area of operations (AOO), including performing localized battle management and other tasks, such as reporting to their superiors or requesting support from headquarters (HQ). The ISS concept is encapsulated in the set of subsystems and functions shown in Figure 1.

The ISS is intended to support the soldier fundamental needs to maintain SA, share information and exercise task planning and task execution functions in the execution of tasks underlying the above-stated operational functions. The ISS can be broken down into five sets of components:

• The Tactical User Interface (TUI); • The ruggedized laptop computer; • Sensors; • Communications equipment; and • Power and data hubs.

Figure 2: Integrated Soldier System with Soldier Robot Teaming Capability

2 Individual SA refers to a soldier, vehicle operator or other crew member’s knowledge of the operating environment and surroundings in which they are performing tasks. Shared SA awareness (SSA) refers to the degree to which the crew or team possess the same SA (see Endsley, 1995).

Page 11 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

The ISS as described above has already been developed and includes fully functional components that are currently being deployed to CAF units. However, the ISS is still the subject of research efforts and additional functionality, improvements and extensions that can be expected in the short- and medium-term. The current report addresses one such area: the integration of robotics asset control as an integral component of the ISS, as represented in Figure 2.

2.1 Capability Gap

This section describes the stakeholder analysis that was conducted to identify future SRT concepts based on the perceived capability gaps from current solider systems. The capability gaps described in this section provide the rationale for the research and development of a range of SRT systems to support future acquisition and procurement. 2.1.1 Introduction The objective is to identify future SRT concepts for further development under the auspices of the HAI TAC. The project team met with several DND and DRDC stakeholders between December 2019 and February 2020 to identify future SRT concepts for the CA across a broad range of missions and operational environments. 2.1.2 Methodology The stakeholder interviews were relatively unstructured to allow the DND interviewees to express their ideas in an informal and non-judgemental setting. Following a brief introduction of the project, the remainder of the interview was structured to support the capture of the following information:

• What is the operational need (i.e., capability gap) that is to be addressed by SRT? • Who is / are the intended target operator(s)? • What SRT technologies are being considered to support this operational need? All discussions were focused on ‘what tasks need to be supported’ rather than on how existing technology capabilities might be deployed. The intent was to convey that SRT concept brainstorming should not be limited by the capabilities of current technology.

The following stakeholder communities were interviewed by the project team:

1. 24th Field Ambulance, Royal Canadian Army Medical Corps (RCAMC); 2. Canadian Army Land Warfare Center (CALWC); 3. Director Land Requirements (DLR) – 2 and 5; 4. Royal Canadian Infantry Corps (RCIC); and 5. Royal Canadian Regiment (RCR).

Page 12 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.1.3 Results The results of the stakeholder interviews are presented in Table 1. It summarises the SRT concepts that were discussed by each interviewee.

Table 1: Summary of SRT Stakeholder Interviews

Date Stakeholder SRT Concepts Discussed Soldier Burden Reduction - Dismounted infantry – backpack, equipment other than ‘fighting kit’, additional ammo, water, rations, batteries. - Pioneer – ladders, explosives, combat engineer specific equipment. - Direct Fire Support (e.g., 50 calibre machine gun, C16 automatic grenade launcher, anti-tank system, mortars) – manually fired with no autonomous capability.

Sensing Environment - Individual sensor to improve individual infantryman Capt. situation awareness. 6/12/2019 Beauchamp - Virtual tripwire using unattended ground sensor or (RCIC) perching UAV sensor. - Micro, lightweight, and hand deployable.

Logistics & Resupply - ‘Just in time’ re-supply of ammo, water, food, batteries. - Sense and Respond (S&R) Logistics - Re-supply via air or ground

Casualty Evacuation (CASEVAC3) - Evacuation of combat casualties from front-line to CASEVAC location

3 Also referred by NATO Allies as Medical Evacuation (MEDEVAC). The terms MEDEVAC and CASEVAC are used inter-changeably in this report.

Page 13 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Date Stakeholder SRT Concepts Discussed Soldier Burden Reduction - Combat medical equipment

Logistics & Resupply Sgt. - Resupply of medical expendables (bandages etc.). 6/12/2019 Lavoie - Re-supply via air or ground. (RCAMC) Casualty Evacuation (CASEVAC) - Evacuation of combat casualties from front-line to CASEVAC location. - Remote monitoring and treatment during evacuation. Soldier Burden Reduction - Direct Fire Support (e.g., C16 automatic grenade launcher) – manually fired (no autonomous capability)

Semi-Autonomous Weapon Platform - Direct Fire Support (e.g., C16 automatic grenade launcher) and semi-autonomously engages targets according to fire plan (operator in the loop / not fully autonomous). - Dropped into battlefield and seeks / destroy target autonomously (operator in the loop / not fully autonomous). - Counter-battery fire – senses incoming rounds and counter-fires (e.g., 81 mm mortar) (operator in the loop / L.Col. not fully-autonomous). 12/12/2019 Corby - Loitering munition launched into area and seeks targets (DLR-5) for engagement (operator in the loop / not fully autonomous).

Sensing Environment - Individual sensor to improve individual infantryman situation awareness. - Ground movement and low-hovering capability so hard to track. - Provides real-time reporting (including imagery and Electronic Warfare; EW). - Deception capability – EW / vibration to emulate other assets - Chemical Biological Radiation and Nuclear (CBRN) detection and warning.

Page 14 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Date Stakeholder SRT Concepts Discussed Logistics & Resupply - Optimised loading of supply containers based on actual needs and unloading requirements. - Information sharing within CAF and between coalition L.Col. partners to identify optimal location of equipment, spare parts, and supplies. Chayer & - Supply demands based on models of predicted usage 18/12/2019 Maj. and/or sensors on soldier/equipment/vehicles to track Champagne expendables (ammo, batteries, spare parts, fuel, food, (CALWC) water). - Container can be dropped at specific location in advance and used as a shelter when supplies are unloaded. - Autonomous convoys of logistic vehicles to reduce need for human drivers. Sensing Environment Maj. - Self-organising swarm of UAV sensors to improve sensor Han & coverage of large area. 9/12/2019 Lt(N). - Minimal operator involvement other than to task swarm to Nelson a specific location to perform a specific function. (DLR-2) - Real-time reporting including imagery. W.O. Sensing Environment 10/02/2020 Evong - Security overwatch for resupply convoys, patrol base, (RCR) section movement, and medical evacuation.

Feedback from DLR indicated that the CA is interested in both enabling and offloading soldiers of their physical burdens by enhancing firepower (i.e., lethality) and sustainment of the force, more specifically light and dismounted forces. However, no current technology is being leveraged in this effort despite the mention of this technology in the CA’s capstone operating concept ‘Close Engagement’ (Aral, 2019).

DLR-5’s current interest in SRT technologies has been inspired by Rheinmetall’s Mission Master platform, which has been integrated with their Argus ISS, as well as the United States Marine Corp’s (USMC) Expeditionary Modular Autonomous Vehicle (EMAV), and the US Army’s Squad Multipurpose Equipment Transport (SMEP).

Furthermore, any SRT concepts identified and developed by the HAI TAC must support the following tenets of the CE concept:

1. The CA must be a modular force, have adaptable equipment, versatile personnel, and must deploy in a scalable and self-sustainable force package. 2. The CA must be capable of operating in a dispersed manner, with the ability to provide

Page 15 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

security over a wide area, while lessoning the impact of its presence on local vulnerable populations. 3. Characteristics of the CA must reflect agility, connectivity, access to lethal and non-lethal effects, modularity, and be capable of adaptive dispersion with rapid aggregation and robustness.

Finally, policy surrounding SRT technologies must be carefully considered during system design and implementation. Currently, policies regarding autonomous systems are nascent in the DND and CAF. In addition, equipment projects can be stopped at the ministerial level if the associated risk with the delivered system is too high. As a result, it is important that DLR ensures that all the necessary scientific research is done to both inform system requirements and assure the Government of Canada that procured systems can operate in a safe/low risk manner. For example, it is important to know the risks associated with how future UAV swarms will operate, what safeguards need to be specified, and what is a reasonable way to deploy and operate a swarm of UAVs.

Page 16 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.1.4 Summary Five SRT concept themes emerged from the stakeholder interviews based on the perceived capability gaps of current soldier systems:

1. Reduction of dismounted soldier burden (including Combat Medic); 2. Evacuation of battlefield casualties; 3. Semi-autonomous weapon platform to provide offensive capabilities; 4. Sensing the local environment to increase the soldier’s individual SA and SSA; and, 5. ‘Just in time’ S&R logistics and resupply to dispersed dismounted infantry units.

The stakeholder interviews identified a wealth of information pertaining to future SRT concepts within the CA. The following observations were made by the project team:

1. The stakeholders had consistent and complementary views regarding current capability gaps identified above and how future SRT concepts could mitigate them. 2. The ISS should be the primary means of interaction with any autonomous system. In addition, the interaction should be as simple as possible, create little additional mental workload, and require minimal training. 3. System autonomy, especially with regards to an autonomous weapon platform, should still require ‘human on the loop’ (HOTL) involvement. For example, the operator would retain the authority to fire on a target. 4. Industry involvement in developing and evaluating future SRT concepts is already well underway (e.g., integration of Rheinmetall’s Mission Master platform with the Argus ISS).

2.2 Overall Mission Area Description

Nearly all the SRT concepts identified by the stakeholders focused on the CA dismounted infantry platoon. This section describes the overall mission area to which the capability gap pertains, including its users and its scope; specifically, a brief overview of a dismounted infantry platoon in terms of its personnel, organisational structure, and mission profile. 2.2.1 Dismounted Infantry Platoon Organisational Structure A platoon is typically the smallest military unit led by a commissioned officer. In the CA, the Platoon Commander is usually a junior officer, such as a Second Lieutenant or Captain. The officer is assisted by a Platoon Warrant, also known as the Platoon 2IC (or 2nd In Command), who holds the rank of Warrant Officer, but can be a Sergeant.

A dismounted infantry platoon is usually divided into three, eight-person sections and a heavy weapons detachment which will deploy a 7.62x51mm General Purpose Machine Gun (GPMG), a

Page 17 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Carl Gustav 84mm anti-tank recoilless rifle, or a C16 automatic 40mm grenade launcher, depending on mission requirements.

Sections are commanded by a Sergeant or Master Corporal with a Master-Corporal or Corporal in the 2IC, position; two members of a section will carry C9 Light Machine Guns (LMG) and the remainder will carry C7 or C8 assault rifles fitted with either optics or a grenade launcher. A section is broken into two assault groups and each assault group is broken down into two, two- person fire teams. Finally, a platoon will also consist of a small headquarters comprising a Combat Medic, Radio Operator, and Forward Observer.

The organisational structure and personnel composition of a typical CA platoon is described in Figure 3.

PLATOON WEAPONS SECTION (3) HEADQUARTERS DETACHMENT

Pla toon Commander Weapons Detachment Leader Platoon 2IC Weapons Detachment 2IC Pla toon Medic Platoon Radio Operator Platoon Forward Observer

tion Commander Rifleman Riflem Section 2IC Rifleman Riflem Sergeant Corporal / Private Corp Master Corpora l Corporal / Private Corp C7 Assault Rifle C7 Assault Rifle C7 C7 Assault Rifle C7 Assault Rifle C7

Figure 3: Canadian Army Dismounted Infantry Platoon Structure and Personnel Composition (including NATO symbols)

Page 18 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.2.2 Dismounted Infantry Platoon Generic Missions A previous cognitive task analysis of CA dismounted infantry operations identified three mission types – attack, defend and patrolling (Tack & Angel, 2005). For the purposes of the SRT use cases, the attack mission was further decomposed into the following components (see Figure 4):

1. Preparation for Battle. Conducted in an assembly area, the platoon reviews orders and readies equipment and supplies. 2. Move to Attack Position. As ordered, the platoon moves to the attack position. 3. Advance to Contact. Using tactical formations (e.g., arrowhead or extended line formations), the platoon advances towards the enemy. 4. Reaction to Enemy Fire. Upon encountering enemy fire, all members of the advancing sections that have been fired upon seek cover and return fire. 5. Locating the Enemy. Speculative fire and changing fire positions are used to locate the enemy. 6. Winning the Firefight. A fire control order is given by the Section Commander to ensure enough weight of fire is applied to neutralize the enemy and win the fire fight. 7. Approach. An assault group approaches within grenade range of the enemy using a covered approach as well as fire and movement techniques. 8. Assault. Once the assault group is within grenade range, an assault of the enemy trench is conducted. 9. Consolidation. Following the assault, the section performs consolidation, including ammo re-distribution, prisoner of war handling, and administering first aid to any casualties. 10. Establish Platoon Patrol Base. The platoon establishes a temporary patrol base.

MOVE TO ESTABLISH PREPARATION ADVANCE TO REACTION TO LOCATING THE WINNING THE ATTAC K APPROACH ASSAULT CONSOLIDATION PLATOON FOR BATTLE CONTACT ENEMY FIRE ENEMY FIREFIGHT POSITION PATROL BASE

Figure 4: Generic Mission Profile of a Canadian Army Dismounted Infantry Platoon

Page 19 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.3 Description of the Proposed Product or System

The ISS depicted in Figure 5 is a wearable device equipped with electronic systems that provides the soldier with sensors, power, connectivity, comms and handheld and ruggedized laptop computing resources equipped with a Battle Management System (BMS). ISS provides soldiers with the capability to communicate and coordinate with other soldiers, and for reporting to superiors. The focus of the current report is to gather requirements for extending the capability of the ISS concept to include access and control of robotic assets that can assist in carrying out their tasks, as illustrated in Figure 5. The robotic assets considered in this report are limited to UGVs and UAVs, but the concept applies to Unmanned Surface Vehicles (USVs) and Unmanned Underwater Vehicles (UUVs) as well as the generic class of all Unmanned Vehicle Systems (UVSs). This group of unmanned assets is collectively referred to as UxV where x can represent, A for aerial vehicles, S for surface vehicles, U for underwater vehicles and G for ground vehicles. A similar nomenclature can be employed for UxS, referring to the unmanned system. This terminology has been used by NATO interoperability activities as early as 2013 in the NATO Technical Activity “Command and Reporting Standards and Associated Development Tools for UxS”.

Small Unmanned Command ISS Aircraft System Post / HQ Miniature Aerial Vehicle

Micro Aerial Vehicle

INTEGRATED Unmanned SOLDIER SYSTEM Ground Vehicle (ISS)

Unmanned Surface Vehicle

Unmanned Underwater Vehicle

Figure 5: Future ISS System Overview

From a system perspective, the ISS is controlled by soldiers and interacts with:

• Other soldiers equipped with ISS or radio comms that are connected to the soldier’s PAN; • Superiors, command posts, headquarters, tasking authorities, and others connected to

Page 20 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

the LCSS voice or data networks; and • Robotic assets.

Requirements for the first two sets of interactions have already been covered by previous research initiatives and have led to the procurement of prototype ISS devices. The control of robotic assets has not yet been integrated and is therefore the focus of the work reported on in this document.

Figure 6: Human Supervisory Control

As shown in Figure 6, the combination of conventional automation and emerging artificial intelligence (AI) technologies contributes to increasing the level of human supervisory control of autonomous systems which have already contributed to commercially available deployable UVS for military applications for air, land and sea applications. For example, the Sea Hunter4 is a highly autonomous USV, which was developed as a Defense Advanced Research Projects Agency (DARPA) project launched in 2016. In 2018 Rheinmetall launched the Mission Master5, a multipurpose UGV for reconnaissance and surveillance operations, and more specifically for medical evacuation (MEDEVAC) and fire missions. As for UAVs, the US and other allies have been actively developing capabilities for micro and miniature unmanned aerial vehicles that can be carried, deployed, and controlled by soldiers6.

4 https://en.wikipedia.org/wiki/Sea_Hunter 5 https://www.rheinmetall.ca/en/rheinmetall_canada/systemsandproducts/electronicsystems/unmanned_vehicles/index.php 6 https://www.c4isrnet.com/unmanned/uas/2017/05/25/mini-drones-a-growing-interest-as-a-us-military-capability/

Page 21 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

With respect to ISS capabilities, the underlying assumption is that as the level of autonomy of UVS increases, they will require less human supervision and therefore it becomes feasible to incorporate simplified UVS control interfaces in either the ruggedized laptop or the TUI.

2.4 Supporting Analysis

This section summarizes the elements of the CE concept that relate directly to the ISS operational requirements and describes two analyses that were conducted to support the development of the SRT concept themes identified during the stakeholder interviews. 2.4.1 Integrated Soldier System The CE concept is at the center of the SRT operational requirements analysis. CE is an evolution of the Adaptive Dispersed Operations (ADO) concept and shares the following goals:

• Agility. The ability to plan and conduct operations at a rapid tempo. • Connectivity. A fundamental requirement that allows commanders to develop a shared understanding, provides combat units with necessary resources and provides support units with the capability to manage and distribute materiel with real-time asset tracking and reach-back to logistics and support organizations. • Modularity. Creating and integrating force groupings for a specific task and rapidly reallocating assets upon task completion. • Adaptive Dispersion. The ability to employ coordinated actions by widely dispersed teams and for these teams to aggregate rapidly to concentrate combat power.

The CA CE Capstone Concept calls for an evolution from conventional, combat-effective, multipurpose forces with enhancements in the areas of connectivity, agility, adaptability, integration and robustness. The CE concept adds the goal of achieving adaptability to a changing operating environment with a battle rhythm on the order of minutes, and therefore the need for a force that can adjust rapidly to large changes in terms of configuration, application, location and use of land forces. The nature of future operations will require working effectively with Joint, Interagency, Multinational and Public (JIMP) partners in an operational environment characterized by new threats (e.g. cyberthreats and degraded networks) and adverse conditions. This imposes an additional requirement for robustness in the Future Land Operating Environment (FLOE) across a range of conditions.

The implications for the Land Forces include establishing real-time shared SA and for a rapid concentration of Combat Service Support (CSS) functions distributed among coordination elements from HQ to units to sub-units (DND, 2011a). The CE concept addresses the need for enhanced decision support tools, training, and the use of vehicles and associated equipment that must balance inter and intra-theatre mobility while still maintaining the ability to operate in rough terrains and urban centres. Robotic assets may be attributed to all levels for medical support, surveillance, logistic, combat support and other operations. Small robotic assets such as

Page 22 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

micro-UAVs may be independently allocated, transported, deployed and controlled as standard- issue soldier equipment. Larger robotic assets, such as a Mule UGV, may be allocated to a sub- unit for a given mission or during a mission. The ISS is well-suited for requesting robotic assets, for controlling them and for reporting the results of intelligence-gathering tasks.

Although the use of robotic assets from other CAF components, (e.g. Royal Canadian Navy (RCN) and the Royal Canadian Air Force (RCAF)), is not considered in this report, USV, UUV and larger UAV platforms may contribute to CA Land Forces operations by providing information, communication relays, and possibly even combat support. Therefore, although CA units may not use the ISS to exercise control RCN or RCAF assets, there may be an interaction between these assets and the ISS. These interactions have not been analysed as part of the current work and should be will be considered as part of future studies. 2.4.2 SRT Use Cases This section first presents the results of a supporting document analysis that was conducted to complement the SRT concept themes identified by the DND stakeholders. It then describes the development process of the SRT use case storyboards.

2.4.2.1 SRT Document Review A brief review of documents provided by DRDC TRC was conducted to provide the project team a wider understanding of the SRT state-of-the art from FVEYS allied partners, industry and academia. A summary of the documents reviewed is presented below.

• Allied Partners

o Autonomous UAVs for Blood Delivery. Conducted by the RAND Corporation, this analysis considered autonomous UAVs for medical resupply missions, in particular blood delivery. The framework developed by this study can be used to understand the requirements, capabilities, and cost drivers of small UAV delivery systems (Gilmore et al, 2019). This research relates to the ‘just in time’ logistics and resupply to dispersed dismounted infantry units SRT concept theme.

o Optimising Human Control of Virtual Robotic Swarms. This document provides a brief review of a psychological approach to supporting human-swarm interactions (Cain, 2019). This research relates to the sensing the local environment to increase soldier LSA and SSA SRT concept theme.

o Platoon-level autonomous Uninhabited Ground Vehicles (UGVs) (United States Marine Corps). The study examines a USMC rifle platoon’s energy and power consumption, operational reach, and operational effectiveness for a scouting and patrolling mission. The intent is to use autonomous UGVs to alleviate the burden on each Marine. A trade-off analysis was conducted to determine the platoon’s success relative to the number of integrated UGVs (Stimpert, 2014). This research relates to the reduction of dismounted soldier burden (including Combat Medic) SRT concept theme.

Page 23 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

o TTCP Autonomous Strategic Challenge. The Autonomy Strategic Challenge (ASC) sought to enhance, demonstrate and evaluate the military utility of autonomous systems for future FVEYS littoral operations. The ASC team developed and subsequently evaluated a novel command and control (C2) system that would enable a single operator to control ten or more autonomous assets. The approach leveraged by the team was to build upon the Air Force Research Laboratory's existing C2 testbed by integrating numerous best-of-breed human- autonomy teaming capabilities from the United States, Australia, Canada, and the United Kingdo (Bartik, 2019). This research relates to the semi-autonomous weapon platform to provide offensive capabilities and sensing the local environment to increase soldier LSA and SSA SRT concept themes. • Industry

o Expeditionary Modular Autonomous Vehicle (EMAV, QinetiQ North America). EMAV is a highly mobile, multipurpose autonomous UGV ground vehicle with a payload capacity of 7,000 pounds. It is meant to provide tactical-scale infantry support at the platoon level and offers precise fires effects. EMAV also features a common controller for vehicle and payload C2 systems. It can be transported inside an MV-22 Osprey and provide exportable power (AFCEA International, 2018). This vehicle relates to the reduction of dismounted soldier burden (including Combat Medic) SRT concept theme.

o Mission Master multi-purpose UGV (Rheinmetall). Rheinmetall has developed a concept of autonomous multipurpose ground vehicles for reconnaissance and surveillance operations, tactical overwatch, CBRN detection, CASEVAC operations, and firefighting missions (Rheinmetall Defence Canada, 2020). Three Mission Master variants have been developed to date: . Mission Master Cargo is designed to alleviate the modern infantrymen’s burden by carrying a half-ton payload of supplies, equipment, and materials. This vehicle relates to the reduction of dismounted soldier burden (including Combat Medic) SRT concept theme. . Mission Master Protection provides close fire support using the Thales 70mm rocket launcher, a 50-calibre machine gun, or the C16 automatic 40mm grenade launcher. This vehicle relates to the semi-autonomous weapon platform to provide offensive capabilities SRT concept theme. . Mission Master Rescue is capable of autonomously evacuating casualties over long distances. The module features two basket stretchers with sliding provisions and can carry specialized, high-performance equipment needed for in-field medical intervention. While transporting a wounded soldier using the UGV’s stretcher, for example, accompanying medics can use the vehicle as a workstation to administer emergency care. This vehicle relates to the evacuation of battlefield casualties SRT concept theme.

Page 24 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

o Small Multipurpose Equipment Transport (SMET, General Dynamics Land Systems). In 2019 the United States Army selected General Dynamics Land Systems to produce the SMET, to lighten soldier burden by providing Infantry Brigade Combat Teams a robotic ‘mule’ capability. The contract is valued at $162.4 million to produce 624 SMETs. Delivery to the United States Army begins in 2021 (Global Security, 2020). This vehicle relates to the reduction of dismounted soldier burden (including Combat Medic) SRT concept theme. • DRDC Research

o Human Systems Performance Project. This DRDC research project seeks to improve the effectiveness of dismounted infantry soldier’s through the incorporation of new technologies including ‘Autonomy and Mobility Enhancement’. Within this theme, three research areas have been identified: manned-unmanned teaming, offloading through autonomy, and integration of autonomy for enhanced situation awareness (Collier & Hou, 2019). This research relates to all five SRT concept themes.

o Innovation for Defence Excellence and Security (IDEaS). The ‘Effective Human- Machine Cooperation with Intelligent Adaptive Autonomous Systems’ micro- network comprises partners from several Canadian universities and research institutes. The micro-network was established in December 2019 and its research will focus on the collaborative relationship between humans and machines which is an essential part of autonomous multi-vehicle single-operator control (Government of Canada, 2018). This research relates to the sensing the local environment to increase soldier LSA and SSA SRT concept theme.

o Preliminary Statement of Requirements for a Micro Aerial Vehicle System. This document provides a statement of requirements to guide the development of a micro UAV, including a draft concept of operations (Hou et al, 2012). This research relates to the sensing the local environment to increase soldier LSA and SSA SRT concept theme. • DND Publications

o Close Engagement – Land Power in the Age of Uncertainty. This document presents the Canadian land operations capstone operating concept for evolving adaptive disperse. It is a conceptual model of how the CA should be configured, equipped, and trained in the medium term, approximately 10 to 15 years in the future. Close Engagement cites the use of autonomous UAV and UGV assets to support persistent wide-area surveillance and security, deliver combat supplies, defend against enemy indirect fire (i.e., C16 automatic 40mm grenade launcher, also known as the Close-In Weapons System), and provide EW jamming capability. The document also provides a brief description of the Battle of Santa Maria 2035 mission vignette which provided a suitable context for the development of the SRT use case storyboards (Aral, 2019). This document relates to all five SRT concept themes.

Page 25 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

o Directorate of Land Requirements 2 Innovation Program – Decision-Action Cycle Optimisation and Technology Adoption. This briefing includes reference to sensor augmented intelligence by which multiple autonomous platforms coordinate their actions to provide persistent situation awareness of the soldier (Directorate of Land Requirements 2, 2019). This research relates to the sensing the local environment to increase soldier LSA and SSA SRT concept theme.

o Reducing Soldier Burden using Autonomous Vehicles. The aim of this project was to explore the possible use of autonomous systems to reduce soldier burden. The work included interviews with CA dismounted battalion leadership on the soldier problem and the way forward. Recommendations were made regarding the type of autonomous systems required (Choi, 2019). This research relates to the reduction of dismounted soldier burden (including Combat Medic) SRT concept theme.

o Soldier Systems Technology Roadmap. This document captures and summarises the findings of the development phase of the Soldier Systems Technology Roadmap (2011-2025) initiative. It includes a description of characteristics of adaptive dispersed operations which cite distributed autonomous systems to support solider situation awareness and information sharing (including micro UAVs), semi-autonomous target detection, identification and engagement, and autonomous payload systems (including autonomous navigation) (DND, 2011a). This document relates to the sensing the local environment to increase soldier LSA and SSA, semi-autonomous weapon platform to provide offensive capabilities, and reduction of dismounted soldier burden (including Combat Medic) SRT concept themes.

The understanding of state-of-the-art SRT concepts that these references provided was used by the project team to build upon the ideas brainstormed during the stakeholder analysis interviews when developing the use case storyboards for each of the five SRT concept themes.

2.4.2.2 SRT Use Case Storyboard Development Five SRT concept themes emerged from the stakeholder interviews based on the perceived capability gaps of current soldier systems.

1. Soldier Burden Reduction; 2. Casualty Evacuation (CASEVAC); 3. Semi-Autonomous Weapon Platform; 4. Sensing Environment; and, 5. Logistics & Resupply.

Page 26 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

For each SRT concept, a unique storyboard use case was developed to document the SRT use case information collected during the stakeholder interviews. The storyboards were developed using a sequence of colour drawings to visualise the performance capabilities of the autonomous platform and how it would interact with the soldier to achieve mission goals. This approach provided the context and atmosphere, as well as the detailed use of the SRT technology by the intended end user(s), as part of the unfolding narrative. The SRT storyboard use cases utilised the following visualisation components:

• Panel. A panel is an individual frame, or single drawing, in the multiple-panel sequence of a storyboard. A panel consists of a single drawing depicting a frozen moment. • Narrative Block. Text that appears in a square block in the panel that talks directly to the reader. It is not part of the action, but it helps to tell the reader what is going on in the panel. • Close-up. Used for emphasis, this simulates when a camera zooms in tight on an object or the character's face. This shot puts the reader's eye just a few inches away from the action. • Medium Shot. When all or most of a character's body, or a couple of character's bodies, are in shot. This shot puts the reader's eye just a couple of feet away from the action. • Long Shot. Used to help establish location or context, almost the entire scene can be seen in a single panel of the storyboard. This simulates the camera being far away from the action. The SRT use case storyboards are presented in the following section.

2.5 Missions that the Proposed System Will Accomplish

This section presents the use case storyboards which help define the missions that the proposed SRT technologies will support.

The stakeholder interviews and analysis identified five SRT concept themes. For each theme, a use case narrative was constructed to capture the following information:

• Operational need or current capability gap; • Intended end user of the SRT technology; • Type of autonomous platform (e.g., UAV, UGV); • Capability and functionality of the SRT technology; • Context of use, such as environmental conditions and adversaries; and, • Enhancements to mission performance from using the SRT technology.

Page 27 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Finally, a suitable SRT use case title was identified to provide a convenient and memorable way of describing the narrative. This information is summarized in Table 2.

Table 2: Identification of SRT Use Cases from Stakeholder Interviews

SRT Concept Themes Discussed SRT Use Case Platform Type

Soldier Burden Reduction Mule UGV

Sensing Environment (including security Wingman Micro-UAV uses)

Semi-Autonomous Weapon Platform Artemis7 UGV

Casualty Evacuation (CASEVAC) RoboMedic UGV

Logistics & Resupply Amazon UGV & UAV

Figure 7 describes the contribution of each stakeholder to each of the five SRT use cases. As discussed, the stakeholders had consistent and complementary views regarding current capability gaps and how future SRT concepts could mitigate them.

Maj Han LCol Chayer Capt Beauchamp LCol Corby WO Evong Sgt Lavoie Lt(N) Nelson Maj Champagne RCIC DLR-5 RCR NDMC DLR-2 CALWC

USE CASE #1: USE CASE #2: USE CASE #3: USE CASE #4: USE CASE #5: ‘Mule’ ‘Wingman’ ‘Artemis’ ‘RoboDoc’ ‘Amazon’

Figure 7: Contribution of each Stakeholder to the SRT Use Cases

7 Artemis is the Olympian goddess of the hunt. https://www.greekmythology.com/Olympians/Artemis/artemis.html

Page 28 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

The following sections describe the SRT use case scenarios developed by the project team. Each section provides a brief description of the use case and operational need (i.e., capability gap), and the intended end user of the system. 2.5.1 Mule The ‘Mule’ SRT use case is presented in Table 3.

Table 3: SRT Use Case Storyboard – ‘Mule’

Description Function End User Operational Need MULE UGV provides load-carrying for 1. Reduce load carrying burden dismounted infantry section during of equipment and combat patrol mission. supplies by Dismounted Section 2IC is responsible for the Infantry. control of the MULE using personal Sustain Section 2IC 2. Increase the mobility of Direct ISS. Fire weapons (e.g. C16 MULE is loaded with combat supplies, automatic grenade launcher) so equipment and C16 automatic that they can be used in an grenade launcher; reducing the load- offensive posture. carrying burden of the section.

Page 29 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 30 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 31 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.5.2 Wingman The ‘Wingman’ SRT use case is presented in Table 4.

Table 4: SRT Use Case Storyboard – ‘Wingman’

Description Function End User Operational Need

Hand-launched, hybrid micro- UGV/UAV to provide tactical sensing 1. Improve soldier situation capability to dismounted infantry awareness. soldier. Combined UGC/UAV 2. Reduce soldier exposure to functionality to provide longer Dismounted enemy fire (i.e., security endurance capability. Infantry overwatch). Dismounted infantry soldier is Soldier 3. Improve engagement responsible for the control of the Sense response times. Wingman using personal ISS. (1 per Fire 4. Provide deception capability Wingman is equipped with an electro- Team) to spoof the enemy’s EW optical / infra-red sensor, an operations. Electronic Warfare (EW) decoy capability, and a Chemical, Biological, 5. Provide CBRN detection and Radiological and Nuclear (CBRN) warning capability. detection capability.

Page 32 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 33 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 34 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.5.3 Artemis The ‘Artemis’ SRT use case is presented in Table 5.

Table 5: SRT Use Case Storyboard – ‘Artemis’

Description Function End User Operational Need

UGV to operate as a semi-autonomous weapon platform (with human-in-the- 1. Reduce soldier exposure to loop authority to engage) for enemy fire. dismounted infantry platoon during 2. Improve engagement Platoon patrol mission. response times. Weapons Platoon Weapons Detachment Gunner Act Detachment 3. Increase the mobility of is responsible for the control of Artemis Gunner Direct Fire weapons (e.g., using personal ISS. 81mm mortar) so that they MULE is loaded with 81mm self-loading can be used in an offensive mortar and ammunition; reducing the posture. load-carrying burden of the platoon.

Page 35 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 36 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 37 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.5.4 RoboMedic The ‘RoboMedic’ SRT use case is presented in Table 6.

Table 6: SRT Use Case Storyboard – ‘RoboMedic’

Description Function End User Operational Need

UGV to provide support to Dismounted Combat Medic performing battlefield 1. Reduce load carrying casualty treatment and evacuation. burden of medical equipment Platoon Combat Medic is responsible for by Platoon Combat Medic. the control of the RoboMedic using 2. Rapid casualty evacuation Platoon personal ISS. (requiring no Platoon Sustain Combat RoboMedic is loaded with medical personnel to transport the Medic supplies and health monitoring casualty) whilst continuing equipment; reducing the load-carrying medical treatment. burden of the Platoon Combat Medic 3. Rapid replenishment of and the number of Platoon members medical expendables. required to evacuate the casualty.

Page 38 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 39 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 40 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.5.5 Amazon The ‘Amazon’ SRT use case is presented in Table 7.

Table 7: SRT Use Case Storyboard – ‘Amazon’

Description Function End User Operational Need

A combination of semi-autonomous assets to provide ‘just-in-time’ delivery of supplies and equipment repair parts to deployed infantry platoon (i.e., ‘sense and respond logistics’) and enhance perimeter security around the patrol base. Specifically: - UGV to deliver resupply container to platoon in the field. - Four UAVs (onboard the resupply container) provide local resupply of combat supplies to dispersed members of the platoon (e.g., 1. Optimise timely resupply of sniper) and enhance perimeter expendables (water, food, security around the patrol base. ammunition, batteries) to Logistics personnel at Battel Group (BG) dispersed troops on the Forward Operating Base (FOB) are Logistics battlefield responsible for determining resupply Sustain Personnel at 2. Reduce exposure of strategy and dispatching autonomous BG FOB manned convoys to enemy resupply UGV convoys. threats. The Platoon 2IC is responsible for Platoon 2IC 3. Enhance perimeter security coordinating the request for resupply around the patrol base. and the control of the Amazon UAV fleet. Resupply container is loaded with: - Ammunition and combat supplies (according to expenditure tracked by ISS). - Water and food (according to energy expenditure and weather conditions tracked by ISS). - Batteries and fuel (according to expenditure tracked by ISS). - Medical supplies (according to expenditure tracked by

Page 41 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Description Function End User Operational Need RoboMedic). - Repair parts (equipment corrective and preventative maintenance requirements tracked automatically through onboard ‘Vetronics’ system). - C2 equipment and communications to establish platoon HQ. Four UAVs to support local re-supply and patrol base security (docking on roof of container includes power supply for recharging).

Page 42 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 43 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Page 44 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2.5.6 Summary The mission context translates into five SRT employment use cases that are further decomposed in section 3.3. They provide details concerning the operational use of the system in the mission context and frame the subsequent use case analysis from which operational requirements are derived and summarized in section 3.3.3.

2.6 Operational and Support Concept

The ISS is consistent with the CE operational concept, also known as the Operations Concept (OPSCON) as related to the deployed system. The ISS main function is to provide the soldier with connectivity. This connectivity supports requirements for maintaining a real-time SSA while providing access to resources such as robotic platforms and facilitates reporting and communication with superiors. The ISS also offers the soldier the opportunity to create a PAN over which information can be shared with other soldiers to coordinate activities and over which control of robotic assets can exercised. 2.6.1 Concept of Operations The CE is an evolution of the ADO for which combat support services is a foundational element. In the context of SRT, these services are provided in part through the use of robotic assets to provide:

• Transport services to reduce load-burdening, intelligence-gathering and other Intelligence Surveillance and Reconnaissance (ISR) services to maintain soldier SA; • Combat support service to engage enemies and demonstrate show of force, as required; and • Medical support and communications support. Figure 8 illustrates the SRT component of the CE CONOPS8. The ISS communicates and provides basic services to soldiers including the capability to communicate with other soldiers that are in the same unit, or more precisely, accessibility through the PAN. Communication with soldiers from other units may be achieved using the ISS Network. If the ISS Network is not available to one or both units, soldiers operating on different PANs may communicate using comms relays provided by fixed or mobile/robotic assets. Soldiers who have been allocated robotic assets will have robotic system monitoring and control functions available to them either across the ISS Network or through direct communication links between the ISS and the robotic asset. Robotic assets and ISS are in communication with Platoon HQ and Battle Group HQ. Since logistics is a key focus of the ISS/SRT CONOPS, the logistics unit will provide supply and distribution,

8 The CONOPS is a broad outline of the organization’s assumptions or intent regarding an operation or a series of operations and is frequently embodied in long-range planning whereas the OPSCON is a statement of an organization’s assumptions and intent regarding an operation or a series of operations as pertaining to a system or a related set of systems.

Page 45 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

maintenance and medical support to the deployed units made possible through coordination among the robotic assets and the ISS-equipped soldiers.

Although robotic assets may include USVs and UUVs as well, the focus of the current study was on UAVs and UGVs. Three categories of vehicles corresponding to three mission-types were considered: ISR vehicles, weaponized vehicles, and utility/supply vehicles.

The focus of the present study is on determining operational requirements for facilitating the use of robotic systems by soldiers using the ISS, including military communication requirements for generating reports and requests. Interoperability requirements to be considered include ensuring that ISS are equipped with the proper protocols to communicate with and control the robotic assets and ensuring that standard messaging protocols are utilized between the ISS and PN/BG HQs.

Figure 8: ISS Solder-Robot Teaming CONOPS

2.6.2 Existing System Shortfalls This section describes how current soldier systems cannot meet current or projected requirements and what new capabilities are needed to address the gap between current capabilities and required capabilities.

The stakeholder analysis identified the following operational needs:

Page 46 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

1. Soldier Overburden (Sustain). Soldiers are required to carry increasingly more equipment, food, water, ammunition (personal and section), and batteries over increasingly longer periods of time between resupply. The problem of overburden is even more severe for Combat Medics who are also expected to carry medical supplies. Currently, there are no systems for alleviating dismounted infantry soldier burden. 2. Battlefield Casualty Evacuation (Sustain). Casualty evacuation from the battlefield is currently achieved using a stretcher and four soldiers, in addition to a security detail if required. Casualty evacuation therefore diverts soldiers away from the frontline and exposes them to additional risks. Currently, there are no systems for supporting battlefield casualty evacuation. 3. Mobility of Platoon-Level Heavy Weapons (Act). Currently, dismounted infantry soldiers are required to carry the platoon’s heavy weapons, such as the 7.62x51mm GPMG, Carl Gustav 84mm anti-tank recoilless rifle, C16 automatic 40mm grenade launcher, anti-tank system, 81mm mortar. Currently, there are no systems for supporting the movement of platoon heavy weapons and as a result they are used in a defensive posture only. 4. Soldier Situation Awareness (Sense). The United States Army doctrine defines SA as “the ability to have accurate real-time information of friendly, enemy, neutral, and non- combatant locations; a common, relevant picture of the battlefield scaled to specific levels of interest and special needs” (TRADOC, 1994). The implementation of ISS will significantly increase the SA of soldiers through blue force tracking and the sharing of enemy contact information within the platoon. However, the sharing of imagery from autonomous assets across the platoon, as well as automatic target detection and identification, will have a profound impact on the soldier’s ability to conduct accurate and timely situation analysis and decision-making activities on the battlefield. It would also reduce soldiers’ exposure to enemy fire by providing a persistent overwatch and security capability. 5. Resupply to Dispersed Units (Sustain). DND is exploring new technology concepts to support Sense & Respond Logistics Adaptive Planning; whereby tactical sustainment in terms of asset control and management is supported within the context of adaptive and dispersed operations. In order to support these activities, the technology concepts must support data access, asset visibility, monitoring, analysis, and forecasting, as well as adaptive planning activities such as distribution, maintenance and supply chain management. However, the current capability gap relates to the ‘last mile’ logistics in which supplies must be delivered to dispersed units in the field without exposing logistics personnel to enemy fire. 2.6.3 Threat The underlying motivation of using the ISS is to establish and maintain an information advantage over adversaries. The principal threat is the inability of allied forces to react and adapt fast enough to respond to new and evolving situations. In the case of degraded or compromised tactical networks, using ISS among soldiers within the same PAN will allow soldiers to regroup and react as per the ADO and CE concepts. The other threats are related to

Page 47 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

exposing soldiers to possible enemy fire and ambushes during patrol, replenishing and supply, escort and other missions. The use of robotic assets can increase soldiers’ autonomy and keep them out of harm’s way. Finally, in the case of casualties, evacuating injured personnel can expose soldiers to enemy fire. The use of robotic platforms can facilitate and expedite medical assessments and medical evacuations. 2.6.4 Support Concept The intent of this section is to describe the support concept for the system in terms of hardware and software maintainability, maintenance personnel, and operator training. On considerations for extending the ISS to include SRT requirements involving the use of robotic assets by soldiers. The emphasis is on refining the ISS operational concept in the context of the broader CE OPSCON and defining the system requirements associated with this refinement and with the integration of the refined ISS with robotic assets. Therefore, the analysis of the support concept, i.e. how the deployed system will be supported, was out of scope and should be considered as part of future studies.

Page 48 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3. CAPABILITIES REQUIRED

3.1 Operational Performance Parameters

The purpose of the current document is to identify system features and high-level operational requirements and therefore a specific set of Operational Performance Parameters (OPPs) will be the subject of future work; including validation with DND SMEs and stakeholders. The following categories of OPPs will need to be considered as part of future analyses:

• Soldier Burden Reduction. OPPs relating to the Mule SRT concept include, but not limited to, the following: - Combined weight of supplies and equipment that can be transported - Range - Speed - Time on task - Terrain type and obstacles - Reporting (accuracy and time taken) • Casualty Evacuation. OPPs relating to the RoboMedic SRT concept include, but not limited to, the following: - Combined weight of casualties, supplies, and equipment that can be transported - Range - Minimum speed - Time on task - Terrain type and obstacles - Reporting (accuracy and time taken) • Semi-Autonomous Weapon Platform. OPPs relating to the Artemis SRT concept include, but not limited to, the following: - Combined weight of weapon and ammunition that can be transported - Range - Speed - Time on task - Terrain type and obstacles - Detect-to-engage time - Target detection accuracy

Page 49 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

- Target identification accuracy - Target engagement accuracy - Reporting (accuracy and time taken) • Sensing Environment. OPPs relating to the Wingman SRT concept include, but not limited to, the following: - Weight of sensor payload to be carried - Range - Altitude - Speed - Time on task (e.g., loiter-time in the air) - Sensor coverage (%) - Terrain type and obstacles - Target detection accuracy - Target identification accuracy - Reporting (accuracy and time taken) • Logistics & Resupply. OPPs relating to the Amazon SRT concept include, but not limited to, the following: - Autonomous Convoy i. Weight of supplies to be transported ii. Range iii. Speed iv. Terrain type and obstacles v. Reporting (accuracy and time taken) - Remote Re-Supply i. Weight of supplies to be transported ii. Range iii. Altitude iv. Speed v. Terrain type and obstacles vi. Detectability vii. Reporting (accuracy and time taken)

Page 50 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

- Convoy / Perimeter Security i. Weight of sensor payload to be carried ii. Range iii. Altitude iv. Speed v. Time on task (e.g., loiter-time in the air) vi. Sensor coverage (%) vii. Terrain type and obstacles viii. Target detection accuracy ix. Target identification accuracy x. Reporting (accuracy and time taken)

3.2 Key Performance Parameters (KPPs)

The definition of key performance parameters (KPPs) will be addressed as part of future work, in conjunction with the definition of the OPP will be defined. These products will be developed and validated with suitable DND SMEs and stakeholders.

For the purposes of this report, an HFE test plan has been developed which outlines the main operator-related research issues pertaining to SRT and describes several techniques for assessing the effectiveness and military utility of the soldier-robot partnership (see Annex C). The test plan provides information that will contribute toward determining the suitability and added value of the proposed system features, described in Section 3.3.3.

3.3 System Performance & Employment Scenarios

Consistent with the CD&E guidelines for developing military capabilities, a stakeholder engagement activity was executed that included a series of stakeholder interviews and workshops (Labbe et al, 2006). This is part of stakeholder engagement activities that typically occur as part of concept discovery and concept refinement activities. The output of this activity was the identification of the capability gap and employment scenarios that include the combined employment of the ISS with robotic assets to perform key operational functions, as highlighted in Section 2.

In this section, the employment scenarios, expressed as five storyboard use cases that connect the mission context to the operation needs, are further analysed to identify SRT system features, information requirements, intended user(s) (including roles and responsibilities), and mission- related tasks.

Page 51 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.3.1 Methodology

A preliminary MFTA was conducted on each SRT use case storyboard. A MFTA is a top down analysis framework developed for military environments and is described in detail in MIL-STD- 46855A (DND, 2011b). The analysis begins with an analysis of mission goals and user-developed scenarios, followed by the identification of top-level functions, which are successively decomposed to a level that equates to an operator task. At each stage, requirements can be extracted to inform the design of a system. The MFTA was applied early in the design process as, by its nature, it tends to answer questions of scope and capability.

The MFTA methodology is described in more detail in the following sections.

3.3.1.1 Mission Analysis

In accordance with HF processes described in MIL-HDBK-46855A, the mission analysis is an essential prerequisite for all subsequent analysis activities. The objective of the mission analysis is to define the boundaries of the subsequent function and task analyses in terms of the scenarios, the operators (and their likely characteristics), the anticipated system functions and features, and the likely environments in which the system will operate.

For the purposes of this project, the SRT use case storyboards fulfilled the objectives of a mission analysis.

3.3.1.2 Function Analysis

The objective of the function analysis is to identify and describe the major activities (i.e., functions) that must be performed to satisfy the mission requirements and objectives (as identified by the Mission Analysis).

Functions are broad categories of activities performed within a scenario, and all functions can be broken down or divided into more detailed functions. Using mission objectives (as described in the SRT use case storyboards as a starting point), the functional decomposition refined these objectives to the point at which a hierarchy of specific operator functions and sub-functions were identified. The identification of the mission-related functions provides the operational context for deriving the SRT operational requirements used by the system to achieve mission goals.

3.3.1.3 Task Analysis

A task is a discrete event or action that enables a mission-related function to be accomplished. Based on the generic infantry attack mission profile described in the previous section, all mission-related functions were further decomposed down one to two levels in order to identify

Page 52 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01 high-level tasks that encapsulate discrete soldier-robot interactions. For each task the following information was captured:

• Information Requirements. A list of the information required to fulfill the task. • Soldier-Robot Teaming Features. A brief description of the SRT functionality required to support the task. Future work will validate the task analysis with suitable DND SMEs and stakeholders.

3.3.2 Results

This section presents the results of the preliminary MFTAs conducted on the five SRT use case storyboards.

Page 53 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.3.2.1 Mule

The functional and task decomposition for the Mule SRT use case is presented in Figure 9. The results of the task analysis are presented in Table 8.

SOLDIER-ROBOT TEAMING USE CASE ‘MULE’

CONDUCT TACTICAL MAINTAIN LOCAL CONDUCT POST- REACT TO ENEMY FIRE MOVEMENT SITUATION AWARENESS ENGAGEMENT ‘RE-ORG’

LISTEN / LOOK FOR TAKE UP COVERING FIRE ADOPT ALL-ROUND VERIFY POSITION ENEMY POSITION DEFENCE

MONITOR LOCATION OF LOCATE ENEMY (WRT MAINTAIN CHOSEN FRIENDLY TROOPS ASSIGNED FIRING ARCS) ASSESS CASUALTIES ROUTE

MONITOR TERRAIN RETURN FIRE ON ENEMY MANAGE PRISONERS FEATURES POSITION

DETERMINE CEASE FIRE ON ENEMY AMMUNITION POSITION EXPENDITURE

SEND SITREP TO HQ

Figure 9: Functional and Task Decomposition – ‘Mule’

Page 54 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 8: Task Analysis – ‘Mule’

Task Description Information Requirements Soldier-Robot Teaming ‘Features’

1. CONDUCT TACTICAL Section 2IC of dismounted infantry section advancing towards objective. MOVEMENT

Current location of Section 2IC (Military Grid Reference System Location of MULE displayed on Section Regular verification of (MGRS)) 2IC’s ISS map. 1.1 Verify Position current position along Current MGRS location of MULE Planned route, including waypoints planned route. Route waypoint location (MGRS) accessed through ISS network.

Objective location (MGRS)

Current location of Section 2IC Section 2IC selects ‘auto follow’ mode on (MGRS) personal ISS and MULE follows location of Section 2IC. Current location of MULE (MGRS) MULE compares location of Section 2IC with Follow chosen route Route waypoint location (MGRS) planned route accessed through ISS between planned waypoint 1.2 Maintain Chosen Route network and alerts Section 2IC if deviating according to terrain type of Objective location (MGRS) from planned route. topography MULE determines optimal path across Terrain type and topography terrain to follow Section 2IC based on appreciation of terrain type and topography, including obstacle detection.

2. MAINTAIN LOCAL Section 2IC maintains awareness of critical cues in their immediate environment. SITUATION AWARENESS

Search for enemy using Enemy location 2.1 Listen / Look for Enemy N/A senses Type & strength of enemy

Page 55 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’

Current location of friendly troops Current location of friendly troops and UVSs 2.2 Monitor Location for Monitor current location of (MGRS) (including MULE) accessed through ISS 9 Friendly Troops friendly troops and UVSs Current location of friendly UVSs network and displayed on Section 2IC’s ISS (MGRS) map.

Monitor terrain for ‘dead Terrain type and topography 2.3 Monitor Terrain Features N/A zones’ Location of terrain ‘dead zones’

Section 2IC reacts to enemy fire by adopting covering position, locating enemy, and returning fire until enemy 3. REACT TO ENEMY FIRE neutralized.

Adopt firing position based 3.1 Take up Covering on suitability of ground Terrain type and topography N/A Position cover / shelter from enemy fire.

Section 2IC views image from C16 gunsight Locate any enemy within remotely using ISS. 3.2 Locate Enemy Enemy location designated firing arcs Section 2IC can slew C16 remotely to locate enemy using ISS.

Returning fire on enemy 3.3 Return Fire on Enemy position with goal of Enemy location Section 2IC can fire C16 remotely using ISS. Position neutralizing enemy.

Cease firing on enemy if Status of enemy 3.4 Cease Fire on Enemy neutralized or receives N/A Position order. Order to ceasefire

4. CONDUCT POST- Section 2IC coordinates the reorganization (REORG) of the section immediately after the resolution of the ENGAGEMENT REORG engagement.

9 Unmanned Vehicle, x: A (aerial), G (ground), S (surface), U (Underwater)

Page 56 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Adopt defensive posture in 4.1 Adopt All-Round response to any enemy Mission orders N/A Defence counterattacks.

Health status of section Health status of each section 4.2 Assess Casualties members as self-reported or N/A member assessed by combat medic

Captured enemy combatants are processed according to 4.3 Manage Prisoners N/A N/A applicable laws of armed conflict.

Automatic tracking of ammunition expenditure of individual section members Assess ammunition Expended ammunition by personnel ISS. 4.4 Determine Ammunition expenditure of all members information (type and quantity) of ISS automatically collates ammunition Expenditure of the section. each section member expenditure of section using data accessed through ISS network and displays report on Section 2IC’s ISS.

See annex A.3. Report should SITREP submitted electronically through ISS 4.5 Send Situation Report include overview of tactical SITREP sent to HQ. network. Date, time, unit and location fields (SITREP) to HQ situation overview and actions of SITREP are automatically completed. taken

Page 57 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.3.2.2 Artemis

The functional and task decomposition for the Artemis SRT use case is presented in Figure 10. The results of the task analysis are presented in Table 9.

SOLDIER-ROBOT TEAMING USE CASE: ‘Artemis’

CONDUCT TACTICAL MAINTAIN LOCAL CONDUCT POST- REACT TO ENEMY FIRE MOVEMENT SITUATION AWARENESS ENGAGEMENT ‘RE-ORG’

LISTEN / LOOK FOR TAKE UP COVERING FIRE ADOPT ALL-ROUND VERIFY POSITION ENEMY POSITION DEFENCE

MONITOR LOCATION OF LOCATE ENEMY (WRT MAINTAIN CHOSEN ASSESS CASUALTIES FRIENDLY TROOPS ASSIGNED FIRING ARCS) ROUTE

MONITOR TERRAIN RETURN FIRE ON ENEMY MANAGE PRISONERS FEATURES POSITION

DETERMINE CEASE FIRE ON ENEMY AMMUNITION POSITION EXPENDITURE

SEND SITREP TO HQ

Figure 10: Functional and Task Decomposition – ‘Artemis’

Page 58 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 9: Task Analysis – ‘Artemis’

Task Description Information Requirements Soldier-Robot Teaming ‘Features’

1. CONDUCT TACTICAL Platoon Weapons Detachment Gunner of dismounted infantry platoon advancing towards objective. MOVEMENT

Current location of Platoon Weapons Detachment Gunner (MGRS) Location of Artemis displayed on Platoon Regular verification of Weapons Detachment Gunner’s ISS map. 1.1 Verify Position current position along Current location of Artemis Planned route (including waypoints) planned route. (MGRS) accessed through ISS network. Route waypoint location (MGRS)

Objective location (MGRS)

Current location of Platoon Platoon Weapons Detachment Gunner Weapons Detachment Gunner selects ‘go to’ mode on personal ISS and (MGRS) enter location on ISS map. Artemis Follow chosen route determines optimal route based on current Current location of Artemis between planned waypoint location, destination point, and topography 1.2 Maintain Chosen Route (MGRS) according to terrain type of (e.g., collision avoidance) and moves to topography Route waypoint location (MGRS) selected location and loiters. Artemis notifies Platoon Weapons Detachment Objective location (MGRS) Gunner that it has arrived at the destination Terrain type and topography point through the ISS.

2. MAINTAIN LOCAL Platoon Weapons Detachment Gunner maintains awareness of critical cues in their immediate environment. SITUATION AWARENES

2.1 Listen / Look for Enemy Search for enemy using Enemy location The loitering Artemis detects movement

Page 59 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ senses and automatically alerts the ISS of the Platoon Weapons Detachment Gunner. Alert includes supporting imagery which is automatically collected and annotated (e.g., Type & strength of enemy target classification, direction of movement). Platoon Weapons Detachment Gunner confirms classification of the contact – ‘Friendly’, ‘Neutral’, or ‘Hostile’.

Current location of friendly Current location of friendly troops and UVSs 2.2 Monitor Location for Monitor current location of troops (MGRS) (including Artemis) accessed through ISS Friendly Troops friendly troops and UVSs Current location of friendly UVSs network and displayed on Platoon Weapons (MGRS) Detachment Gunner’s ISS map.

Monitor terrain for ‘dead Terrain type and topography 2.3 Monitor Terrain Features N/A zones’ Location of terrain ‘dead zones’

Platoon Weapons Detachment Gunner reacts to enemy fire by adopting covering position, locating enemy, 3. REACT TO ENEMY FIRE and returning fire until enemy neutralized.

Adopt firing position based on suitability of ground 3.1 Take up Covering Position Terrain type and topography N/A cover / shelter from enemy fire.

Artemis automatically detects incoming enemy mortar rounds and identifies firing point (origin) based on trajectory. Artemis Locate any enemy within 3.2 Locate Enemy Enemy location sends request for permission (including designated firing arcs map location of origin) to counter-fire to Platoon Weapons Detachment Gunner via personal ISS.

Page 60 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Returning fire on enemy Platoon Weapons Detachment Gunner 3.3 Return Fire on Enemy position with goal of Enemy location gives permission for Artemis to counter-fire Position neutralizing enemy. with 81mm mortar via personal ISS.

Cease firing on enemy if Status of enemy Platoon Weapons Detachment Gunner 3.4 Cease Fire on Enemy neutralized or receives gives Artemis order to cease fire via Position order. Order to ceasefire personal ISS.

4. CONDUCT POST- Platoon Weapons Detachment Gunner participates in the REORG of the platoon immediately after the ENGAGEMENT REORG resolution of the engagement.

Adopt defensive posture in Platoon Weapons Detachment Gunner response to any enemy selects ‘auto follow’ mode on personal ISS counterattacks. and Artemis follows location of Platoon Weapons Detachment Gunner. 4.1 Adopt All-Round Defence Mission orders. Artemis determines optimal path across terrain to follow Platoon Weapons Detachment Gunner based on appreciation of terrain type and topography (including obstacle detection).

Health status of section members as self-reported Health status of each section 4.2 Assess Casualties N/A or assessed by combat member. medic.

Captured enemy combatants are processed 4.3 Manage Prisoners N/A N/A according to applicable laws of armed conflict.

Page 61 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Artemis reports ammunition expenditure to Assess ammunition Expended ammunition (type and 4.4 Determine Ammunition ISS network. Platoon 2IC is notified of the expenditure of all members quantity) of each section Expenditure ammunition expenditure (see Amazon use of the section. member. case).

Current situation (date, time, 4.5 Send SITREP to HQ SITREP sent to HQ. unit, location, tactical situation N/A overview, actions taken)

3.3.2.3 Wingman

The functional and task decomposition for the Wingman SRT use case is presented in Figure 11. The results of the task analysis are presented in

Page 62 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 10.

SOLDIER-ROBOT TEAMING USE CASE: ‘Wingman’

MAINTAIN LOCAL DISGUISE INTENDED CONDUCT ROUTE RECCE SITUATION AWARENESS MOVEMENT

DETERMINE LOCATION / ADVANCE TO LOCATION LISTEN / LOOK FOR TYPE OF DECOY / FORWARD OF PLATOON ENEMY COUNTER-MEASURES

VERIFY LOCATION IS FREE MONITOR LOCATION OF DEPLOY DECOY / OF ENEMY TROOPS FRIENDLY TROOPS COUNTER-MEASURES

MONITOR TERRAIN FEATURES

Figure 11: Functional and Task Decomposition – ‘Wingman’

Page 63 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 10: Task Analysis – ‘Wingman’

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Dismounted Infantry Soldier (as part of a section) conducting a route recce in advance of the platoon to verify 1. CONDUCT ROUTE RECCE planned route of patrol is free of enemy troops.

Current location of Dismounted Regular verification of Infantry Solider (MGRS) 1.1 Advance to Location N/A current position along Forward of Platoon Route waypoint location (MGRS) planned route. Objective location (MGRS)

Current location of friendly troops Dismounted Infantry Solider selects (MGRS) ‘overwatch’ mode on personal ISS and selects location (and height) on ISS map. Wingman is hand-launched and hovers to Enemy location selected location. Three other Wingmen are deployed from the section. The ‘swarm’ of four Wingmen Type & strength of enemy self-synchronize to optimize sensor Search location for presence coverage. Consideration of current location 1.2 Verify Location is Free of of enemy activity. If present, of section members and terrain/building Enemy Troops determine type and strength blind spots to determine optimal sensor of enemy forces. coverage. Imagery from all Wingmen EO/IR sensors sent to ISS network in real-time. Terrain type and topography Dismounted Infantry Solider selects which (including blind spots) of the four Wingmen imagery feeds to view. CBRN detection status of Wingmen is also accessible by ISS network and alerts automatically sent to all section soldiers if CBRN materials are detected.

Page 64 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’

2. MAINTAIN LOCAL Dismounted Infantry Soldier maintains awareness of critical cues in their immediate environment. SITUATION AWARENESS

Enemy location Dismounted Infantry Solider selects ‘perch’ mode on personal ISS and selects location on ISS map. Perching Wingman determines optimal route based on current location, destination point, and topography (e.g., collision avoidance) and flies to selected location and lands. The remaining three airborne Wingmen reconfigure their locations to maintain optimal overwatch over the section. Search for enemy using 2.1 Listen / Look for Enemy Consideration of current location of section senses Type & strength of enemy members and terrain/building blind spots to determine optimal sensor coverage. The perching Wingman detects movement and automatically alerts the ISS of the soldier that deployed it. Alert includes supporting imagery which is automatically collected and annotated (e.g., target classification, direction of movement). Dismounted Infantry Solider confirms classification of the contact.

Current location of friendly troops Current location of friendly troops and 2.2 Monitor Location for Monitor current location of (MGRS) UVSs (including Wingman) accessed Friendly Troops friendly troops and UVSs Current location of friendly UVSs through ISS network and displayed on (MGRS) Dismounted Infantry Soldier’s ISS map.

2.3 Monitor Terrain Features Monitor terrain for ‘dead Terrain type and topography N/A

Page 65 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ zones’ Location of terrain ‘dead zones’

3. DISGUISE INTENDED Dismounted Infantry Soldier uses EW countermeasures to disguise intended movement and route of advancing MOVEMENT platoon.

Determine what EW Mission orders countermeasures are 3.1 Determine required to spoof enemy EW Wingman decoy payload and Location/Type of N/A operations. capability Decoy/Countermeasures Anticipated enemy course of action

Dismounted Infantry Solider selects ‘Decoy’ mode on personal ISS and the Wingman emits a brief radio transmission decoy to spoof enemy EW operations. Dismounted Infantry Solider selects ‘overwatch’ and the perching Wingman takes off and rejoins the swarm. Perching Deploy EW countermeasures Wingman determines optimal route to 3.2 Deploy to spoof enemy EW Wingman decoy payload status rejoin swarm based on current location, Decoy/Countermeasures operations. swarm location, and topography (e.g., collision avoidance). All four airborne Wingmen reconfigure their locations to maintain optimal overwatch over the section. Consideration of current location of section members and terrain/building blind spots to determine optimal sensor coverage.

Page 66 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.3.2.4 RoboMedic

The functional and task decomposition for the RoboMedic SRT use case is presented in Figure 12. The results of the task analysis are presented in Table 11

SOLDIER-ROBOT TEAMING USE CASE: ‘RoboDoc’

CONDUCT TACTICAL MAINTAIN LOCAL REACT TO ENEMY FIRE MANAGE CASUALTY REJOIN PLATOON MOVEMENT SITUATION AWARENESS

DETERMINE OPTIMAL LISTEN / LOOK FOR TAKE UP COVERING FIRE PREPARE TO RECEIVE REQUEST CASUALTY VERIFY POSITION ASSESS CASUALTY STABILISE CASUALTY EVACUATE CASUALTY ROUTE TO LOCATION OF ENEMY POSITION CASUALTY EVACUATION (MEDEVAC) PLATOON

DETERMINE DETERMINE OPTIMAL MONITOR LOCATION OF RECEIVE NOTIFICATION SEND 9-LINE MEDEVAC NOTIFY PLATOON OF MAINTAIN CHOSEN LOCATE ENEMY CATASTROPHIC CLEAR AIRWAY ROUTE TO CASUALTY FRIENDLY TROOPS OF INCOMING CASUALTY REQUES T INTENTION TO REJOIN ROUTE HAEMORRHAGE EXTRACTION POINT

MONITOR TERRAIN RETURN FIRE ON ENEMY MOVE TO CASUALTY DETERMINE BREATHING TRANSPORT CASUALTY MOVE TO PLATOON RESTORE BREATHING SEND MISTAT REPORT FEATURES POSITION COLLECTION POINT & VENTILATION TO EXTRACTION POINT LOCATION

RESTORE CIRCULATION MONITOR CASUALTY CEASE FIRE ON ENEMY DETERMINE (INCLUDING IV) & STEM VITAL SIGNS EN ROUTE POSITION CIRCULATION BLEEDING TO EXTRACTION POINT

DETERMINE DISABILITY ADJUST CASUALTY DUE TO NEUROLOGICAL ADMINSTER PAIN RELIEF TREATMENT EN ROUTE DETERIORATION TO EXTRACTION POINT

RDV WITH MEDEVAC DETERMINE EXPOSURE & KEEP CASUALTY WARM TEAM AT EXTRACTION EXAMINE FOR WOUNDS POINT

RESUPPLY MEDICAL PROVISIONS

Figure 12: Functional and Task Decomposition – ‘RoboMedic’

Page 67 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 11: Task Analysis – ‘RoboMedic’

Task Description Information Requirements Soldier-Robot Teaming ‘Features’

1. CONDUCT TACTICAL Platoon Combat Medic advancing towards objective (one tactical bound behind line of advance). MOVEMENT

Current location of Platoon Combat Medic (MGRS) Location of RoboMedic displayed on of Regular verification of Current location of RoboMedic Platoon Combat Medic’s ISS map. 1.1 Verify Position current position along (MGRS) Planned route (including waypoints) planned route. Route waypoint location (MGRS) accessed through ISS network.

Objective location (MGRS)

Current location of Platoon Platoon Combat Medic selects ‘auto follow’ Combat Medic (MGRS) mode on personal ISS and RoboMedic follows location of Platoon Combat Medic. Current location of RoboMedic (MGRS) RoboMedic compares location of Platoon Follow chosen route Combat Medic with planned route between planned waypoint Route waypoint location (MGRS) (accessed through ISS network) and alerts 1.2 Maintain Chosen Route according to terrain type of Objective location (MGRS) Platoon Combat Medic if deviating from topography planned route. RoboMedic determines optimal path across Terrain type and topography terrain to follow Platoon Combat Medic based on appreciation of terrain type and topography (including obstacle detection).

2. MAINTAIN LOCAL SITUATION Platoon Combat Medic maintains awareness of critical cues in their immediate environment. AWARENESS

2.1 Listen / Look for Enemy Search for enemy using Enemy location N/A

Page 68 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ senses Type & strength of enemy

Current location of friendly troops Current location of friendly troops and UVSs 2.2 Monitor Location for Monitor current location of (MGRS) (including RoboMedic) accessed through Friendly Troops friendly troops and UVSs Current location of friendly UVSs ISS network and displayed on Platoon (MGRS) Combat Medic’s ISS map.

2.3 Monitor Terrain Monitor terrain for ‘dead Terrain type and topography N/A Features zones’ Location of terrain ‘dead zones’

Platoon Combat Medic reacts to enemy fire by adopting covering position, locating enemy, and returning fire 3. REACT TO ENEMY FIRE until enemy neutralized.

Adopt firing position based 3.1 Take up Covering on suitability of ground Terrain type and topography N/A Position cover / shelter from enemy fire.

Locate any enemy within 3.2 Locate Enemy Enemy location N/A designated firing arcs

Returning fire on enemy 3.3 Return Fire on Enemy position with goal of Enemy location N/A Position neutralizing enemy.

3.4 Cease Fire on Enemy Cease firing on enemy if Status of enemy neutralized or receives order. N/A Position Order to ceasefire

Platoon Combat Medic receives casualty at Casualty Collection Point and begins assessment and stabilization of 4. MANAGE CASUALTY casualty, followed by medical evacuation (MEDEVAC) if required.

Page 69 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Platoon Combat Medic notified of casualty Receive Notification of over personal ISS by Tactical Casualty Casualty location (MGRS) Incoming Casualty Combat Care (TCCC) trained infantryman. 4.1 Prepare to Receive Location of casualty presented on ISS map. Casualty RoboMedic determines optimal path across Move to Casualty Collection Location of Casualty Collection terrain to follow Platoon Combat Medic Point Point (MGRS) based on appreciation of terrain type and topography (including obstacle detection).

Determine Catastrophic Casualty injury indicators and N/A Haemorrhage symptoms

Vital signs sensors attached to casualty are Determine Breathing and Casualty Vital Signs monitored and linked to Platoon Combat Ventilation Medic’s ISS.

Vital signs sensors attached to casualty are 4.2 Assess Casualty Determine Circulation Casualty Vital Signs monitored and linked to Platoon Combat Medic’s ISS.

Determine Disability due to Casualty injury indicators and N/A Neurological Deterioration symptoms

Determine Exposure and Casualty injury indicators and N/A Examine for Wounds symptoms

Casualty Vital Signs (and 4.3 Stabilise Casualty Clear Airway N/A symptoms)

Page 70 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Automated administration of ventilator support based on treatment initiated by Casualty Vital Signs (and Platoon Combat Medic. Monitored and Restore Breathing symptoms) controlled through touch-screen interface on RoboMedic and/or Platoon Combat Medic’s ISS.

Automated administration of IV based on Restore Circulation treatment initiated by Platoon Combat Casualty Vital Signs (and (including Intra-Venous Medic. Monitored and controlled through symptoms) saline) and Stem Bleeding touch-screen interface on RoboMedic and/or Platoon Combat Medic’s ISS.

Automated administration of pain relief medication based on treatment initiated by Casualty Vital Signs (and Platoon Combat Medic. Monitored and Administer Pain Relief symptoms) controlled through touch-screen interface on RoboMedic and/or Platoon Combat Medic’s ISS.

Heated blanket on RoboMedic. Monitored Casualty Vital Signs (and and controlled through touch-screen Keep Casualty Warm symptoms) interface on RoboMedic and/or Platoon Combat Medic’s ISS.

MEDEVAC submitted electronically through ISS network. Date, time, unit and location fields of report are automatically Send 9-Line MEDEVAC See MEDEVAC 9-line information 4.4 Request MEDEVAC completed. Other autocompletion of known Request fields in annex A.1 fields possible based on Radio Frequency Identification (RFID) tag worn by casualty (e.g., nationality, age of patient).

Page 71 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ A MISTAT report is submitted electronically through ISS network. Date, time, unit and location fields of report are automatically See MISTAT information fields in completed. Other autocompletion of known Send MISTAT10 Report annex A.2 fields possible based on RFID tag worn by casualty (e.g., nationality, age of patient) and vital signs monitoring (e.g., circulation, pulse, breathing).

Current location of RoboMedic (MGRS) ROBOMEDIC determines optimal route for Determine Optimal Route to evacuation (based on terrain) using current Casualty Extraction Point location Casualty Extraction Point location and casualty pick-up site from 9- (MGRS) Line MEDEVAC request. Terrain type and topography

Current location of RoboMedic (MGRS) ROBOMEDIC moves along optimal route for Transport Casualty to evacuation (based on terrain) using current Casualty Extraction Point location Casualty Extraction Point location and casualty pick-up site (from 9- 4.5 Evacuate Casualty (MGRS) Line report). Terrain type and topography

Platoon Combat Medic receives automatic Monitor Casualty Vital Signs notification of changes to casualty vital Casualty Vital Signs En Route To Extraction Point signs on personal ISS based on pre- configured thresholds.

Casualty Vital Signs Platoon Combat Medic can change Adjust Casualty Treatment administration of IV / medication by En Route to Extraction Point Treatment parameters / settings RoboMedic remotely using personal ISS.

10 MISTAT = Mechanism of Injury/illness, Signs and Symptoms, Treatment, Age and Time of wounding (see Annex A.2)

Page 72 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Current location of RoboMedic RDZ with MEDEVAC Team at (MGRS) N/A Extraction Point Casualty Extraction Point location (MGRS)

Medical expendables re-supplied on Resupply Medical Provisions Expended medical provisions RoboMedic based on known treatment of casualty (9-line and MISTAT).

5. REJOIN PLATOON RoboMedic rejoins platoon immediately after the casualty has been evacuated.

RoboMedic determines Current location of Platoon RoboMedic determines optimal path across 5.1 Determine Optimal optimal route back to Combat Medic (MGRS) terrain to follow Platoon Combat Medic Route to Location of location of Platoon Combat based on appreciation of terrain type and Platoon Current location of RoboMedic Medic (MGRS) topography (including obstacle detection).

Platoon Combat Medic notified over personal ISS by RoboMedic. RoboMedic Platoon Combat Medic is 5.2 Notify Platoon of Current location of RoboMedic current location presented on ISS map. notified that RoboMedic Intention to Rejoin (MGRS) Platoon Combat Medic accepts request wishes to rejoin platoon. from RoboMedic to rejoin using personal ISS.

Current location of Platoon Platoon Combat Medic selects ‘auto follow’ Combat Medic (MGRS) mode on personal ISS and RoboMedic follows location of Platoon Combat Medic. RoboMedic moves to Current location of RoboMedic 5.3 Move to Platoon RoboMedic moves along optimal path location of Platoon Combat (MGRS) Location across terrain to follow Platoon Combat Medic and rejoins platoon. Medic based on appreciation of terrain type Terrain type and topography and topography (including obstacle detection).

Page 73 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.3.2.5 Amazon

The functional and task decomposition for the Amazon SRT use case is presented in Figure 13. The results of the task analysis are presented in Table 12.

SOLDIER-ROBOT TEAMING USE CASE: ‘Amazon’

DETERMINE RESUPPLY TRANPORT SUPPLIES TO OPTIMISE RESUPPLY REPLENISH PLATOON REQUIREMENTS OF PLATOON ACCORDING ESTABLISH PLATOON HQ STRATEGY FOR PLATOON SUPPLIES PLATOON TO RESUPPLY STRATEGY

DETERMINE OPTIMAL DEPLOY PLATOON HQ TRACK EXPENDITURE OF TRANSIT TO PLATOON PLATOON RDV WITH CONDUCT PLATOON CONDUCT PLATOON RESUPPLY ROUTE & SHELTER & AMMUNITION RESUPPLY LOCATION RESUPPLY CONTAINER LOCAL RESUPPLY REMOTE RESUPPLY SCHEDULE COMMUNICATIONS

DETERMINE OPTIMAL UNLOAD RESUPPLY DETERMINE REMOTE DETERMINE PERIMETER TRACK EXPENDITURE OF LOCATE RESUPPLY SUPPLY LOADING CONTAINER AT RESUPPLY RESUPPLY AMMUNITION RESUPPLY SURVEILLANCE FOOD & WATER CONTAINER CONFIGURATION LOCATION REQUIREMENTS REQUIREMENTS

TRACK EXPENDITURE OF DETERMINE OPTIMAL ACCESS RESUPPLY RESUPPLY FOOD & ESTABLISH PERIMETER POWER (BATTERIES & RETURN TO DEPOT REMOTE RESUPPLY CONTAINER WATER SURVEILLANCE FUEL) ROUTE

TRANSPORT SUPPLIES TO TRACK EXPENDITURE OF RESUPPLY POWER REMOTE LOCATION MEDICAL SUPPLIES (BATTERIES & FUEL) ACCORDING TO ROUTE

TRACK REPAIR PARTS RESUPPLY MEDICAL REQUIREMENTS SUPPLIES

RESUPPLY REPAIR PARTS

Figure 13: Functional and Task Decomposition – ‘Amazon’

Page 74 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 12: Task Analysis – ‘Amazon’

Task Description Information Requirements Soldier-Robot Teaming ‘Features’

1. DETERMINE RESUPPLY REQUIREMENTS OF Platoon 2IC determines resupply requirements for the Platoon. PLATOON

1.1 Track Expenditure of Expenditure of Type and quantity of ammunition Tracking of equipment expendables (e.g., Ammunition ammunition, food & expended ammo, batteries, fuel, medical supplies) water, power 1.2 Track Expenditure of Food & through ISS network. (batteries & fuel), Quantity of food & water expended Water Expenditure of food and water tracked medical supplies, and through AI modeling. Repair parts 1.3 Track Expenditure of Power equipment repair Power (batteries & fuel expended) requirements tracked through equipment (Batteries & Fuel) parts are tracked, ‘vetronics’ (corrective maintenance) and/or collated by Platoon 1.4 Track Expenditure of Medical preventative maintenance schedule. 2IC, and resupply Medical supplies expended Supplies request sent to Resupply requirements are collated Logistics Personnel automatically and sent to Platoon 2IC’s ISS. Equipment parts to be replaced 1.5 Track Repair Parts at Battle Group Platoon 2IC revising resupply request and (including preventative maintenance Requirements Forward Operating forwards to Logistics Personnel at Battle schedule) Base. Group Forward Operating Base.

2. OPTIMISE RESUPPLY Logistics Personnel at Battle Group Forward Operating Base determine optimal resupply route, schedule, STRATEGY FOR PLATOON and transport loading to meet the Platoon’s resupply requirements.

Determine optimal Location of available inventory supply route according to location AI tools at HQ determine optimal re-supply 2.1 Determine Optimal Supply of units to be strategy for all assets under its purview, Route and Schedule resupplied by Location of units to be resupplied including schedule. transportation convoy.

Page 75 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Supplies are optimally loaded into Decision aids to optimize supply chain of 2.2 Determine Optimal Supply containers Order of units to be resupplied required supplies and loading on Loading Configuration depending on re- transportation vehicles. supply schedule.

3. TRANSPORT SUPPLIES TO Resupply container is transported to platoon resupply rendez-vous (RDV) point, as part of resupply convoy, PLATOON ACCORDING TO according to resupply strategy. RESUPPLY STRATEGY

Location of platoon resupply RDV Autonomous transportation vehicles Resupply transport point (MGRS) comprise a single convoy led by human 3.1 Transit to Platoon Resupply convoy transits to Resupply route waypoints (MGRS) operator in lead vehicle. Autonomous Location platoon resupply vehicle control to follow manned lead RDV point Location of resupply transport vehicle. (MGRS)

Resupply Container 3.2 Unload Resupply Container is unloaded at N/A N/A at Resupply Location platoon resupply location

Resupply transport Location of Depot (MGRS) Autonomous transportation vehicles convoy returns to Resupply route waypoints (MGRS) comprise a single convoy led by human Depot (following 3.3 Return to Depot operator in lead vehicle. Autonomous completion of all Location of resupply transport vehicle control to follow manned lead remaining scheduled (MGRS) vehicle. resupply stops)

4. REPLENISH PLATOON Containers are dropped off at specific locations and dismounted infantry platoon RDV with it. SUPPLIES

Page 76 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’

Locate Resupply Location of Resupply Container Resupply Container alerts Platoon 2IC of its Container (MGRS) arrival at resupply RDV point through ISS. Location of Platoon 2IC (MGRS) Resupply Container tracks location of approaching platoon through ISS network 4.1 Platoon RDV with Resupply Resupply Container security access and shares security access code with Container code Platoon 2IC through personal ISS. Access Resupply Container Platoon 2IC enters access code using ISS to get access to resupply container to remove supplies.

Resupply Ammunition

Resupply Food & Water

4.2 Conduct Platoon Local Resupply Power Resupply requirements for platoon N/A Resupply (Batteries & Fuel)

Resupply Medical Supplies

Resupply Repair Parts

Platoon sniper, located remotely, selects Determine Remote ‘resupply’ mode on personal ISS and 4.3 Conduct Platoon Remote Resupply Resupply requirements for sniper requests ammo, food, batteries and water. Resupply Requirements Request from sniper sent to 2IC ISS with recommendation for which of the four

Page 77 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ UAVs to send based on current UAV taskings, UAV battery level, weight of requested supplies, own location, and Location of sniper (MGRS) location of sniper. Platoon 2IC accepts the request through personal ISS.

Location of sniper (MGRS) Amazon UAV automatically determines Determine Optimal Location of Amazon UAV (MGRS) optimal (stealthy) route to sniper’s location Remote Resupply – making use of dead-ground to avoid Route Location of enemy (MGRS) detection by the enemy. Terrain type and topography

Platoon 2IC loads the recommended UAV Location of sniper (MGRS) with requested supplies and sends the UAV to location of sniper using ISS. Location of Amazon UAV (MGRS) Amazon UAV travels to location of enemy Transport Supplies to sniper, drops supplies, and returns to the Remote Location Location of enemy (MGRS) container, where it recharges and notifies According to Route Platoon 2IC through personal ISS. Flight of the Amazon UAV based on appreciation of Terrain type and topography terrain type and topography (including obstacle detection).

5. ESTABLISH PLATOON HQ Empty container is used as a shelter and contains based communication equipment to act as a temporary platoon HQ.

Platoon HQ shelter is 5.1 Deploy Platoon HQ Shelter & manually deployed, Platoon HQ communications joins local ISS N/A Communications and communications network. equipment initialized

Page 78 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Task Description Information Requirements Soldier-Robot Teaming ‘Features’ Platoon 2IC selects ‘Perimeter surveillance mode’ using personal ISS to deploy Requirements for Mission orders Amazon’s fleet of four UAVs to maintain surveillance is persistent perimeter surveillance around established to platoon HQ location. Amazon determines provide local security 5.2 Determine Perimeter that two (of the four) UAVs are required to for the platoon Surveillance Requirements execute the perimeter surveillance task and patrol base notifies Platoon 2IC of recommendation. considering terrain Terrain type and topography Manual selection of perimeter coordinates type and topography using ISS by Platoon 2IC is used by Amazon (dead ground). to determine optimal coverage by Amazon UAV fleet.

Perimeter Platoon 2IC accepts the recommendation surveillance is via personal ISS and Amazon deploys two 5.3 Establish Perimeter established to Perimeter surveillance parameters UAVs. Surveillance provide local security (MGRS) Self-synchronization of Amazon UAVs to for the platoon optimize coverage based on recharging patrol base. requirements and asset availability.

Page 79 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.3.3 Summary

Five SRT use cases were analysed using the MFTA technique to identify SRT system features, information requirements, intended user(s), and mission-related tasks. The level of analysis was sufficient to identify the main SRT system features required to support each use case. More detailed analyses will be conducted on a sub-set of the SRT use cases based on operational impact and technological feasibility as determined by the stakeholder community and the project scientific authority.

In summary, the following four classes of SRT system features were identified from the use case task analyses (see Figure 14).

USE CASE #1: USE CASE #2: USE CASE #3: USE CASE #4: USE CASE #5: ‘Mule’ ‘Wingman’ ‘Artemis’ ‘RoboDoc’ ‘Amazon’

Intelligent Tasking Intelligent Route Planning Intellig

• Minimal user control input (desired location) • Real-time optimisation of joint tasking (swarm) based on availability of assets (e.g., re-tasking, failures, attrition) • Automatic updates to user on tasking progress (incl. requesting permission to engage target)

Figure 14: Summary of SRT Features Identified from Use Case Task Analyses

1. Intelligent Tasking. This category of SRT features support the intelligent tasking of autonomous assets by the soldier. For example:

Page 80 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

a. Minimal user control input, such as entering a location or desired objective, leaving the robot to decide the best way of achieving the objective. b. Automatic real-time optimisation of multiple autonomous asset taskings (e.g., swarm behaviour) based on the availability of assets (e.g., due to re-tasking, failures, or attrition). c. The user is updated automatically by the system on task progress and kept ‘in the loop’ during engagements (e.g., permission to fire on a target). 2. Intelligent Route Planning and Following. This category of SRT features support the intelligent tasking of autonomous assets by the soldier. For example: a. Real-time tracking of the location of the autonomous asset and the intended destination (e.g., geographical location, location of user). b. Avoidance of obstacles without user involvement. c. Terrain appreciation (e.g., topology, terrain type) without user involvement. d. Automatic exploitation of dead ground to reduce probability of detection and exposure to enemy fire. 3. Intelligent Reporting. This category of SRT features support the intelligent reporting by autonomous assets to support the soldier. For example: a. Automatic reporting of contact detection and identification by autonomous assets to the intended user(s). b. Imagery collected by autonomous assets sent automatically to the intended user(s), with rudimentary annotations. c. Automatic completion of ISS report fields using knowledge from the ISS network (e.g., location, unit names), and information RFID (e.g. casualty information, equipment status, ammunition level) ISS reports include Situation Reports (SITREPs), CASEVAC requests and MISTAT reports. 4. Intelligent Logistics. This category of SRT features supports logistics personnel resupply dispersed units in the field. For example: a. Automatic tracking of ammunition expenditure. b. Automatic tracking of power/fuel usage. c. Predictive analytics for food, water and spare parts. d. Optimisation of resupply container loading and re-supply schedule.

3.4 Operational Requirements Summary

This section identifies the operational requirements that have been derived by the use case descriptions and task analysis presented in the previous section. These requirements are

Page 81 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

expressed as features, i.e. high-level requirements collectively contribute to satisfying the operational needs associated with capabilities of the OPSCON. That is to say, when implemented, they will assist the soldier in accomplishing tasks in the operational context identified as part of the present study.

In order to address the capability gap identified and described in the present study, there are three sets of required features that must be developed, and they correspond to three types of systems:

• Command and Control Information Systems (C2IS); • The ISS and subsystems; and • Robotic systems, in the form of different types of UVS.

The following sections present these feature sets.

C2IS requirements refer to BMS, logistics management systems and other operational systems used by HQ and any other C2IS that is not co-located with the ISS user. For example, there is a BMS that is available on the ruggedized laptop and on the TUI and these systems are part of the ISS that is described in Section 2.4.1. The robotic systems considered in the current study are limited to the UGV and UAV assets described in the use case scenarios for providing primarily combat support, logistics support and medical support. When a UVS is used in the statement of features, it implies that the feature is common to all unmanned vehicle types.

The operational requirements are organized as sets of features required to support the tasks identified in the task analysis. For each feature, the related task is identified as well as the category of the task, e.g. combat, logistics, C2, ISR etc. The feature description captures the operational requirement and the note section is used to provide additional information such as rationales, constraints or other contextual information.

3.4.1 Command and Control Information System Requirements

C2IS at HQ will be utilized to handle supply, maintenance and logistics support requests. Therefore, if the resulting actions related to these requests involves the use of robotic assets, then the C2IS must have the capabilities to allocate these assets in the most efficient manner, for example by determining the optimal supply strategy.

Table 13 presents features related to the command and control information system.

Page 82 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 13: Command and Control Information System Features

Feature Task Category System Description

HQ C2IS can determine FEATURE-001 optimal re-supply strategy PLAN-Supply- Supply Strategy Logistics HQ-C2IS for all assets under its Route-Schedule purview, including schedule.

HQ C2IS is equipped with FEATURE-002 decision aids to optimize PLAN-Supply- Supply Strategy Logistics HQ-C2IS supply chain of required Transportation supplies and loading on transportation vehicles.

3.4.2 ISS Requirements The focus of the current study is on identifying operational requirements for future versions the ISS in order to support the use of robotic assets in SRT scenarios. This section presents the set of features that supports the execution of the tasks identified during the task analysis and use case analysis presented in the previous section.

3.4.2.1 ISS Combat Support Requirements Table 14 presents features related to combat support services.

Table 14: ISS Combat Support Features

Feature Task Category System Description

FEATURE-003 ISS can join local ISS- COMMS-AD- Communications C2 ISS Network and identify itself HOC-Network as improvised HQ.

FEATURE-004 ISS can create and send REQUEST-FIRE- Communications C2 ISS request for weapons fire on On-Enemy- enemy position to UVS. Position

FEATURE-005 Unit commander can receive DEPLOY-UAV- surveillance plan on ISS and Communications C2 ISS Perimeter- can accept, reject or modify Surveillance it before dispatching UAVs.

FEATURE-006 Unit commander can request REQUEST-UAV- Communications C2 ISS perimeter surveillance of Perimeter- area specified using ISS TUI. Surveillance

Page 83 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.4.2.2 ISS Logistics Requirements

Table 17 presents features related to logistics.

Table 15: ISS Logistics Features

Feature Task Category System Description

Unit commander queries system to determine which FEATURE-007 UAV assets are available and PLAN-Sniper- Resupply Logistics ISS have the required resources Resupply to transport supplies based on resupply request.

FEATURE-008 Unit commander can receive REP-LOGREP- Resupply Logistics ISS LOGREP and view content Unit-Commander on TUI.

FEATURE-009 Sniper can use ISS to request REQUEST- Resupply Logistics ISS resupply for munitions, food, LOGREQ-Sniper water and batteries.

Based on auto-generated report from UGV, unit FEATURE-010 commander can generate REQ-LOGREQ- Resupply Logistics ISS and send logistics supply UGV- request for parts and Maintenance maintenance to FOB/HQ over ISS-Network.

FEATURE-011 Unit commander can send a REQUEST- Resupply Logistics ISS request for supplies over LOGREQ-Unit- ISS-Network. Commander

ISS processes report from FEATURE-012 SA- UGV and displays supplies DISPLAY- Resupply Logistics ISS ETA and location on TUI SUPPLIES-ETA map and show a notification message.

ISS will automatically FEATURE-013 SA- process report from UGV DISPLAY- Resupply Logistics ISS and display security access SUPPLIES- code in preparation of Security-Code container arrival.

Page 84 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.4.2.3 ISS Medical Support Requirements

Table 16 presents features related to medical support.

Table 16: ISS Medical Support Features

Feature Task Category System Description

ISS can display casualty FEATURE-014 SA- Manage status on TUI based on DISPLAY-CASUALTY- Medical Support ISS Casualty information collected by Information UGV.

FEATURE-015 User can complete MISTAT Manage REPORT-MISTAT- Medical Support ISS 9-line message using TUI Casualty Write-Send and send via ISS-Network11.

User can complete FEATURE-016 Manage MEDEVAC 9-line message REQUEST-MEDEVAC- Medical Support ISS Casualty using TUI and send via ISS- Write-Send Network.

Combat medic can use TUI FEATURE-017 to make UGV requests for REQUEST-UGV- Manage Medical Support ISS modifications to parameters CASUALTY-Adjust- Casualty of the heated blanket that is Heated-Blanket covering the casualty.

FEATURE-018 Combat medic can use TUI REQUEST-UGV- Manage Medical Support ISS to make UGV requests to CASUALTY-Apply- Casualty apply ventilator to casualty. Ventilator

Combat medic can use TUI to make UGV requests to FEATURE-019 Manage modify the parameters of REQUEST-UGV- Medical Support ISS Casualty the IV being administered CASUALTY-Modify-IV to the casualty, including pain relief medication.

Combat Medic can receive casualty notification FEATURE-020 SA- Manage updates over ISS-Network DISPLAY-CASUALTY- Medical Support ISS Casualty when changes to vital signs Notification-Update are larger than specified threshold.

11 Majority of fields are completed automatically based on soldier RFID tag and UGV sensor data.

Page 85 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Feature Task Category System Description

FEATURE-021 Combat medic can REQUEST- Manage Medical Support ISS authorize or deny UGV AUTHORIZATION- Casualty request to rejoin unit. UGV-Return-to-Unit

ISS automatically displays FEATURE-022 SA- Manage notification and location of DISPLAY-UGV-En- Medical Support ISS Casualty returning UGV on TUI and Route map.

Page 86 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.4.2.4 ISS UVS Navigation Control Requirements

Table 17 presents features related to UVS navigation control.

Table 17: ISS UVS Navigation Control Features

Feature Task Category System Description

FEATURE-023 UGV can advance behind REQUEST-NAV-UGV- Conduct Tactical Navigation ISS line-of-advance in support Autofollow-Med- Movement of combat operations12. Support

FEATURE-024 UGV can execute a request Conduct Tactical REQUEST-UGV-NAV- Navigation ISS to automatically follow a Movement Autofollow-Activate specific ISS.

FEATURE-025 ISS automatically monitors REQUEST-UGV-NAV- Conduct Tactical deviation w/r/t planned MR Navigation ISS Autofollow-Deviation- Movement and generates alert if Alert threshold is reached.

FEATURE-026 PLAN- Conduct Tactical UGV MR can be modified UGV-Modify-Mission- Planning ISS Movement using TUI. Route

3.4.2.5 ISS Reporting Requirements

Table 18 presents features related to reporting.

Table 18: ISS Reporting Features

Feature Task Category System Description

Combat Medic can generate FEATURE-027 React to Enemy and send health status of REPORT-Health- Reporting ISS Fire self or specified members of Status unit.

ISS automatically sends a FEATURE-028 Conduct Post- summary of type and REPORT-LOGREP- Engagement Reporting ISS quantity of ammunition for Munitions-Summary Reorganisation each section member over ISS-Network.

12 Requires that UGV receives localized section lead position. UGV must be able to negotiate rough terrain based on information of terrain type and topography.

Page 87 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Feature Task Category System Description

Soldier can complete and FEATURE-029 Conduct Post- send SITREP using TUI that REPORT-SITREP- Engagement Reporting ISS includes autocomplete Write-Send Reorganisation functions for date, time, unit and location.

ISS can automatically estimate the food and water FEATURE-030 consumption of the soldier REPORT-LOGREP- Resupply Soldier Reporting ISS and can send a logistic Food-Water-Tracking report at regular intervals over ISS network.

ISS can automatically detect FEATURE-031 type and quantity of REPORT-LOGREP- Resupply Soldier Reporting ISS munitions that were fired Munitions-Tracking and send reports at regular intervals over ISS network.

ISS can automatically estimate battery FEATURE-032 consumption of the ISS and Resupply Soldier Reporting ISS REPORT-LOGREP- sends a logistic report at Power-Tracking regular intervals over ISS network.

Combat Medic can automatically receive a FEATURE-033 SA- casualty update notification DISPLAY-CASUALTY- Manage Casualty Reporting ISS over ISS-Network, issued by Notification Tactical Casualty Combat Care.

3.4.2.6 ISS Situation Awareness Display Requirements

Table 19 presents features related to the display of SA information.

Page 88 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 19: ISS Display Features

Feature Task Category System Description

ISS can automatically display FEATURE-034 SA- Situation vital signs received from DISPLAY-CASUALTY- Manage Casualty ISS Awareness UGV over ISS Network on Vital-Signs combat medic's TUI.

ISS can automatically FEATURE-035 SA- React to Enemy Situation retrieve munition status and DISPLAY-Munitions- ISS Fire Awareness display on TUI or ruggedized Status laptop.

FEATURE-036 SA- TUI map display can be Conduct Tactical Situation DISPLAY-CASUALTY- ISS automatically updated to Movement Awareness Map-Location show location of casualty.

FEATURE-037 SA- Current MGRS location of Conduct Tactical Situation DISPLAY-Current- ISS section lead can be Movement Awareness Location displayed on TUI.

ISS can automatically Maintain Local analyze terrain type and FEATURE-038 SA- Situation Situation ISS topography and determine DISPLAY-Dead-Zone Awareness Awareness dead zones that can then be displayed on TUI map.

FEATURE-039 SA- Maintain Local Situation ISS can display friendly force DISPLAY-Friendly- Situation ISS Awareness unit locations. Troop-Locations Awareness

FEATURE-040 SA- Maintain Local Situation ISS can display friendly force DISPLAY-Friendly- Situation ISS Awareness UVS locations. UVS-locations Awareness

Current MGRS location of FEATURE-041 SA- UGV can be received over Conduct Tactical Situation DISPLAY-UGV- ISS ISS-Network and Movement Awareness Current-Location automatically displayed on TUI13.

UGV planned mission FEATURE-042 SA- Conduct Tactical Situation objective can be received DISPLAY-UGV- ISS Movement Awareness over ISS-Network and Mission-Objective displayed on TUI or

13 In the absence of a connection to ISS network, UGV should provide location directly to ISS via PAN.

Page 89 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Feature Task Category System Description Ruggedized laptop via BMS application14.

ISS can receive planned UGV FEATURE-043 SA- Conduct Tactical Situation MR from ISS-Network and DISPLAY-UGV- ISS Movement Awareness display it on TUI Tactical Mission-Route Map15.

14 MR WP using MGRS. 15 UGV planned MR is retrieved from LCSS and section lead location is available from ISS. If Network connection is not available, UGV communicates directly with ISS.

Page 90 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3.4.2.7 ISS Sensor Requirements

Table 20 presents features related to sensors.

Table 20: ISS Sensor Features

Feature Task Category System Description

ISS can process UVS sensor FEATURE-044 SA- data and automatically Maintain Local SENSOR-Enemy- Sensing ISS detect enemy location from Awareness Detection sensor imagery or based on enemy fire.

ISS can process UVS sensor FEATURE-045 SA- Maintain Local data and automatically SENSOR-Enemy-Size- Sensing ISS Awareness detect enemy size and Estimation strength.

3.4.3 Robotic System Requirements

Table 21 presents features relating to the robotic systems.

Table 21: Robotic System Features

Feature Task Category System Description

Multi-UAV system can analyze surveillance request FEATURE-046 PLAN- and calculate number of UAV-Perimeter- Surveillance ISR UAV UAVs required to meet Surveillance requirements and then send recommended plan to unit commander.

Multiple UAV system performing surveillance can FEATURE-047 LOG- automatically synchronize UAV-Recharge- Surveillance ISR UAV periodic recharging activities Synchronization to maximize coverage based on asset availability.

Page 91 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Feature Task Category System Description

UAV system can determine the optimal route to sniper's FEATURE-048 PLAN- location. Resupply Logistics UAV Supply-Route NOTE: Must avoid being detected by enemy, e.g. by making use of dead ground.

UAV can automatically FEATURE-049 LOG- return to UGV container Maintenance Logistics UAV UAV-Recharge after supply mission for recharging.

UGV can generate and send FEATURE-050 a summary of required parts REPORT-LOGREP- Maintenance Logistics UGV and maintenance to platoon UGV-Parts- commander over ISS- Maintenance Network.

UGV can determine its own FEATURE-051 LOG- predictive/preventive, UGV-Maintenance- Maintenance Logistics UGV scheduled maintenance and Repair repairs.

UGV can generate and send FEATURE-052 a report with supplies ETA Transport REPORT-LOGREP- Logistics UGV that will be sent over ISS- Supplies Arrival Network to ISS of specified unit.

Container UGV can track FEATURE-053 TRACK- Transport destination unit location and Logistics UGV UNIT-Location Supplies provide security access code with unit via ISS PAN16.

UGV transport vehicle can FEATURE-054 operate in auto-follow mode TRANSPORT- Transport w/r/t other UGV. Lead UGV Logistics UGV SUPPLIES-UGV- Supplies can be configured for auto- Convoy follow mode of human- operated lead vehicle.

16 Container that is dropped off can only be opened using security access code.

Page 92 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Feature Task Category System Description

FEATURE-055 UGV can drop-off container TRANSPORT- Transport Logistics UGV at pre-specified RDV SUPPLIES-UGV- Supplies location. Convoy-Dropoff

FEATURE-056 SA- Medical UGV touchscreen can display DISPLAY-CASUALTY- Manage Casualty UGV Support casualty status information. Information-On-UGV

Combat medic requests that FEATURE-057 Medical UGV modify the heated CASUALTY-Adjust- Manage Casualty UGV Support blanket temperature using Heated-Blanket UGV Touchscreen.

Combat medic requests that FEATURE-058 Medical UGV apply ventilator to CASUALTY-Apply- Manage Casualty UGV Support casualty using UGV Ventilator Touchscreen.

FEATURE-059 UGV is equipped with a CASUALTY- Medical Manage Casualty UGV remotely controllable heated Equipment-Heated- Support blanket. Blanket

UGV is equipped with an FEATURE-060 intravenous system that can Medical CASUALTY- Manage Casualty UGV automatically administer an Support Equipment-IV IV, once inserted in casualty17.

Combat medic can make a request using UGV Touchscreen for the UGV to FEATURE-061 Medical Manage Casualty UGV modify the parameters of CASUALTY-Modify-IV Support the IV being administered to the casualty, including pain relief medication.

UGV is equipped with a FEATURE-062 Medical ventilator to restore CASUALTY- Manage Casualty UGV Support breathing when initiated by Ventilator-Equipment combat medic.

17 It does not need to be changed manually and flow can be adjusted, and solution modified, as required.

Page 93 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Feature Task Category System Description

FEATURE-063 NAV- The UGV can optimize the Medical MEDEVAC-Minimize- Manage Casualty UGV trajectory and speed to Support Shock minimize shock to casualty.

FEATURE-064 LOG- Medical UGV automatically makes UGV-RESUPPLY- Resupply UGV UGV Support request to replace supplies. Medical-Supplies

Upon completing MEDEVAC, FEATURE-065 UGV can determine optimal Medical REQUEST-NAV- Manage Casualty UGV path to rejoin unit and Support RETURN-TO-UNIT request confirmation from combat medic18.

UGV can automatically FEATURE-066 NAV- Conduct Tactical Medical calculate the optimal path Optimal-Path- UGV Movement Support based on topography, Determination obstacles and terrain type.

UGV can navigate to FEATURE-067 NAV- casualty location upon Manage Casualty Navigation UGV MEDEVAC-Location receiving a MEDEVAC request19.

FEATURE-068 UGV is equipped with SENSOR-CASUALTY- Manage Casualty Sensing UGV sensors to identify Hemorrhage hemorrhaging.

FEATURE-069 UGV is equipped with SENSOR-CASUALTY- Manage Casualty Sensing UGV sensors to identify Neurological neurological deterioration.

FEATURE-070 UGV is equipped with SENSOR-CASUALTY- Manage Casualty Sensing UGV sensors to measure casualty Vital-Signs vital signs.

FEATURE-071 UGV is equipped with SENSOR-CASUALTY- Manage Casualty Sensing UGV sensors to identify wounds. Wounds

UVS can execute a weapons FEATURE-072 FIRE- React to Enemy Combat UVS fire request on an enemy On-Enemy-Position Fire position.

18 Request should include MR and ETA. 19 Should require additional confirmation input from combat medic.

Page 94 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Feature Task Category System Description

FEATURE-073 Maintain Local UVS can automatically REPORT-UVS- Situation Reporting UVS publish its own location over Location Awareness ISS Network.

Page 95 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

4. FUTURE WORK

4.1 Next Steps for Requirements Development

The work reported on in this document represents a first iteration in the first two steps of CD&E: concept discovery and concept refinement. The requirements that have been produced should now be the subject of a concept assessment activity in order to determine whether further refinement is required or whether prototype development activities are required to further define and/or validate the operational concept. The requirements specified in this document dictate the need to further define the broader system in which the ISS is integrated. The current report addresses the need to establish the basic feature set and therefore the derivation of requirements from this feature set will be the object of future work. The following sections are placeholders for tasks required to complete the operational requirements document and for information that will be the outputs of the HFE experimentation, followed by the next phase of analysis:

• System Performance Parameters. Identify system performance parameters. Identify KPPs placing an asterisk in front of the parameter description, as part of concept refinement and prototype definition activities that build upon the requirements specified in this document. • Interoperability Requirements. Identify all requirements for the system to provide data, information, materiel, and services to and accept the same from other systems, and to use the data, information, materiel, and services so exchanged to enable them to operate effectively together. These requirements will depend on the specific protocols and systems with which the ISS must interoperate. • Soldier-Robot Interface Requirements. Consider the broad cognitive, physical, and sensory requirements for the operators, maintainers, or support personnel that contribute to, or constrain, total system performance. To that end, the next step will be to validate the SRT use cases storyboards within the wider stakeholder community and create a composite mission scenario upon which more detailed requirements analyses can take place (Hou et al, 2012). • Logistics and Readiness. Determine the requirements for the system to be supportable and available for operations. Provide performance parameters for availability, reliability, system maintainability, and software maintainability. • Other System Characteristics. Characteristics that tend to be design, cost, and risk drivers.

4.2 System Support

Page 96 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

This section describes the support objectives for initial and full operational capability and considers how systems will be interfaced, how they will be transported and stored in facilities. System support considerations should include the configuration management approach (e.g. for the software components), the repair, scheduled maintenance of the robotic assets and user support (such as embedded help, on-line help, and training. In particular, the following issues need to be addressed:

• Maintenance. Identify the types of maintenance to be performed and who will perform the maintenance. Describe methods for system upgrades and inserting new assets, e.g. control of additional robotic platforms. It is also necessary to consider post- development software support requirements, including evolving cyberthreats and the associated need to update and test the system. • Support Equipment. It is necessary to define the standard support equipment to be used by the system and to consider the need for special test equipment or software development environment. • Training. It is necessary to determine how the training will ensure that users are certified as capable of operating and using the proposed system based the determination of what competencies are required. • Transportation and Facilities. It is necessary to determine how the system will be transported to the field and what facilities will be needed for staging and training.

4.3 Force Structure for Expected Employment

The intent of this section is to estimate the number of systems needed, including spares and training unit, and identify which CA organisations and units will employ the system. The current report focuses on refining the operational concept and defining the system requirements and therefore consideration of force structure for expected employment was out of scope and should be considered as part of future studies. However, a top-level depiction of a generic force structure at the dismounted infantry platoon can be compiled based on combining all unmanned assets identified by the SRT use case storyboards. Taken as a whole, the full complement of UAVs and UGVs in the platoon would be (see Figure 15):

• Three ‘Mule’ UGVs (one per section); • One ‘Artermis’ UGV (one per heavy weapons detachment); • One ‘RoboMedic’ UGV (one per Platoon Combat Medic); • Thirteen ‘Wingman’ micro-UAVs (one per fire team and Platoon Commander); and, • Four ‘Amazon’ UAVs (including the re-supply container from the autonomous logistics vehicle).

Page 97 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

4.4 Schedule

The intent of this section is to define target dates for system availability. However, the current report focuses on refining the operational concept and defining the system requirements and therefore consideration of schedule was out of scope and should be considered as part of future studies.

4.5 System Affordability

The intent of this section is to identify a threshold or objective target price to DND at full-rate production. However, the current report focuses on refining the operational concept and defining the system requirements and therefore consideration of system affordability was out of scope and should be considered as part of future studies.

Page 98 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

g g g g

Rifleman Rifleman / Grena Section 2IC Rifleman Rifleman / Grena Corporal / Private Corporal / Priva Master Corpora l Corporal / Private Corporal / Priva C7 Assault Rifle C7 Assault Rifl C7 Assault Rifle C7 Assault Rifle C7 Assault Rifl w/ M203 grena w/ M203 grena launcher launcher

g g g g

ection Commander Rifleman Rifleman / Grena Section 2IC Rifleman Rifleman / Gre Sergeant Corporal / Private Corporal / Priva Master Corpora l Corporal / Private Corporal / Pr C7 Assault Rifle C7 Assault Rifle C7 Assault Rifl C7 Assault Rifle C7 Assault Rifle C7 Assault R w/ M203 grena w/ M203 gre launcher launcher

g g g g

ection Commander Rifleman Rifleman / Grena Section 2IC Rifleman Rifleman / Grena Sergeant Corporal / Private Corporal / Priva Master Corpora l Corporal / Private Corporal / Priva C7 Assault Rifle C7 Assault Rifle C7 Assault Rifl C7 Assault Rifle C7 Assault Rifle C7 Assault Rifl w/ M203 grena w/ M203 grena launcher launcher

Artemis Gunner Assistant Gunner toon Commander Medic Platoon 2IC Sergeant Corporal / Private eutenant / Captain Sergeant Wa rrant Officer C7 Assault Rifle C7 Assault Rifle C7 Assault Rifle C7 Assault Rifle C7 Assault Rifle

Figure 15: Summary of Platoon Level Unmanned Assets

Page 99 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

5. CONCLUSIONS

The study reported on in this document explored how the existing ISS prototype system could be utilized and possibly extended to include the capability for exploiting robotic assets as part of a future SRT concept that has been identified by the CAF. It represents a first iteration in the first two steps of CD&E: concept discovery and concept refinement. Consistent with emerging concepts from the CAF for CE, the underlying goals are to improve mission effectiveness and increase force agility through the judicious and efficient use of robotics assets and autonomous systems.

Although the high-level CE concepts have been described, specific use cases and operational requirements have not yet been established for mission scenarios involving SRT in the context of CE CONOPS. As a CD&E activity, this study therefore focused on identifying stakeholder needs, defining the capability gap and on identifying potential use cases in order to propose a set of preliminary operational requirements that can then be vetted through stakeholder review and experimentation.

In parallel, the development of future SRT capabilities will pose technical challenges in terms of applying emerging technology and developing new technology to address existing gaps. Therefore, the current study includes an analysis of potential technology gaps and technical challenges and their impact on future systems and capabilities development.

As a preliminary version of an ORD, the present report proposes a mission context, a high-level description of five relevant use-cases, a task analysis and a set of features required to satisfy the use cases. The features involve primarily extensions to the ISS but also include requirements for robotic assets and autonomous systems and C2IS.

Recommendations for future work include vetting the concepts, use cases and feature sets through stakeholder review and HFE in order to refine and extend the ISS/SRT concept. This future work should provide the necessary information to create a more complete version of the present ORD and establish a preliminary version of the ISS/SRT statement of requirements.

Page 100 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

REFERENCES

Aral, S. (2019). Analytics in Defence: Emerging Digital Transformation and Empowering All Leaders. JCSP 45, Master of Defence Studies, Canadian Force College 2019.

AFCEA International (2018). Robots Lighten the Load for Marines. Retrieved in 2020 from https://www.afcea.org/content/robots-lighten-load-marines.

Bartik, J., et al. (2019). Autonomy Strategic Challenge Allied IMPACT Final Report. TTCP Technical Report, TR-ASC-01-2020.

Cain, M. (2019). Optimising Human Control of Virtual Robotic Swarms. Unpublished PowerPoint presentation.

Canadian Army Land Warfare Centre (2019). Close Engagement. Land Power in the Age of Uncertainty – Evolving Adaptive Dispersed Operations. Army Publishing Office, Kingston, Ontario. ISBN-978-0-660-27740-0.

Choi (2019). Reducing Soldier Burden using Autonomous Vehicles. Department of Applied Military Science, Royal Military College of Canada.

Collier, J., & Hou, M. (2019). DRDC Human Robot Teaming Interests. Internal DRDC document. 2 April 2019.

Department of National Defence (DND) (2011a). Soldier Systems Technology Roadmap – Capstone Report and Action Plan. Government of Canada, 2011.

Department of National Defense (2011b). MIL-STD-46855A, Department of Defense Standard Practice: Human Engineering Requirements for Military Systems, Equipment, and Facilities.

Directorate of Land Requirements 2 (2019). DLR 2 Innovation Program: Decision-Action Cycle Optimisation and Technology Adoption. Unpublished PowerPoint document. 27 February 2019.

Endsley, M. R. (2017). Toward a theory of situation awareness in dynamic systems (pp. 9-42). In Situational awareness. Routledge.

Gilmore, C., Chaykowsky, M., & Thomas, B. (2019). Autonomous Unmanned Aerial Vehicles for Blood Delivery. RAND Corporation technical report. ISBN: 978-1-9774-0346-9.

Global Security (2020). Squad Multipurpose Equipment Transport SMET. Retrieved from https://www.globalsecurity.org/military/systems/ground/smet.htm

Government of Canada (2018). IDEaS Innovation Networks: List of accepted Letters of Intent for the First Call for Proposals for Advanced Materials Micro-nets. Retrieved from

Page 101 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

https://www.canada.ca/en/department-national-defence/programs/defence-ideas/list- accepted-letters-proposals.html

Heffner, K. (2020). A Design Methodology for Intelligent Adaptive Systems: Case Study for the Authority Pathway for Weapon Engagements. Document Number DND-1144.1.1-01, April 2020.

Hou, M., Hillier, P., Ste-Croix, C., & Angel, H. (2012). A preliminary statement of requirements for a micro aerial vehicle system. DRDC technical report TR2012-043. April 2012.

Hou, M., & Kobierski, R. (2015a). The Arctic Composite Scenario: A Manned-Unmanned Aerial Vehicle Teaming Concept. DRDC-RDDC-2015-R241.

Labbé, P., Bowley, D., Comeau, P., Edwards, R., Hiniker, P., Howes, G., Kass, R., Morris, C., Nunes, V., & Vaughan, J. (2006). Guide for Understanding and Implementing Defense Experimentation GUIDEx The Technical Cooperation Program. 10.13140/2.1.4937.6648.

MITRE (2014). Systems Engineering Guide. Retrieved from: http://www.mitre.org/sites/default/files/publications/se-guide-book-interactive.pdf

Rheinmetall Defence Canada (2020). Rheinmetall Mission Master. Retrieved from https://www.rheinmetall.ca/en/rheinmetall_canada/systemsandproducts/electronicsyste ms/unmanned_vehicles/index.php

Serrano, D. (2015). Key Initiatives for Interoperability through Standardization – Applicability to Small Unmanned Vehicles. NATO Science and Technology Organisation Report STO-EN- SCI-271, 2015

Stimpert, S. (2014). Lightening the load of a USMC rifle platoon through robotics integration. Master’s Thesis, Naval Postgraduate School, Monterey, CA. (June 2014).

Tack, D. & Angel, H. (2005). Cognitive Task Analysis of Information Requirements in Dismounted Infantry Operations. DRDC Contract Report CR-2005-057.

TRADOC (1994). Force XXI Operations, TRADOC PAM 525-5. United States Department of the Army, Washington DC.

Page 102 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

ANNEX A. INFORMATION REQUIREMENTS

A.1 9-LINE MEDEVAC REQUEST TEMPLATE

Page 103 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

A.2 MISTAT REPORT TEMPLATE

Page 104 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

A.3 SITREP REPORT TEMPLATE

Page 105 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

ANNEX B. ANALAYSIS ON TECHNICAL CHALLENGES OF SRT OPERATIONS

Based on SRT concepts and use cases, a review of the state-of-the-art technical development is provided. Under each development, its high-level technical description is summarized, and the relevance to the SRT operations is also presented. In addition, the technical challenges in supporting SRT operations are analyzed.

B.1 SRT TECHNICAL DEVELOPMENT

The concept of SRT involves manned and unmanned mobile assets (air, land, sea, space) that interoperate to pursue a common (military) mission. In a generally well accepted platform, the manned asset commands and control the unmanned assets (as well as their mission payloads), where the unmanned assets may be given a certain level of authority (autonomy) to carry out their mission tasks. The key technology can be identified under the category of human machine systems (HMS), where two-way interactions indicate distinctive manned (human) assets and unmanned (machine) assets respectively. From human to machine, manned assets play the role of command and control (C2), i.e., decision-making, supervision, that can be understood by the machine unambiguously; from machine to human, information collected will assist the human assets in the decision-making process, with acceptable task load to human operators, the information needs to be intelligent from human’s perspective, the information will also support unmanned assets autonomous (with a certain level of authority) tasks without human interference. This concept is captured by the term of intelligent adaptive systems (IAS) (Hou et al, 2015b).

An alternative term of SRT is often referred to as manned-unmanned teaming (MUM-T). It stands for the close cooperation of manned and unmanned aircraft, in which each of these takes over the role it is best at. “This approach shall maximize the benefit of UAVs, compensate their weaknesses, and lower the risk to manned assets. In a MUM-T scenario, the human UAV operator is typically located inside a manned aircraft that is participating in the mission, allowing quick reaction to change the course of the operation, because the corresponding communication overhead is kept low. In the MUM-T configuration, the pilot in command (PIC), (i.e., the pilot non-flying and commander of a manned transport helicopter, is entitled to control multiple UAVs, which support the mission by taking over reconnaissance tasks). With UAV guidance concepts that are in use today, the main problem in this configuration is the overwhelming task load of the UAV operator” (Strenzke et al, 2011).

In terms of technical challenges in SRT operations, we focus on the concept of human machine intelligent and adaptive interactions, and identify 10 technical areas of interest:

1) Mission planning: the mission planning technologies address the human machine cooperative mission profiling and decision making process;

Page 106 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2) Capacity and workload model: this is a specialized domain in HMS of discussing the human capacity assessment so as to deliver acceptable task load to human operator; 3) Cognitive automation: as the term suggests, cognitive automation stresses the cooperative automation between human and machines; while 4) Human-autonomy interaction: it establishes a more general frame with emphasis on interface design; 5) Sensor and perception management system: it is customized in SRT operations to illustrate how sensors and sensing techniques are managed to support MUM-T; 6) Guidance, navigation and control (GNC): after the discussions of adaptive interactions, GNC provides enabling technologies to integrate the concept of human machine interactions; 7) Multi-UxV autonomy: one of the promising features that future SRT will bring is the human-multiple machines interactions, where the multi-machine system autonomy provides the supporting foundation for it; 8) Human-swarm intelligence: based on the multi-UxV autonomy, it is possible to address the adaptive and intelligent interactions between human operators and swarming robotic systems; 9) Network-centric navigation and control: like traditional GNC technologies, we would like to point out the network-oriented GNC to enable the human-swarm operations; and finally 10) Artificial intelligence (AI): AI is identified as the potentially disruptive technology for future SRT development.

B.1.1 Mission Planning

A typical SRT scenario or use case contains both soldier and robot(s) tasks to support a mission. The technical development of mission planning involves planning of soldier tasks; coordinate planning of robot(s) tasks; and assessment and re-planning.

For example, a reference mission is given in Heilemann et al (2020) for a troop transport or air assault mission of a single manned transport helicopter (representing a squadron of about four helicopters), that are supported by a smaller number of rotorcraft UAVs (e.g. three). In this constellation, the helicopter is responsible for the transport and the support of the ground troops, whereas the UAVs shall acquire reconnaissance information about the helicopter routes and landing sites located in hostile territory in real-time (Strenzke et al, 2011). In terms of mission planning, the helicopter crew has to plan and re-plan the mission. Hence, the crew must be able to upload mission data into the helicopters mission computer, i.e. mission-relevant locations and times as well as briefed constraints (which corridor to prefer, where to land). Then, the crew has to have the possibility to plan the mission for all aircraft. The mission computer

Page 107 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

supports the crew in this task by calculating a flight plan (flight log & communication plan) for the manned helicopter. In the aforementioned cases of mission re-planning, the crew also needs the support by the helicopter mission computer. From this example, a successful technical development of mission planning to support SRT operations shall have the following requirements or features:

1. To have an efficient centralized or distributed architecture. 2. To develop a semi-automatic mission planner that addresses the following key aspects (Schmitt & Schulte, 2015): a. The mission planner shall optimize mission performance, i.e. the overall mission plan shall comply with given standards and optimality criteria; b. The mission planner shall ensure adequate pilot workload, i.e. the cooperation shall ensure that the pilot is supported adequately to the current tactical situation as well as to the situation in the cockpit; c. The mission planner shall ensure pilot plan and situation awareness, (i.e., the pilot shall be aware of the current situation and upcoming decisions at all times); and d. The human pilot shall ensure full world knowledge, (i.e., the cooperation shall ensure that the limited world model of the automation can be enlarged by the pilot). 3. Given a complex planning application and only a single human operator, an assisting automation is necessary to increase overall mission performance and situation awareness. The concept of the Assistant System will (Brand & Schulte, 2015): a. fulfill mission objective; b. minimize risks for human and equipment; c. support crew works on most urgent and important task; and d. balance crew workload. 4. To develop communication protocol, allowing smooth information transition (e.g., sending commands to robots, or receiving information collected from robots). 5. To optimize plan with optimal task assignment, allocation, and best support to the missions and to conduct re-planning in real-time. 6. To have an acceptable human workload. Solving a complex planning problem as described may result in excessive human workload, especially when the planning task is combined with other tasks arising, for example, the task of sensor-management.

The context “Human Supervisory Control” (HSC) illustrates the concept of mission planning architecture. It refers to the relationship between a human supervisor and his human

Page 108 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

subordinates in hierarchical structures. The human operator does not act as a traditional outer- loop controller for the automated system, but rather acts as a supervisor, who gives low- frequent, intermittent commands (Clauss et al, 2013). The supervisor performs five generic functions in an automated system: plan, teach, monitor, intervene, and learn. UAV Capability Management includes (Clauss et al, 2013):

• Flight management; • Route planning; • Data link management; and • Perception management.

One of the technical development integrating the mission planner and assistant system is the so-called mixed-initiative planner assistant (MIPLA) (Heilemann, 2020). There are three key pillars necessary for a MIPLA: i) a solver library; ii) a model of human-system interaction, and iii) a human behavior model. It is developed to enable two key points:

1. The guidance of the UAVs at scalable delegation levels: team, mission, parameter based. 2. Integration of the human’s tasks into the mission planning to proactively avoid mission phases with excessive mental workload.

Regarding the assistant system, a cognitive assistant system is introduced (Schmitt & Schulte, 2015). “An assistant system shall know and understand the mission objective and make active contributions to reach it by cooperating with the [human operator]. It can give hints to direct the attention of the human operator or may take over task commitments if required. Thereby, the automation can be seen as an artificial crew member that ‘looks over the pilot’s shoulder’ but is not a replacement.”

Our Remarks

The mission planning technology for SRT operation shall include an efficient architecture to support cooperative planning and re-planning; a semi-automatic planner; an assistant system to coordinate between human operators and robots; a proper communication protocol. Further, the mission planner is expected to optimize performance, to enable re-planning, and to consider the balance of workload.

B.1.2 Capacity and Workload Model

As mentioned, an autonomous or semi-autonomous SRT operation shall ensure that the human machine interactions are well within the human capacity and workload, such that the unmanned robotic systems are playing the role of assisting human operators, not adding extra burden to

Page 109 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

the soldiers. It is therefore important to have the human capacity and workload assessment implemented in a successful human machine system.

The mental models are developed to assess the human capacity as well as the workload. The capacity model of attention states that “there is a general limit to the capacity that an operator can perform. Depending on the difficulty of the task, more or less demand is required. In case the attention level is smaller than the demands, the performance will collapse. The attention capacity depends on the arousal of the person and can be increased by effort or stress. The state of exertion and thus the level of capacity can be determined from measurement data such as the pupil reaction or the pulse rate. A capacity allocation distributes the available capacity to individual processing modules. The ability to perform tasks in parallel depends on whether the sum of the task-related capacities do not exceed the available capacity.” The model is simplified “by abstracting the arousal and defining a constant capacity for each subject at any time. Additionally, instead of a capacity allocation to the individual processing modules, the sum of the allocated task capacities is calculated and checked whether it exceeds the total capacity” (Hou et al, 2015b).

In the multiple resource model, four different categories with dichotomous dimensions are used to describe the mental resources for a task (Hou et al, 2015b). That means, each dimension has two levels without intersecting each other. The four dimensions are: 1) processing stages, 2) perceptual modalities, 3) processing codes, and 4) response generation. Perception and cognitive tasks claim the same resource, but this resource is functionally separate from those responsible for the selection and execution of responses. The third dimension, processing codes, states that spatial and verbal processes (codes) in perception, information processing and reaction are based on different resources. One procedure to estimate the mental workload for the parallel task execution is called the workload index (W/INDEX).

In order to evaluate the impact of workload optimization in the plans, a subjective rating method may be selected to asses mental workload during mission execution. NASA Task Load Index (NASA-TLX) consists of two parts, weights and ratings. In a first step the contribution (weights) for each of the six factors, mental demand, physical demand, temporal demand, performance, effort and frustration, is rated. For each of the 15 possible pair-wise comparisons the subject must specify which contributed more to the workload of the task. The weight for the respective dimension results from the number of times the factor was chosen. The second part, ratings, determines the magnitude of a factor for the specific task. The subject marks the desired location on a 21-tick scale (e.g., low/high for the mental demand).

Our Remarks

The concept of capacity and workload modelling is a subject of different human factors and psychology discipline. The study of the model development is not the technical focus from the SRT perspective. However, the results, the assessment and deployable models shall be investigated to integrate into the human machine systems for SRT operations.

Page 110 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

B.1.3 Cognitive Automation

To increase the level of autonomy to support SRT operations, the technical area of cognitive automation brings the automation technology under the consideration of human cognition.

The approach of cognitive automation is an “automation approach [...], which [...] uses suitable architecture and processing approaches and algorithms, to realize higher cognitive functions, like planning, problem solving or decision making, on machines” (Brand & Schulte, 2015). This “enables the automation to take over higher cognitive tasks from human.” The aim is neither replace conventional automation nor the human crew. Instead, the aim is to extend conventional automation with higher cognitive abilities to support the human crew whenever it is necessary. Cognitive automation can be treated as a more general frame to enable cooperative mission planning and beyond.

The following technical requirements or features in cognitive automation are identified (Clauss et al, 2013):

1. As one key feature for cooperation between the pilot and the unmanned team, an efficient communication between unmanned and manned members shall be identified, probably in a similar way like manned teams do. In order to come one step closer to this vision we propose the concept of a self-explanation capability for machine agents, which enable the unmanned aircraft to provide information-feedback beyond simple status reporting. 2. A cooperative automation that a. strives towards complementary or identical goals, if this is not the case team members are not in a cooperative but in a competitive situation; b. is capable to coordinate, which means to be able to manage interdependencies; c. supports the generation of mutual understanding and acceptance of behavioural pattern of the other; and d. supports the development of trust by team members toward own behaviour. 3. A cognitive system architecture. 4. A priori knowledge. 5. In the domain of conventional automation is in operation, human operators supervising such complex automated systems are exposed to several problems like “opacity”, “literalism” or “brittleness” as stated by the well-known critics. These problems need to be addressed in cognitive automation.

Page 111 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

A recent development of Artificial Cognitive Units (ACUs) symbolized the further course of study in this area. These are basically artificial, knowledge-based agents, which are able to develop rational goal- directed behaviour by processing knowledge.

Our Remarks

Cognitive automation presents a key component of cooperative automation to enable coordinated interactions between humans and robotic systems. It also reflects how the previously studied mission planning and capacity modelling shall be implemented.

B.1.4 Human-Autonomy Interaction

Another aspect of autonomy in SRT operations concerns of the human-autonomy interaction. While the cognitive automation or cooperative automation in the previous section addresses the automation framework, the human-autonomy interaction here concentrates on the interactions with the emphasis on the interface.

First, current metrics and methods need to be identified for measuring team communication, including qualitative coding, computational approaches, network analysis, and voice analysis. These methodologies provide insights into what information could be shared and how that information should be exchanged during human autonomy teaming (HAT). These insights will be critical to developing effective teaming because communication is directly related to performance, coordination, team cohesion, and appropriately calibrated team trust. As future HATs are developed, we can use communication analysis tools to understand how the capabilities of humans and autonomy can be better integrated as well as how the autonomy can be designed to improve human understanding, trust, and situation awareness.

A human agent interface was developed for the successful cooperation of cognitive agents and human crews, based on the approach of cognitive and cooperative automation, the so-called Dual-mode Cognitive Automation (DMCA) (Gangl, 2013). A work process is defined as a dynamic process, with the purpose to achieve a certain work objective by a human-machine system under environmental conditions. A work system is the physical representation of the referred work process. One most important variation of DMCA envisages adding an Artificial Cognitive Agent (ACA) to the operating force of the work system. Because of its higher cognitive abilities, the assistant system is able to work in parallel with the human crew to generate a “reference plan” and therefore to detect discrepancies between the human’s activities and its own solution.

In the remotely piloted aircraft (RPA) case, the crew is supported by adaptive, intelligent user interfaces (IUIs) that dynamically switch display modes according to current crew tasks (Hou et al, 2015b; Strenzke et al, 2011). The concept is based on detecting the tasks actually worked upon and supporting the crew in these tasks. Within a so-called Cognitive Cockpit (CogPit) program, a Cognition Monitor (COG-MON) has been developed to monitor the physiology and behaviour of pilot (or a unmanned ground or aerial vehicle (UxV) operator) in order to make

Page 112 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01 assumptions about his current objectives, attention, and workload. In order to enable the management of multiple UxVs from a manned vehicle or ground control station, the UxV has to be equipped with an additional onboard human machine interface (HMI). This interface facilitates the information exchange between the human operator and the unmanned sub-team. The human operator receives continuous feedback about the ongoing situation and thus can adapt his mental model. The HMI also provides an interface to the pilot to transmit information about events to the unmanned sub-team (Russ et al, 2013).

Again, the following technical requirements or features of human-autonomy interaction are identified:

1. Basic requirement for operation-supporting means (Brand & Schulte, 2015): a. If the assistant system detects a discrepancy between the humans and its own plan (i.e., the operator does not work on the most urgent task) it is necessary to intervene. The first stage recommends guiding the attention of this discrepancy. b. If human operator is overtaxed (i.e., the crew is not capable to handle the task situation), the assistant system shall transfer the task situation into one, which can be handled by the crew. c. If the human operator in principle cannot perform the task or the neglecting costs would be too high, only then the assistant system shall perform the task automatically. 2. The concept of the human-agent interface modalities of communication (Schmitt & Schulte, 2015): a. Implicit communication is the observation of the human system interactions, which enables the assistant system to reason about the current executed tasks. This knowledge can be used to estimate the current mental state of the human operator. b. Explicit communication is the dialog, which is initiated by the assistant system in case of an assistive intervention. There is no explicit operating interface to configure the assistant system, but the HMI is used to exchange information with the human operator. c. Human Agent Interface for implicit communication (Software-Framework): i. Stored data. ii. Highlighting. iii. Interaction Exporting Interface. iv. Human Agent Interface for explicit communication (Assistant System Dialog Interface). v. Intelligent user interfaces (Strenzke et al, 2011).

Page 113 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Our Remarks

Human-autonomy interaction focuses on interface development and its associated communication protocol. Based on the technical development, the direction points to the intelligent adaptive interface concept (Hou et al, 2015b).

B.1.5 Sensor and Perception Management System

From SRT operation perspective, the unmanned systems are equipped with sensors and payload to support a specific MUM-T mission. Therefore, a sensor and perception management system (SPMS) specifies some unique features and illustrates once again how human and machines interact.

The system receives and computes high level tasks (e.g. observing a determined area) and dynamically reconfigures itself using appropriate algorithmic solutions considering available onboard sensory and computational resources as well as domain knowledge and actual environmental information during flight. In the work of Gangl et al (2013), a civil Search and Rescue (SAR) scenario was discussed in detail, describing implementation aspects and investigating performance and capabilities.

A successful technical development of sensor and perception management system to support SRT operations shall have the following requirements or features:

1. The SPMS is responsible for performing perception tasks and managing sensors during mission execution. It utilizes heterogeneous sensor sources, image and signal processing methods as well as related domain knowledge. To better execute a specific task, the SPMS may recommend platform maneuvers (platform requests) e.g. changing the altitude or attitude of a UAV. Eventually, after task completion, perception results are passed back to the SPMS. 2. The SPMS comprises into a Perception Management (PM) and a Sensor Management (SM) layer. The PM layer selects and executes appropriate algorithms for individual perception tasks. The results are validated subsequently taking into account plausibility and consistency. The SM layer controls available hardware resources like sensors and gimbals, based on capability models. 3. Market-based Coordination of Perception Chains. 4. Trustworthiness Modeling of Perception Chains.

Our Remarks

Page 114 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

The SPMS provides a representative subsystem to illustrate SRT operations. In addition to the suggested features, it depends on the development of sensor and sensing technologies. Technical challenges in this area shall pay close attention to the sensor technology status and development.

B.1.6 Guidance, Navigation and Control

In general, guidance, navigation and control (GNC) provides the enabling technology to execute missions. Customized and specialized GNC development related to SRT operations are reviewed here.

A representative loyal wingman example illustrates the need for advanced UxV Guidance, Navigation and Control (GNC) development (Semnani et al, 2020). It established a definition of the uninhabited loyal wingman and demonstrated through simulation the use of direct orthogonal collocation as a technique to achieve rapid outer-loop global optimal path planning. Again this example, the optimal control problem and multiple scenarios are established for a static, deterministic threat environment (Semnani et al, 2020). Additionally, a dynamic and measurement update model are established for tracking and successfully avoiding dynamic, non-deterministic threats. A first set of results demonstrates a loyal wingman dynamic route re- planning algorithm in the midst of pop-up stationary threats and a changing mission rendezvous requirement. A second set of results demonstrate the loyal wingman dynamic re- planning algorithm in a dynamic, non-deterministic threat environment and recommends a loyal wingman dynamic route re-planning frequency and time based on the estimated speed of the moving, non-deterministic threat (Humphreys et al, 2016).

Intelligent mission and guidance systems providing the flexibility and cooperative behaviour needed to complete an entire mission autonomously are therefore a topic of active research (Semnani et al, 2020). It discusses the development, implementation, and evaluation of such a system containing a manned vehicle and a UAV operating in a partially known environment. It enables the operators of the manned aircraft to issue tasks and mission-level commands to the unmanned aircraft in real time using a natural language parsing and interface. The latter translates English sentence commands from the crew to a set of codes understood by the UAV and vice versa. The overall mission system thus transforms the natural language commands of the manned aircraft operators into a mathematical programming problem producing real-time trajectories that implement the dynamic mission plan of the UAV.

A number of challenges had to be overcome during the development of this system (Schouwenaars et al, 2006):

1. First, the mechanism allowing both vehicles to communicate with one another needed to be designed. 2. Second, the UAV guidance technology had to be made robust to changes and threats in the vehicle’s environment, including changing wind conditions, no-fly zones, and other

Page 115 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

obstacle- sand produce safe trajectories through the partially known environment. 3. Third, because the system was intended for real-time missions, all developed algorithms needed to reach a solution and resolve any unexpected issues reliably in a predefined period of time.

Our Remarks

GNC provides the enabling technology to support SRT operations. The key concept is its integration with other subsystems, and the success of GNC delivery depends on the clarity of command and control.

B.1.7 Multi-UAV Autonomy

We will now move to the domain of technical areas for long-term future SRT operation scenarios. It is identified that the multi-UAV autonomy is one technical area that will scale and improve SRT operations and autonomy in the long run.

A multi-UAV team perception system POCA (Perception-Oriented Cooperation Agent (Schmitt & Stuetz, 2019)) is discussed. It has integrated signal and data processing algorithms for the multi- UAV based reconnaissance and assessment of landing zones along with experimental results of human factor engineering evaluations and flight tests. Three key components were identified which had to be addressed individually in order to conceptualize and integrate POCA in the MUM-T context:

1. The planning and scheduling of the perceptive and navigational subtasks; 2. The probabilistic and reliability-aware fusion of the individual reconnaissance results; and 3. The self-explanatory presentation and visualization.

Fusing heterogeneous data reflecting heterogeneous physical phenomena, are required for multi-UAV reconnaissance. Therefore, preprocessing and preevaluation of the associated sensor data is mandatory to enable the subsequent fusion step. Proposed approaches include the Bayesian Network (BN) to explicitly model the pilots’ information needs and their interdependencies in a probabilistic fusion graph.

A more advanced topic beyond mission planning is the so-called motion planning to cover dynamic scenarios (Semnani, 2020). MULTI-AGENT motion planning has recently attracted much interest in the research community. This problem concerns finding trajectories connecting each agent’s initial location to its goal location. Each agent has a fixed final position that cannot be exchanged with another agent. In addition, the motion planning algorithm should be able to satisfy constraints such as the maximum/preferred velocity of the agents and the minimum

Page 116 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

required separation distance between them. These constraints are dependent on the agent’s dynamic limitations and may vary from one application to another. In this problem agents make decisions in a dynamic environment with a (possibly varying) number of other agents, whose policies and intents are unknown. The latest development involves using intelligent algorithms developed from machine learning or artificial intelligence, such as the reinforcement learning.

Our Remarks

We gave a couple of examples to demonstrate the capabilities of multi-UAV autonomy as well as its technical challenges. It is anticipated that the multi-UAV autonomy will greatly scale the SRT operations to improve the performance. There exists much uncertainty in terms of technical advancement and maturity.

B.1.8 Human-Swarm Intelligence

The multi-UAV automation establishes the foundation that brings intelligent and adaptive cooperation among the unmanned systems themselves. This foundation will help to develop a framework between a single human operator and a group of robotic systems, termed by human-swarm intelligence.

With the increased presence of unmanned systems in the current battle space, there is a fundamental need for increased levels of automation, coordination and intelligent collaboration among various physical agents in the current and future battle space. One of the major concerns of UxV development, according to the documentation of the Joint Robotics Program, is the evolution from human intervention to Robot autonomy/intelligence (Coleman et al, 2008). The system autonomy has been evaluated by a spectrum of measures from obstacle detection/avoidance, route planning to mission planning, pattern recognition and situation awareness. According to the program timeline and current deployment progress, many technologies regarding to standalone unit operation are well developed and tested. However, many system level issues pertaining to collaborative behaviour and swarm intelligence still need to be investigated.

Our Remarks

Human-swarm intelligence is at its very early conceptual stage. The potential of human operating swarming robotic systems shows promising features, however, many system level issues pertaining to collaborative behaviour and swarm intelligence still need to be investigated.

B.1.9 Network-Centric Navigation and Control

Similar to the GNC enabling technology to support SRT operations, specialized GNC technologies are also required to enable multi-UAV automation, and human-swarm intelligence. A promising feature of the GNC specialty is the network based GNC.

Page 117 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Jack et al (2020) presents personnel/platform tracking, navigation and communication system as an essential part for Future Combat System, Future Force Warrior, Objective Force Warrior, Land Warrior and Homeland Defense applications where one urgently needs to have novel position/location tracking, communications system and decision making devices that would permit multi-tracking, reporting and recording operations. This Intelligent Precision Geolocation and Control Networked Multiple UxV Demonstration system provides precision interruption-free position for multiple tracking of UxV and other combat platforms, in complicated environments and terrains. The system is realized by using a core micro Palm Navigators with wireless ad hoc networks to perform communications and wireless location at the same time, with improved geolocation accuracy, and increased tracking area coverage. A suite of information multi-sensor fusion and wireless communications was demonstrated. The system demonstration of the fully functional toolsets for robotics, robotic autonomy, robotic human interface, remote targeting and surveillance is presented in fusion algorithms:

• Autonomous Multi-Agent Manned/Unmanned System Collaborative Engagement; • Enhanced Wireless Wide Area Networked Precision Geolocation and Control system; • The Augmented UMS Robotic System within the ARDEC MPC Architecture; and • Wide Area Wireless Networked Navigators System Architecture.

Our Remarks

One reviewed article served as a representative example of network-centric GNC, where the architecture and fusion algorithms illustrate the features of future GNC technical challenges to support SRT operations.

B.1.10 Artificial Intelligence

AI has become an active area of exploring its capabilities to support SRT operations. The focus is placed on the intelligent agent algorithms.

Encouraging successful teamwork between these heterogeneous teammates leads to improved overall performance and mission success. Previous work has focused on improving teamwork by monitoring human physiological and emotional states, including situation awareness, fatigue, trust, etc. Researchers on the autonomous side of the equation have also worked to improve team functionality by explaining the decisions made by autonomous systems. The resulting explainable artificial intelligence (XAI) allows human operators to better understand AI decisions, increasing trust and allowing for verifiability and certification. Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR), part of NASAs Convergent Aeronautics Solutions (CAS) project, seeks to establish bases for certifying trust and trustworthiness in order to facilitate such heterogeneous teams (Meszaros et al, 2019). This work builds upon prior evaluation of textual and verbal input modalities for a SAR mission utilizing a

Page 118 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

heterogeneous human and autonomous system team. In addition to expanding the initial investigation to include multiple images, this work also explores new evaluation techniques to better observe how different input modalities contribute to overall mission success. Continued investigation will also incorporate a secondary user study, providing significantly more data. The goal of these evaluations is the recommendation of specific input modalities that increases usability and allows for intuitive, explainable interactions between human and autonomous agents within a team.

Another field of AI also shows great potential in autonomous SRT operations and human machine interactions, that is the reinforcement learning. For example, a hybrid algorithm of deep reinforcement learning and force-based motion planning is developed to solve distributed motion planning problem in dense and dynamic environments (Semnani et al, 2020). The intelligent algorithms are also developed for specific scenarios, such as border patrolling (Lau et al, 2014). It presents an online path planning algorithm for unmanned vehicles in charge of autonomous border patrol. In a pursuit-evasion game, the unmanned vehicle, acting as the pursuer, is required to capture multiple trespassers on its own before any of them reach a target safe house where they are safe from capture. In persistent tracking, where a cooperative nonlinear filter was developed to estimate the states of a maneuvering target with unknown dynamics and bounded acceleration using multiple mobile sensors (Zhang et al, 2015).

Our Remarks

The review of artificial intelligence (AI) for the potential applications in SRT operations only touches several preliminary work. The majority of relevant AI research lies in the robotics community, where the reinforcement learning is actively investigated in its application and integration with existing autonomy research. The AI research is also active in human factors, it is expected that the AI will play a significant role for future human machine interactions in SRT operations.

B.2 SRT SUPPORT USE CASES

The categorized technical development lists are analyzed in the identified five SRT use cases. The overall relevance of the technical development to the use cases are shown in Table 22. Detailed technical challenges will be specified in each use case.

Table 22: Technical Development Supporting SRT Use Cases (X: Relevant, !: Critical)

Technical Development Mule Wingman Artemis RoboMedic Amazon

Mission Planning X X ! X X

Capacity and Workload Models X X

Cognitive Automation ! X

Page 119 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Technical Development Mule Wingman Artemis RoboMedic Amazon

Human Autonomy Interaction X X X ! X

Sensor and Perception Management X X X X X

Guidance, Navigation and Control X X X X X

Multi-UAV Autonomy X !

Human-Swarm Interaction X ! X

Network Centric Navigation and X Control

Artificial Intelligence X X X X X

B.2.1 Technical Challenges: Mule

The Mule UGV provides load-carrying for dismounted infantry section during patrol mission. Mule is loaded with combat supplies, equipment and C16 automatic grenade launcher; reducing the load-carrying burden of the section.

In this use case, the technical development of Human and Autonomy Interaction plays a critical role to the operation. Technical challenges include:

1. Auto follow-me mode. 2. UGV operator assistance. If a UGV has no more queued mission tasks, it needs to continue with a new plan as an assistant system to support the operator in appending a follow-up task. In addition, the assistant system supports the operator in activating or stopping the execution of certain UGV tasks (e.g., activating initial departure). 3. Collision or obstacle avoidance On the other hand, the technical development of Sensor and Perception Management, and Guidance, Navigation and Control are all relevant to this use case. Some general and specific technical challenges include:

1. Obstacle avoidance; 2. Data link management; 3. Perception management, remote sensing 4. Detection sensor suits; and 5. Robust GNC.

Our Analysis

Page 120 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

The role of robot (UGV) is to carry load for soldiers. The key technical challenge to be investigated is the capability of UGV following soldiers on command and return to safety zone if necessary. The most critical technical challenge in this case of MULE is the cognitive automation. We suggest to design and develop an “auto follow-me” scenario in the immediate experimentation, to investigate how a soldier operator gives command of “follow me” and the UGV executes the command, including the automatic path planning (to follow closely) and path following (to avoid obstacles); the UGV can stop or return to initial point on command.

B.2.2 Technical Challenges: Wingman

The Wingman use case describes a scenario: Hand-launched, hybrid micro-UGV/UAV to provide tactical sensing capability to dismounted infantry soldier. Dismounted infantry soldier is responsible for the control of the Wingman using personal ISS. Wingman is equipped with an electro-optical / infra-red sensor, an EW decoy capability, and a CBRN detection capability.

In this use case, the technical development of Capacity & Workload Models, Cognitive Automation, and Human Swarm Interaction are critical to the success of the mission.

1. Cognitive system: a cooperative automation is established between the soldier and the dispatched UAV (UAVs), where the soldier gives a clear surveillance and reconnaissance (SAR) command and the UAV(s) can automatically take off, cruise and collect information, and communicate with the soldier; 2. Multi-UAV automation: in the case of dispatching multiple UAVs, it is expected that these UAVs have a certain level of autonomy to manage their coordinated flight and information among themselves; 3. Capacity modelling: based on the human capacity assessment, a soldier capacity model is implemented that acceptable amount of information is communicated between the soldier and the UAV(s) that fulfil the requirements: soldier needs to have a full control of situation awareness and the information will not overload to the soldier; 4. Perch mode – landing: the UAV(s) shall have the built-in capability in any case of losing communication with the soldier for safe landing, not to jeopardise the mission.

Schmitt & Stuetz (2019) present implementation aspects and experimental evaluation results for the cooperative multi-UAV team perception system POCA (Perception-Oriented Cooperation Agent) for helicopter landing zone reconnaissance and landing point evaluation. The presented multi-UAV on-board system is integrated in a full-mission R&D helicopter simulator used for MUM-T research in complex military search & rescue scenarios requiring field landings in uncontrolled and unsafe areas. In addition, the system is deployed onto multiple small rotorcraft UAVs to prove the feasibility and maturity of the system concept. The presented SPMS thereby incorporates knowledge on pilot’s information demands on safe landing points and probabilistic reliability estimations of applied perceptive capabilities to derive its own course of actions for landing zone reconnaissance. Expert knowledge is used to weight the single criteria in a multi-

Page 121 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

dimensional landing point assessment while a perceptive trustworthiness estimation is used to handle the uncertainty of the measuring and perception processes when fusing the individual perceptive results to rank individual landing points and to create a recommendation for the helicopter crew. The system was evaluated in an extensive human-in-the-loop experimental campaign with military trained helicopter pilots in terms of human factors issues, system interaction, and user acceptance. In addition, ongoing multi-UAV flight test validated the technological readiness of the cooperation and automation concept to perform perception- driven mission tasks semi-autonomously.

On the other hand, the technical development of Sensor and Perception Management, and Guidance, Navigation and Control are all relevant to this use case. Some general and specific technical challenges include:

1. Obstacle avoidance; 2. Data link management; 3. Perception management, remote sensing; 4. Detection sensor suits; and 5. Robust GNC.

Our Analysis

To investigate the SRT technical challenges in this use case of Wingman, the focus is recommended to be placed on the human-swarm interaction, where a finite number of UAVs are deployed to perform SAR missions to support the soldier on the ground. The soldier communicates either in broadcast mode to all UAVs or direct his/her command to the leader of the UAV team. On the other hand, the collected information by the UAVs will be processed and coordinated such that a comprehensive, executive message is sent back to the soldier for situation awareness analysis and decision making.

B.2.3 Technical Challenges: Artemis

The Artemis use case indicates one UGV to operate as a semi-autonomous weapon platform (with human-in-the-loop authority to engage) for dismounted infantry platoon during patrol mission. Platoon Weapons Detachment Gunner is responsible for the control of Artemis using personal ISS. Artemis is loaded with 81mm self-loading mortar and ammunition; reducing the load- carrying burden of the platoon.

In this use case, the technical development of Mission Planning, and Human and Autonomy Interaction are critical to the success of the mission:

1. Optimal route planning;

Page 122 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

2. Decision making; and, 3. Assessment.

1. Obstacle avoidance; 2. Data link management; 3. Perception management; 4. Detection sensor suits; and, 5. Robust GNC.

Our Analysis

To investigate the SRT technical challenges in this use case of Artemis, the focus is recommended to be placed on the cooperative mission planning. Unambiguous command from the soldier needs to communicate with the robot for execution. Situation assessment and re- planning needs to be conducted in real-time.

B.2.4 Technical Challenges: RoboMedic

The RoboMedic use case provides a scenario for a single UGV to provide support to Dismounted Combat Medic performing battlefield casualty treatment and evacuation. Platoon Combat Medic is responsible for the control of the RoboMedic using personal ISS. RoboMedic is loaded with medical supplies and health monitoring equipment; reducing the load-carrying burden of the Platoon Combat Medic and the number of Platoon members required to evacuate the casualty.

In this use case, the technical development of Mission Planning, and Human and Autonomy Interaction are critical to the success of the mission.

• The mission planner shall optimize mission performance, i.e. the overall mission plan shall comply with given standards and optimality criteria; • The mission planner shall ensure adequate pilot workload, i.e. the cooperation shall ensure that the pilot is supported adequately to the current tactical situation as well as to the situation in the cockpit; • The mission planner shall ensure pilot plan and situation awareness, (i.e., the pilot shall be aware of the current situation and upcoming decisions at all times);

Page 123 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

• The soldier shall ensure full world knowledge, i.e. the cooperation shall ensure that the limited world model of the automation can be enlarged by the pilot; • One key element is the problem solver library. This library contains domain knowledge, task specific and domain independent problem solvers used to compare and rate plans. It can be used to compute solutions fully automatically and to determine next planning steps, given the mission objective and other constraints, the tactical situation, and available resources (UAVs) including their capabilities; • The second key element is a model of human-system interaction/cooperation, which contains rules about how to interact with the pilot. Especially the timing of interaction is challenging. If a flaw is detected using the planner library, the interaction model is used to decide when and how to convey the information to the pilot; • The third key element is a pilot behaviour model, which describes human capabilities and limits, goals and intentions, as well as the expected performance regarding the mission planning process. A general model of the planning process is used to determine how the pilot approaches a planning problem. The model is, for example, used to make judgements, which tasks can be solved by the pilot on his own. Basis for the planning support is that the mission objective is known.

On the other hand, the technical development of Sensor and Perception Management, and Guidance, Navigation and Control (GNC) are all relevant to this use case. Some general and specific technical challenges include:

1. Obstacle avoidance; 2. Safe, stable and smooth transportation; 3. Route planning; 4. Data link management; 5. Perception management; 6. Detection sensor suits; and 7. Robust GNC.

Our Analysis

To investigate the SRT technical challenges in this use case of RoboMedic, the most critical technical area is human-automation intelligence. The focus is recommended to be placed on the cooperative human-machine interface and adaptive communication to carry out the use case scenarios.

B.2.5 Technical Challenges: Amazon

Page 124 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

The Amazon use case presents a combination of autonomous assets to provide ‘just-in-time’ delivery of supplies and equipment repair parts to deployed infantry platoon (i.e., ‘sense and respond logistics’) and enhance perimeter security around the patrol base. Specifically:

• UGV to deliver resupply container to platoon in the field; and • Four UAVs (onboard the resupply container) provide local resupply of combat supplies to dispersed members of the platoon (e.g., sniper) and enhance perimeter security around the patrol base.

Logistics personnel at BG FOB are responsible for determining resupply strategy and dispatching autonomous resupply UGV convoys. The Platoon 2IC is responsible for coordinating the request for resupply and the control of the Amazon UAV fleet. Resupply container is loaded with: • Ammunition and combat supplies (according to expenditure tracked by ISS). • Water and food (according to energy expenditure and weather conditions tracked by ISS). • Batteries and fuel (according to expenditure tracked by ISS). • Medical supplies (according to expenditure tracked by #RoboMedic). • Repair parts (equipment corrective and preventative maintenance requirements tracked automatically through onboard Vetronics system). • C2 equipment and communications to establish platoon HQ. • Four UAVs to support local re-supply and patrol base security (docking on roof of container includes power supply for recharging).

In this use case, the technical development of multi-UAV autonomy and human swarm interaction will play critical roles to support the mission. The technical challenges include:

1. Coordinated supply and delivery by four UAVs; 2. Persistent patrolling; 3. Autonomous vehicle operations; and 4. Human monitoring and intervention

On the other hand, the technical development of Mission Planning, Capacity and Workload Models, Sensor & Perception Management, and Guidance, Navigation and Control are all relevant to this use case. Some general and specific technical challenges include:

1. Path planning, motion planning of UAVs; 2. Collision avoidance;

Page 125 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3. Flight management; 4. Route planning; 5. Data link management; 6. Perception management; 7. Detection sensor suits; and, 8. Robust GNC.

Our Analysis

To investigate the SRT technical challenges in this use case of Amazon, the most critical technical area is multi-UAV automation. The focus is recommended to be placed on the coordinated automation among the UAVs in task assignment, allocation, and performing delivery.

B.3 SUMMARY

A review of the state-of-the-art technical development is presented regarding integrating manned and autonomous systems in the context of soldier robot teaming concepts and use cases.

Ten (10) technical area challenges are identified. From a short-term viewpoint, 1) cooperative mission planning between human and robotic team supported by a semi-automatic mission planner and assistant system; 2) capacity and workload model to assess the human operator capacity and to be integrated in MUM-T to balance workload; 3) cognitive automation to address cooperative automation strategy with cognitive architecture; 4) human-autonomy interaction to focus on human machine interface; 5) sensor and perception management system to illustrate information collection and processing to support decision making; and 6) guidance, navigation and control to enable automation and autonomy in execution. From a long-term perspective, 7) multi-UAV autonomy to encourage self-regulation among autonomous systems as a foundation, to support 8) human-swarm intelligence to scale solider robot teaming operations, further enabled by 9) network-centric navigation and control; and in the foreseeable future, the 10) artificial intelligence will play a significant role to realize full autonomy in soldier and robot teaming operations.

In addition, the selected technical area challenges are allocated to five (5) use cases for soldier robot teaming projects. In each case, relevant technical challenges are specified, one critical technical area is identified to be placed as a focal point for operational requirements analysis and experimentation planning. 1) Mule is a payload-carrying assistant, where the cognitive automation is the critical technical focus; 2) Wingman are a group of UxVs dispatched by the soldier to provide surveillance and reconnaissance capability to the soldier, where human-swarm interaction is the critical technical component; 3) Artemis operates as a semi-autonomous

Page 126 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

weapon platform, where cognitive automation serves as the critical technical challenge to deploy and safeguard the autonomous weapon robot(s); 4) RoboMedic plays an active role of medical assistance during the missions, where cooperative mission planning assures various tasks are carried out successfully; and 5) Amzon is the case when multiple UAVs help to deliver medical supplies, where multi-AUV automation is the identified critical technical challenge to be focused on.

Page 127 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

B.4 REFERENCES

Brand, Y., & Schulte, A. (2015). Human agent interfaces as a key element for the dialog between human crews and cognitive automation (pp. 1-13). In AIAA Infotech@ aerospace.

Clauss, S., Kriegel, M., & Schulte, A. (2013). UAV capability management using agent supervisory control (pp. 49-100). In AIAA Infotech@ Aerospace Conference.

Coleman, N., Lam, K., Patel, K., Roehrich, G., & Lin, C. F. (2008). Network Centric Multiple Manned/Unmanned Systems (UMS) Navigation and Control Coordination (pp.1-15). In AIAA Guidance, Navigation and Control Conference and Exhibit.

Fang, S. X., O’Young, S., & Rolland, L. (2018). Online risk-based supervisory maneuvering guidance for small unmanned aircraft systems (pp. 2588–2603). Journal of Guidance, Control, and Dynamics, vol. 41, no. 12.

Gangl, S., Lettl, B., & Schulte, A. (2013). Management of multiple unmanned combat aerial vehicles from a single-seat fighter cockpit in manned-unmanned fighter missions (pp. 1- 18). In AIAA Infotech@ Aerospace Conference.

Heilemann, F. Hollatz, F., & Schulte, A. (2020). Integration of Mental Resources in the Planning of Manned-Unmanned Teaming Missions: Concept, Implementation and Evaluation (pp. 1– 10). In AIAA SciTech Forum, January.

Hou, M., Banbury, S., & Burns, C. (2015b). Intelligent Adaptive Systems: an interaction-centered design perspective. CRC Press.

Humphreys, C. J., Cobb, R., Jacques, D. R., & Reeger, J. A. (2016). Dynamic Re-plan of the Loyal Wingman Optimal Control Problem in a Changing Mission Environment (pp. 1-15). In AIAA Infotech@ Aerospace.

Jack, D. P., Hoffler, K. D., Roper, R. D., Trujillo, A., Lewis, T., & Tsakpinis, D. (2020). Human-in-the- Loop Flight Simulation Experiment on Unmanned Aircraft Terminal Operations (pp. 1–23). AIAA Scitech Forum, January.

Lau, G., & Liu, H. H. (2014). Real-time path planning algorithm for autonomous border patrol: design, simulation, and experimentation. Journal of intelligent & robotic systems (pp. 517- 539). 75(3-4).

Lettl, B., & Schulte, A. (2013). Self-explanation capability for cognitive agents on-board of UCAVs to improve cooperation in a manned-unmanned fighter team (pp. 1-11). In AIAA Infotech@ Aerospace Conference.

Meszaros, E. L., Le Vie, L. R. , Barrows, B. A. , Last, M. C. , Smith, M., & Allen, B. D. (2019). Evaluating communication modality for improved human/autonomous system teaming

Page 128 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

(pp. 1–12, 2019). AIAA Scitech Forum, January.

Russ, M., Schmitt, M., Hellert, C., & Stütz, P. (2013). Airborne sensor and perception management: Experiments and Results for surveillance UAS (pp. 1-16). In AIAA Infotech@ Aerospac Conference.

Schmitt, F., & Schulte, A. (2015). Mixed-initiative interaction in manned-unmanned-teaming mission planning: Design and evaluation of a prototype (pp. 01-14). In AIAA Infotech@ Aerospace.

Schmitt, M., & Stuetz, P. (2019). Cooperative Multi-UAV Sensor and Perception Management for Helicopter Landing Zone Reconnaissance: Implementation and Experimental Evaluation (pp. 1-8). In AIAA Scitech Forum.

Schouwenaars, T., Valenti, M., Feron, E., How, J., & Roche, E. (2006). Linear programming and language processing for human-unmanned aerial-vehicle team missions. Journal of Guidance, Control, and Dynamics. 29(2), 303-313.

Semnani, S. H., Liu, H., Everett, M., de Ruiter, A., & How, J. P. (2020). Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning. IEEE Robotics and Automation Letters.

Strenzke, R., Uhrmann, J., Benzler, A., Maiwald, F., Rauschert, A., & Schulte, A. (2011). Managing cockpit crew excess task load in military manned-unmanned teaming missions by dual- mode cognitive automation approaches (pp. 1–24). AIAA Guidance, Navigation, and Control Conference, August.

Zhang, M., & Liu, H. H. (2015). Nonlinear estimation of a maneuvering target with bounded acceleration using multiple mobile sensors (pp. 1375-1385). IEEE Transactions on Aerospace and Electronic Systems, 51(2).

Page 129 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

ANNEX C. SRT HUMAN FACTORS ENGINEERING EXPERIMENTATION PLAN

C.1 INTRODUCTION

Based on the SRT operational requirements presented in the main body of the report and the review of technical challenges of SRT operations described in the previous annex, a HFE Experimentation Plan was developed to guide future HFE experimentation activities conducted under the auspices of the HAI TAC.

The SRT HFE experimentation plan described in this annex comprises the following sections:

1. SRT HFE Experimentation Themes. This section identifies and summarises several SRT HFE experimentation themes that will influence the HFE experimentation strategy of the HAI TAC. In addition, the themes were mapped against the five SRT use cases identified by the stakeholder analysis. 2. SRT HFE Experimentation Approach. This section presents an experimentation approach that will support all human-in-the-loop (HITL) research activities conducted during the HAI TAC. The process outlined will be used to design, develop and execute each SRT HFE evaluation identified in the SRT HFE experimentation themes, and involves DND stakeholder input throughout. 3. SRT HFE Evaluation Design. This section provides an overview of the critical components of the experimentation approach for designing a specific HFE evaluation. Specific guidance relates to experimental design, analytical methods to identify suitable data collection approaches and metrics, scenario development, and data analysis techniques. The scope of this section is restricted to laboratory-based approaches and does not include informal SME evaluation (e.g., heuristic analysis) or focus group approaches. The SRT HFE experimentation plan was developed, in part, from the following source materials:

1. Banbury, S., Zobarich, K., Dubé,G., and Filardo, E.-A. (2019). Canadian Surface Combatant User Style Guide. Lockheed Martin Contract Report. CAE. October 2019. 2. Banbury, S., Pelletier, S., Baker, K., and Tremblay, S. (2014). TA2: Ground Control Station Design Concepts and Human Factors Engineering Test Plan. DRDC Contract Report. Thales. May 2014. 3. Brown, M., Herdman, C., Banbury, S., and Baker, K. (2005). INCOMMANDS TDP: Demonstration and Experimentation Plan. Defence R&D Canada – Valcartier Contract Report. SP 2005-999. 4. Hou, M., Banbury, S., and Burns, C. (2014). Intelligent adaptive systems: An interaction- centered design perspective. CRC Press.

Page 130 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

C.2 SRT HFE EXPERIMENTATION THEMES

In order to determine the overall strategy for HFE experimentation relating to any SRT concepts developed during the HAI TAC, it is important that the most prevalent research themes are identified from the academic literature.

This section presents a brief review of the literature pertaining to Human Autonomy Teaming (HAT); the rubric which encompasses SRT and human interactions with autonomous systems.

C.2.1 Visual Perception of Remote Imagery

In UAV operations, the physical environment is decoupled from natural perceptual processing, which can compromise the operator’s accurate perception of the remote scene (Chadwick, 2008). For example, operators remote viewing imagery from UAVs are faced with limiting technological factors, such as image latency and degraded image quality. Latency in image processing refers to the temporal discrepancy between an actual event and when the event is viewed on a display (Prewett et al, 2010). Image degradation refers to the quality of video feeds that operators rely on for remote perception. Common forms of video degradation are caused by low frame rates, reduced display resolutions, and low image brightness and/or contrast. Poor remote perception is detrimental to the operator’s ability to acquire and maintain an adequate spatial awareness and build mental models of the environment (Chen et al, 2007). Degraded spatial orientation, distance estimation, and obstacle detection can reduce solider SA and decrease mission effectiveness as a result.

Many of the SRT use cases identified in this report include features relating to the viewing of remote imagery by soldiers. As such, the presentation of this information should consider the limitations of visual perception and identify suitable counter-measures depending on the type of remote imagery presented (e.g., HMI overlays to support distance estimation).

C.2.2 Human-Machine Interface Design Considerations

Feedback received during the stakeholder interviews consistently mentioned the following requisites of any Human-Machine Interface (HMI) that would be used by a soldier to interact with an autonomous system:

1. The HMI should be intuitive to the soldier and require minimal training; 2. The HMI should require minimal user input to task the autonomous system (i.e., no manual control of the movement of the autonomous system); and, 3. The HMI should not increase the workload of the soldier significantly and use voice or gestures as much as possible. Much of the research conducted in the area of HAT has focused on the optimisation of the HMIs used to task autonomous systems. For example, ‘Playbook’ HMIs have been developed to

Page 131 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

facilitate the effective and flexible tasking of multiple autonomous UVSs while keeping the operator in the decision-making loop (Miller et al, 2005). Playbook, and similar approaches, are a way of reducing operator mental workload and increasing SA by making the UVSs follow a predictable, pre-determined sequence of events, or ‘plays,’ that the operator has been trained to understand. Although the control of UVSs is simpler (a play initiates a sequence of behaviour by the UVSs), a significant limitation of this approach is that the operator cannot easily understand how well the play is performing against mission objectives, and based on that understanding, make ad hoc and real-time changes to the play in response to the evolving situation. Other HMI approaches have used video gaming-type icons, pictorial symbology, predictive (adaptive) displays, plan visualisation techniques, temporal displays, and HAT-oriented visualisations to present information in a concise, integrated manner (Behymer et al, 2015; Brandt et al, 2017; Calhoun et al, 2017, 2018; Cummings et al, 2018; Roldán et al, 2017). Other studies have evaluated the use of touch, voice, and multi-modal input devices to interact with multiple UAVs (Levulis et al, 2016).

All the SRT concepts identified in this report rely heavily on the HMI, specifically the ISS TUI, and it is anticipated that the evaluation of HMI concepts will factor significantly into the majority of SRT HFE evaluations conducted during the HAI TAC.

C.2.3 Human-Autonomy Teamwork

Human involvement with increasingly autonomous systems must accommodate for a more dynamic relationship involving cooperation and teamwork (Brandt et al, 2017). An evaluation of subject matter expert responses to how autonomous systems did or did not support human- machine teaming effectiveness identified the following critical characteristics of the system (McDermott et al, 2017):

• Observability. Ability of system to communicate to the human what it’s thinking, doing, and how far along it is in accomplishing joint work. • Predictability. Ability of system to communicate with human about its’ intentions, goals, and future actions in various contexts. • Directability. Ability of human to direct or redirect the tasks, resources, and priorities of an automated partner. • Directing Attention. Ability of system to notify human and present time-critical or task- critical information in a helpful way, so that human can stay aware of what’s important and act proactively. • Exploring Solution Space. Ability for system to help problem solve by suggesting things to consider and offering alternative suggestions. • Adaptability. Ability for system to recognize and handle unplanned situations. • Calibrated Trust. Ability of system to be clear about what it is good at and what its shortcomings are so human can determine how much and when to trust it.

Page 132 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

• Common Ground. Ability for human and the system to have a two-way street of communication about each other’s intentions, beliefs, and assumptions. • Information Presentation. Ability of system to present information so that it is easily understandable and does not require unnecessary work to calculate or interpret.

Other research has afforded us a better understanding of how team coordination dynamics differ between all-human teams and mixed human-autonomy teams and how these dynamics relate to team performance and team SA (Demir et al, 2018a). The results showed that all-human teams were more stable but less rigid than mixed human-autonomy teams leading to better team performance and team SA. However, with enough training and experience, teams should become increasingly stable, ultimately reaching an optimal blend of stability and flexibility. In addition, as autonomous agents become increasingly flexible and adaptive, HATs should trend toward this moderate level of stability (Demir et al, 2018b). Future SRT concepts developed by the HAI TAC should endeavor to support these coordination dynamics; specifically, by reducing rigidity in coordination.

Finally, recent research has also examined the role that team interaction (i.e., team verbal behaviors) play in team SA and team performance in order to better understand HAT (Demir et al, 2017; McNeese et al, 2018). The results show that pushing information was positively associated with team SA and team performance, and human-autonomy teams had lower levels of both pushing and pulling information than all-human teams. In order to make the SRT concepts identified in this report more effective in terms of teamwork, it is important to develop mechanisms that enhance the pushing of information from the autonomous system to the soldier, without distracting or overburdening them.

C.2.4 Operator Trust in Autonomous Systems

Although the performance of autonomous systems is constantly improving, these technologies, like humans, are not and will not be perfect. For humans and machines to work together successfully, machines should be clear about what it is good at and what its shortcomings are so that the human can determine how much and when to trust it (McDermott et al, 2017). Specifically:

• Calibrated Trust. When operator trust matches system capabilities, leading to appropriate use; • Overtrust. When operator trust exceeds system capabilities, leading to misuse; and • Distrust. When operator trust falls short of system capabilities, leading to disuse.

Trust can therefore be defined as the extent to which an operator is confident in and willing to act based on the recommendations, actions, and decisions of an automated system (e.g., contact identification recommendations).

Page 133 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Operator trust and acceptance of autonomous systems is also determined from the outcome of a comparison process between the perceived reliability of the automated aid (i.e., trust in aid) and the perceived reliability of manual control (i.e., trust in self) (Madsen & Gregor, 2000). Furthermore, overall trust in automation is determined by cognition-based trust (i.e., trust relating to the operator’s perception of the automation) and affect-based trust (i.e., trust relating to the operator’s emotive response to automation). Three factors underpin cognition-based trust (i.e., perceived understandability, technical competence, and reliability [of the system]), and two factors underpin affect-based trust (i.e., faith [in the system] and personal attachment [to the system]) (Kelly, 2003).

Recent research has identified six critical characteristics must be exhibited by autonomous systems in order to gain and maintain the trust from their human partners thus creating an effective partnership to achieve common goals (Hou, (in press)). Specifically:

1. Intention. The autonomous system should behave in a way that is aligned with the operator’s intentions. 2. Measurability. The behaviour of the autonomous system should be measurable so that it can be observed by the operator. 3. Predictability. The autonomous system should exhibit consistent and predictable behaviour and interactions with the operator. 4. Agility. The autonomous system should be able to cope with, and/or exploit changes in the situation or environment. 5. Communication. The autonomous system should, through a suitable HMI, communicate its actions, intentions, and goals to the operator. 6. Transparency. The behaviour of the autonomous system should observable and/or measurable so that the operator can develop a good mental model of their machine partner.

The Intention, Measurability, Predictability, Agility, Communication, and Transparency (IMPACT) model will be used to guide the development of SRT concepts developed during the HAI project. It will also be used to identify applicable metrics for each of the six characteristics; particularly observation-based subjective techniques to measure operator trust (Calhoun et al, 2017). In doing so, the IMPACT model can be validated and refined accordingly. Finally, a more extensive literature review should be conducted on the current state-of-the-art of operator trust measurement techniques at the earliest opportunity to support future SRT experimentation.

C.2.5 Single Operator Control of Multiple Autonomous Systems

Up until recently, multiple operators were usually required to control a single autonomous system (e.g., one operator piloting a UAV and one operator controlling the sensor payload). The

Page 134 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

vision of future UVS operations envisions the inverting of the operator-to-vehicle ratio so that one operator would control multiple, heterogeneous (air, sea and land) unmanned vehicles, or ‘swarms’ (Clare et al, 2012; Giles et al, 2017; Rosenfeld et al, 2017) . This entails shifting the role of humans from operators who directly control one UVS, to supervisors who will provide high level directions to up to 250 UVSs, in order to achieve mission goals (Donmez et al, 2010; Haring et al, 2018).

However, it is not clear which multi-agent architecture should be employed, and which tasks should be allocated to human or computer agents in order to allow the most proficient supervisory control systems (Hou et al, 2010, 2014). It will require finding the proper balance between decisions made by an autonomous system and the intermittent human interactions related to his or her central role of overseeing operation of all vehicles (Miller & Parasuraman, 2007).

Aside from the problem of deploying tasks over the HAT, a fundamental issue in inverting the operator-to-vehicle ratio lies in the potential augmentation of workload. Human operators will have to integrate a large quantity of data from numerous sensors and vehicles, and then make real-time decisions in high-risk environments and under uncertainty (Cummings et al, 2005). As it becomes clear that automation is useful to alleviate several cognitive limits of human operators, there is a growing body of studies trying to determine which level of automation should be used, and when and how to engage this automation (De Greef et al, 2010; Parasuraman et al, 2009; Wickens et al, 2006). Increasing system autonomy can also introduce negative consequences; decreased operator SA and trust (Cummings et al, 2007).

It is recommended that this body of research should be reviewed in more detail to identify specific HMI approaches to support a single operator control multiple UVSs. For example, ‘Playbook’ HMIs and virtual reality can support the tasking of multiple UVSs while keeping the operator in the decision-making loop (Haring et al, 2018; Miller et al, 2005). In addition, the HAI project team should keep a ‘technology-watch’ on any DRDC IDEaS projects that are developing augmented reality applications for supporting operator SA. Any promising concepts that could be applied to the control of multiple of UVSs should be integrated within the SRT HFE experimentation plan.

C.2.6 Summary

The literature review identified several research themes that apply directly to the SRT concepts identified in this report and will be integrated into the future SRT HFE experimentation conducted during the HAI TAC.

Page 135 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 23: Mapping SRT Use Cases against SRT HFE Experimental Themes

SRT HFE Experimental Themes Mule Wingman Artemis RoboMedic Amazon

Visual Perception of Remote Imagery X X X X

Human-Machine Interface Design X X X X X Considerations

Human-Autonomy Teamwork X X X X X

Operator Trust in Autonomous Systems X X X X X

Single Operator Control of Multiple X X Autonomous Systems

Based on the use case mapping against SRT HFE experimental themes, both the Wingman and Amazon use cases provide the most fruitful context for exploring HFE issues pertaining to SRT.

Page 136 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

C.3 SRT HFE EXPERIMENTATION APPROACH

This section of the report presents a detailed description of the SRT HFE experimentation approach that will be adopted to support all HITL research activities conducted during the HAI TAC.

C.3.1 Overview

A formal HITL experimental approach will enable the researcher to manipulate independent variables while controlling for nuisance variables by using controls such as random assignment of subjects to experimental conditions to test a research hypothesis (Hendrick, 2004; Kirk, 1998). Experimentation is essential when designing a system in order to build an understanding of what works and why (Hendrick, 2004).

The following process will be used to design, develop and execute each SRT HFE evaluation outlined in the previous section (Figure 16) (Brown et al, 2005). This process directly maps on to the evaluation component of the CD&E process.

The process comprises five discrete stages, all of which require the engagement of DND stakeholders to ensure that:

• Any high-level issues identified by the stakeholder community (e.g., ease of use, minimal impact on training and mental workload) are addressed by the evaluation; • A validity check of the evaluation against the context of the SRT use cases (i.e., external validity; to be discussed in the next section); and, • The evaluation design and analytical methods that are used generate scientifically and operationally meaningful results.

Page 137 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

STAGE 1: CONNECT WITH DND STAKEHOLDERS

STAGE 2: FORMULATE EVALUATION PLAN

Step 1: Identify high-level evaluation objectives C.2 SRT HFE Experimentation Themes Step 2: Define the evaluation schedule and objectives

STAGE 3: DESIGN AND DEVELOP EVALUATION

C.3.2 SRT HFE Step 3: Define the evaluation design Evaluation Design (variables, experimental design)

C.3.3 Data Collection Step 4: Define the data collection plan C.3.4 SRT Measures of (scenario, measures) Effectiveness and Performance

C.3.5 Overview of Step 5: Define and implement synthetic SRT Research environment research testbed requirements Facilities

STAGE 4: CONDUCT EVALUATION

Step 6: Execute the evaluation according to the plan

STAGE 5: CONDUCT ANALYSIS AND REPORTING

Step 7: Conduct data analysis and C.3.6 Data Analysis documentation activities

Step 8: Identify lessons learned and recommendations for next steps

Figure 16: Overview of SRT HFE Evaluation Planning Process (including applicable sections of the report)

Page 138 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

The test plan comprises the following high-level stages that describe an eight-step process.

• Stage 1: Connect with DND Stakeholders. This stage includes all activities in which stakeholder goals are articulated, and against which the evaluation plan is developed and refined. • Stage 2: Formulate Evaluation Plan. This stage includes activities relating to the formulation of the high-level evaluation objectives, the determination of the schedule for evaluation, and the requirements for the evaluation testbed. The formulation of experiments involves the decomposition of the high-level project goals into empirically addressable questions that are expected to yield objectively clear results. The formulation of evaluations involves two steps:

o Step 1: Define High-Level Evaluation / Demonstration Objectives. Determination of the high-level evaluation objectives and aims of the evaluation experimentation. For example, to establish the utility, usability and impact on operational performance of proposed key SRT functionality.

o Step 2: Define Evaluation / Demonstration Schedule and Objectives. Determination of scheduling, and hardware and software requirements that must be implemented to ensure that the evaluation objectives can be met. For example, specification of the capabilities of the autonomous systems and content of the evaluation scenarios. • Stage 3 Design and Develop Evaluation. This stage includes activities relating to the design and development of the experimental design, measures, and evaluation testbed. Specifically:

o Step 3: Define Evaluation Experimental Design. In an experimental design, the Independent Variable (IV) is the variable which is manipulated or selected by the experimenter to determine its relationship to an observed phenomenon, or the Dependent Variable (DV).

o Step 4: Define Data Collection Plan. Determination of an experimental design that specifies how the IVs are manipulated (i.e., within- or between-subjects), when and how the DVs will be administered, the scenarios used (and counter-balancing requirements), what hypotheses and predictions can be determined.

o Step 5: Identify, Define and Implement Evaluation Test-bed Requirements. Determination of synthetic environment research test-bed requirements from experimental planning activities. For example, the requirement to freeze the scenario so that questionnaires can be administered to participants will have an impact on the implementation of the testbed. • Stage 4: Conduct Evaluation. This stage includes activities relating to the actual execution of the evaluation. Specifically:

o Step 6: Execute Evaluation. The physical execution of the evaluation in terms of the training and briefing of participants, running of the scenarios, the collection

Page 139 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

of data (e.g., administration of questionnaires, behavioural marker observation, etc.), and de-briefing of the participants following the completion of the evaluation. • Stage 5: Conduct Analysis and Reporting. This stage includes activities relating to the analysis, interpretation and reporting of the evaluation. Specifically:

o Step 7: Conduct Data Analysis and Documentation. The data collected during the evaluation can be analysed using a variety of statistical methods. The data analysis is then interpreted in light of the hypotheses and predictions, relevant psychological frameworks of human behaviour, and the operational requirements described in section 3.2 of this report. Finally, the results of the evaluation are documented.

o Step 8: Identify Lessons Learned and Next Steps. The results of the evaluation are appraised for improvements or modifications that can be implemented for the next evaluation. Any recommendations for SRT capability improvements or modifications are identified and documented.

The following sections detail the experimental approach described above.

C.3.2 SRT HFE Evaluation Design

This section provides an overview of the critical components of the SRT HFE experimentation approach for designing a specific HFE evaluation.

C.3.2.1 Research Questions and Hypotheses

The research question presents what will be examined and serves as the foundation of the study. In design studies, the research question typically focuses on system usability and utility (Hendrick, 2004). The research hypothesis is what is predicted; a hypothesis is a ‘if A, then B’ statement predicting the effects of one or more variables on others. The null hypothesis states that there is no effect beyond that caused by chance and is the one being statistically tested (Hendrick, 2004; Kirk, 1998).

C.3.2.2 Variables

There are three types of variables involved in a laboratory experiment (Hendrick, 2004; Kirk, 1998):

• Independent Variable. The causal event under investigation. The independent variable is determined from existing body of research in the domain and theory. A laboratory experiment involves at least two levels of one or more independent variables. • Dependent Variable. What is being measured, such as measures of performance and

Page 140 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

effectiveness described in the following sections. The dependent variables are determined by the following considerations: sensitivity, reliability, distribution, and practicality. • Nuisance Variable. An undesired source of variation affecting the dependent variables. The nuisance variables are limited through experimental controls such as randomization and standardisation of process.

C.3.2.3 Experimental Design

The experimental design refers to the assignation of participants to the experimental conditions (or groups) and the statistical analysis that are associated with the plan. There are two main types of experimental designs, and all have their strengths and limitations (Kirk, 1998):

• Independent Measures (between-subjects). A different group of participants is used in each condition. This is used to measure the difference between two populations. Participants are assigned randomly to one condition. This design requires more participants to achieve a sufficient level of statistical power. In addition, the results may be affected by differences between the groups (e.g., variation in age). • Repeated Measures (within-subjects). All participants are exposed to all conditions. This controls for individual differences, hence reducing the number of participants required to obtain a satisfying level of statistical power. However, the participants are subject to order effect, which can be controlled by counter-balancing the order of presentation of the conditions.

C.3.3 Data Collection

This section describes best practices for collecting data. Mission scenarios, automated data collection, and manual data collection are described below. Finally, the section provides an overview of the research facilities that could support the future SRT HFE evaluations.

C.3.3.1 Mission Scenario

The objective of the Mission Scenario is to provide a framework to support data collection. Typically, the Mission Scenario defines current and future operational protocols, processes and goals, as well as the environment and circumstances that these scenarios must be carried out in from the end user’s point of view (Hsia et al, 1994). The Mission Scenario should include both a defensive and offensive element to cover as many aspects of the mission as possible (Hsia et al, 1994). A Mission Scenario should include:

• Events. Specific stimuli that change the current situation, trigger another event, or both. • Agents. External users, external stimuli, or functional components (like modules or

Page 141 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

objects). • Schema. Sequence of events. The Mission Scenario should be developed in collaboration with SMEs to ensure a sufficient level of functional validity. Functional fidelity (also known as psychological, cognitive or behavioral fidelity) refers to the extent to which a scenario triggers the same behaviors and cognitive activities than as the real world, which is key for the identification of requirements and validation of a system (Hochmitz et al, 2011; Stanton et al, 1996).

The SRT use cases described in this document provide the starting point for constructing a suitable Mission Scenario for an HFE evaluation focusing on a specific SRT theme as described in section C.2. For example, Figure 17 presents the mapping between the SRT uses cases described in this document to the platoon generic mission profile.

MOVE TO ESTABLISH PREPARATION ADVANCE TO REACTION TO LOCATING THE WINNING THE ATTAC K APPROACH ASSAULT CONSOLIDATION PLATOON FOR BATTLE CONTACT ENEMY FIRE ENEMY FIREFIGHT POSITION PATROL BASE

USE CASE #1: ‘Mule’

USE CASE #2: ‘Wingman’

USE CASE #3: ‘Artemis’

USE CASE #4: ‘RoboDoc’

USE CASE #5: ‘Amazon’

Figure 17: Mapping SRT Use Cases against Generic Platoon Mission Profile

In addition, twelve CA dismounted infantry scenarios have been collated by HumanSystems Incorporated and DRDC Toronto in support of the Human System Performance project. Specifically:

1. SIREQ Attack. This scenario describes a section performing a hasty attack during an advance to contact. 2. SIREQ Patrol. This scenario describes the move of a reinforced platoon to conduct point and area recce’s, establish a secure patrol base inside enemy lines, and conduct a platoon size fighting patrol which has been tasked to destroy an enemy installation. This scenario includes airmobile, mounted and dismounted movement, obstacle crossing, reaction to enemy contact, and general conduct of the patrols.

Page 142 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

3. SIREQ Defend. In this scenario a section participates in a company sized defensive position. It includes dismounted movement to a reverse slope defensive position, occupation and preparation of the defensive position, routine in the defence, and conduct of the defence. 4. Platoon House Pearson. This scenario describes a near-peer attack on a CAF platoon house complex. The Platoon House Pearson scenario is divided into three vignettes: attack at dawn, assault from Arbab, and bandits in the wire. 5. FSAR Counter Insurgency Scenario. This scenario details counter insurgency scenarios developed to provide goal-based standardized framework for organizing, describing, and discussing, dismounted infantry weapons engagement tasks in a contextually relevant mission environment. 6. ISS Project Platoon Group Ambush Scenario. This scenario describes a Platoon Group Ambush. The purpose of this scenario is to provide a high-level Use Case to assist stakeholders to understand how ISSP equipment could be used when it is deployed operationally. 7. ISS Project Combat Attack Team. This scenario describes a combat team attack. The purpose of this scenario is to provide a high-level Use Case to assist stakeholders to understand how ISSP equipment could be used when it is deployed operationally. 8. Battle at Combat Outpost Keating. This scenario describes in graphic detail the day-long battle at Combat Outpost Keating and Outpost Fritsche in Afghanistan on 03 October 2009. 9. OPFOR Break Contact. This scenario details an Opposing Force (OPFOR) mission to distribute food to an isolated village to gain public support. Concurrently BLUFOR Infantry Platoon's mission is to conduct security patrols within the Area of Responsibility (AOR). 10. OPFOR Raid. This scenario describes a hybrid threat tactical doctrine and techniques associated with the OPFOR task "Execute a Raid." 11. OPFOR Actions on Contact. This scenario describes the hybrid threat actions and synchronization associated with an OPFOR tactical drill "Actions on Contact." An OPFOR company mission to Secure 120 mm mortar section's movement to Guerilla battalion. 12. Defend a Complex Battle Position. This scenario describes the hybrid threat doctrine and synchronization associated with an OPFOR Tactical Task - Defend a Complex Battle Position (CBP).

The SRT use cases described in this document can be integrated within the HSP scenarios based on the mapping presented in Table 24.

Table 24: Mapping SRT Use Cases against DRDC HSP Project Scenarios

Page 143 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

DRDC HSP Project Scenario Mule Wingman Artemis RoboMedic Amazon

SIREQ Attack X X X

SIREQ Patrol X X X X X

SIREQ Defend X X

Platoon House Pearson X

FSAR Counter Insurgency Scenario X X X X

ISS Project Platoon Group Ambush X X X X Scenario

ISS Project Combat Attack Team X X X X

Battle at Combat Outpost Keating X X

OPFOR20 Break Contact X

OPFOR Raid X X

OPFOR Actions on Contact X X X X

Defend a Complex Battle Position X X X

C.3.3.2 Automated Data Collection

Data can be collected automatically during laboratory experiments. There are multiple ways to automatically collect data (Greenley & Boyle, 2003):

• Video Recording. Using digital camera or a recording software. Ideally, the recordings should be time stamped to facilitate analysis. • Audio Recording. Recording only verbal communications. Once again, the recording should include time stamps. • Time stamped Event Log. Recording of key events from the system being used. This is not available with all systems and it may require extensive analysis.

C.3.3.3 Manual Data Collection

Data can be collected manually by the assessors. There are multiple methods available to manually gather data (Greenley & Boyle, 2003):

• Checklists. The purpose of checklists is to reduce the errors in the data gathering process, to ensure that data can be gathered consistently by a number of observers and across a number of separate evaluation activities, and to provide a mechanism for recording

20 Mapping of SRT use cases for all OPFOR scenarios are from the perspective of the Blue Force.

Page 144 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

observations made by the evaluators. • Behavioural and Communication Analysis. Evaluation of technical and non-technical skills such as performance, collaboration or communication. These analyses are usually conducted online (or offline through video/audio recordings) by an SME and/or an HFE specialist. This is often performed with the support of a checklist or a coding scheme. • Questionnaires. Collect subjective data through a set of written questions. Questionnaires are easy to use and can cover a wide range of subjects. The validity and fidelity of questionnaires vary from one tool to another. When possible, a recognised and validated questionnaire should be used. • Interviews and Post Trial Debriefing. Collect impressions and observations from the participants through a set of questions. This technique allows the researcher to focus on specific aspects of the task, or specific functions of the system.

C.3.4 SRT Measures of Effectiveness and Performance

A Measure of Effectiveness (MOE) is concerned with assessing the ability of a system to achieve its goal; its impact on the operational environment. In this respect, an MOE tends to be more complex than a Measure of Performance (MOP) and may include two or more MOPs in its determination. An MOE focuses on a holistic assessment of how well a system performs overall and can be considered an overarching measurement that comprises more than one MOP. MOEs focus on overall mission goals such as whether the crew met their objectives, completed their mission, minimized casualties and collateral damage, and so on (Dubé & Lamoureux, 2017).

A MOP is a quantifiable measurement that can take any number of forms, such as a simple count of something, a physical measurement, an average, a rate, a percentage, and so on. A MOP may be stated as a declarative statement and indicates a system’s achieved level of performance. A MOP does not provide an assessment of the overall impact of the measurement attribute on the goal of the system. A MOP focuses on the absolute measurement of unidimensional qualities. MOPs, unlike MOEs, directly address the use of the mission system through absolute metrics (how long they spent using something, how many key presses were required) and through more intangible metrics such as workload and situation awareness (Matthews et al, 2002).

In general, MOP frameworks should be based on the following concepts (Matthews et al, 2002):

• Response Time. The response time is the time an operator (or the combat team as a whole) takes to react to a stimulus and execute a response. In other words, response time is the time elapsed between two events, at least one of which must be a system event (i.e. one can, but does not have to be, a scenario event). • Duration. The duration of an action or a state is another widely used MOP. It is measured using the digital capture of time the operator or the combat team took to perform a

Page 145 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

specific task, such as respond to a threat. • Accuracy. Accuracy is often used as a MOP in combination with response time as there is a speed/accuracy trade off. Therefore, both should be measured and plotted. Accuracy measures are largely manual, post-event analysis expressed in the form of percentage or proportions of error. • Errors. The number or proportion of errors committed by an operator is used as an indicator of poor performance. • Detection Rate. Detection rate is the proportion of stimuli that are detected by an operator during a pre-defined period of time. MOPs and MOEs should be tailor-designed based on goals (or an ‘end state’ of a course of action) that must be reached in order to accomplish important mission-related functions, including their sequence and timing. MOPs should also be composed of both direct and indirect (i.e., inferred) measures of operator’s performance.

C.3.4.1 Objective Measures of Performance

MOEs and MOPs need to be designed specifically for the system being evaluated and are identified using a suitable analytical technique. The choice of the analytical technique is dependent on several factors, including the subject matter to be analysed, the experience of the team conducting the analysis, and the level of effort that can be allocated to the analysis.

Figure 18 illustrates a decision tree for the identification of a suitable analysis technique to support the identification of measures of performance (Hou et al, 2014). For example, if a reduction in operator errors was identified as an operational priority by the stakeholders, this would led to the selection of a hybrid Cognitive Task Analysis technique whose emphasis is to understand the sources of these errors, how they occur, why they occur, and when they occur, so that strategies for operator support by the system can be identified.

Page 146 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Figure 18: Decision Tree for Selecting an Analysis Technique for Suitable for Identifying Cognitive Measures of Performance

Page 147 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Table 25: Example MOEs and MOPs Relating to SRT Evaluations

MOPs MOEs Response Time Accuracy Detection Rate

Detection Mean Time from Entry Percentage of Objects Percentage of Tracks into Battlespace to Initial Detected and Tracked Detected Detection

Tracking Percentage of False Tracks Positional Accuracy Track Completeness

Identification Mean Time between Completeness of ID Percentage of time an Initial Detection and Percentage of time a incorrect ID is applied to Correct Identification correct ID is applied to an an object object

Effect Delivery Percentage of successful engagements

C.3.4.2 Subjective Measures of Performance

Subjective measures of performance are self-report or external observations based on the participant’s or evaluator’s own judgement. Although subjective measures are subjected to personal interpretation and opinion, they can provide useful information regarding the experience of the users. This section describes the subjective measures that may be used to evaluate SRT concepts.

C.3.4.2.1 Situation Awareness

An assessment of both individual and team (or shared) SA is required in order to provide accurate measures of SA. As a result, any method that is used to measure operator and team SA should possess three distinct capabilities (Salmon et al, 2006):

1. The technique should be capable of measuring SA simultaneously at different geographical locations. In order to gain a true measure of team of shared SA, each participant involved should be simultaneously assessed for their SA. 2. The technique should be capable of measuring both individual and team or shared SA. Individual team members may possess individual goals, mental models and SA, whilst simultaneously pursuing team goals, and maintaining a level of team or shared SA. 3. The technique should be capable of measuring SA in real-time.

Page 148 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

The following categories of SA measurement techniques are available:

• SA Requirements Analysis. SA requirements analysis represents the first step in any assessment of SA and is conducted in order to determine what actually comprises operator SA in the task or environment under analysis (Endsley, 1993). The procedure for conducting a SA requirements analysis involves the use of unstructured interviews with SME’s, goal-directed task analysis and questionnaires in order to determine the relevant SA requirements. The output of a SA requirements analysis is then used during the development of the SA assessment technique, in order to determine which elements, comprise operator SA and thus, what should be assessed. • Free Probe Techniques. Free probe techniques involve the administration of SA related queries on-line during ‘freezes’ in a simulation of the task under analysis. Typically, a task is randomly frozen and a set of SA queries regarding the current situation are administered. The participant is required to answer each query based upon his knowledge of the situation at the point of the freeze. During these ‘freezes’ all operator displays, and windows are typically blanked. A computer is used to select and administer the queries and also to record the responses. The primary advantage associated with the use of freeze probe techniques is their direct nature. However, freeze probe techniques are criticized for their intrusion upon primary task performance, and also can only be applied where there is a simulation of the task under analysis.

o Situation Awareness Global Assessment Technique (SAGAT). SAGAT was developed to assess pilot SA across the three levels of SA proposed in the three- level model (Endsley, 1995a). SAGAT comprises a set of queries of designed to assess participant SA, including level 1 SA (perception of the elements), level 2 SA (comprehension of their meaning) and level 3 SA (projection of future status). • Real-Time Probe Techniques. Real-time probe techniques involve the administration of SA related queries on-line (during task performance), but with no freeze of the task under analysis. Answer content and response time are taken as a measure of participant SA. The main advantage associated with these ‘real-time’ probe techniques are reduced intrusiveness, due to the fact that no freeze in the task under analysis is required. However, real-time probe queries may also serve to direct participant attention to the required elements in the environment, resulting in biased data.

o Situation Present Assessment Method (SPAM). SPAM distinguishes workload from SA by warning the operator that a question is in the queue and waiting until the operator accepts the question (Durso et al, 1998). SPAM query accept time is measured as the time between when the experimenter asks the participant whether they were ‘ready’, and the time that the participant accepts the question. Following this, the SA question is asked and SPAM query response time is measured as the time between when the experimenter completes asking the question and the time the participant responds (Durso et al, 2004). The logic underlying SPAM is that operators who have better SA will know where to find appropriate information and thus be able to respond faster or more accurately.

Page 149 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

o Situation Awareness for SHAPE (SASHA-L). The Solutions for Human Automation Partnerships in European Air Traffic Management (SHAPE) project developed a technique based on the SPAM technique but with five modifications to enhance it for application in multiple-sector simulations. The SMEs need to be trained to apply the measure and view the controller’s screen, including any decision support tools on a supplementary display terminal, separate from the main simulation room. The queries are formulated by the SME in real time taking into account the real scenario as it unfolds (i.e., the SME asks a question when he/she decides it is pertinent to do so). The queries should be about relationships between two or more items, concern the future and current situation and are rated by the SME as to their operational importance. The queries can also directly address the use of the particular tools or other aspects of the automation support. Finally, the time for the controller to respond to the query is rated as either “OK”, “too long”, or “too short”, the latter indicating perhaps a lucky guess. • Self-Rating Techniques. Self-rating techniques are used to gain a subjective assessment of participant SA. Typically administered post-trial, self-rating techniques involve participants providing a subjective rating of their perceived SA via a SA related rating scale. The primary advantages of self-rating techniques are their ease of application (easy, quick and low cost) and their non-intrusive nature (since they are administered post-trial).

o SASHA-Q. SASHA-Q is administered after the SASHA-L. A questionnaire is completed by the controllers involved at the end of the simulation. Questions relating to tools must be tailored to the specific system being simulated.

o Situation Awareness Rating Technique (SART). SART is a subjective rating technique that was originally developed for the assessment of pilot SA (Taylor, 1990). SART uses the following ten dimensions to measure operator SA: familiarity of the situation; focusing of attention; information quantity; information quality; instability of the situation; concentration of attention; complexity of the situation; variability of the situation; arousal; and spare mental capacity. SART is administered post-trial and involves the participant rating each dimension on a seven-point rating scale (1 = Low, 7 = High) in order to gain a subjective measure of SA. The ten SART dimensions can also be condensed into the quicker 3-dimensional (3-D) SART, which involves participants rating attentional demand, attentional supply and understanding.

o Crew Awareness Rating Scale (CARS). CARS is used to assess command and control commanders SA and workload (McGuinness et al, 2000). CARS comprises two separate sets of questions based upon the three-level model of SA (Endsley, 1995b). The content subscale consists of three statements designed to elicit ratings based upon ease of identification, understanding and projection of task SA elements (i.e. levels 1, 2 and 3 SA). A fourth statement is designed to assess how well the participant identifies relevant task related goals in the situation. The workload subscale also consists of four statements, which are designed to assess

Page 150 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

how difficult, in terms of mental effort, it is for the participant to identify, understand, and project the future states of the SA related elements in the situation. CARS is administered post-trial and involves participants rating each category on a scale of 1 (ideal) to 4 (worst).

o Quantitative Analysis of Situational Awareness (QUASA). QUASA combines participant self-ratings with on-line probes in order to assess actual and perceived SA in military command and control environments (McGuinness et al, 2000). Participants are probed for their SA during task performance and then simultaneously asked to rate their confidence in their answer to the probe in question. QUASA uses true or false probes and a confidence ratings scale (very low – very high) in order to assess actual and perceived SA.

o Situation Awareness and Workload In-Flight Measure (SAW-IM). SAW-IM is a self-rating technique where participants provide a subjective rating of SA via a simple rating scale (Banbury et al, 2004). This is an effective technique to track the profile of crew SA throughout the scenario, as opposed to SART, which provides an aggregate measure of SA over the entire scenario. Subjective ratings of SA “How do you feel about your overall awareness during the last n minutes (or mission vignette)?” are probed at every n minutes (or after each mission vignette) during the evaluation session using a 5-point Likert scale where the ratings go from 1 to 5: (1) behind the situation, (3) with situation, and (5) ahead of the situation. • Observer-Rating Techniques. Observer-rating techniques are commonly used to assess SA during tasks performed ‘in-the-field’. Observer rating techniques typically involve a SME observing participants performing the task under analysis and then providing an assessment or rating of each participants SA. The SA ratings are based upon observable SA related behaviour exhibited by the participants during task performance. The main advantages associated with the use of observer rating scales to measure SA are their non-intrusive nature and their ability to be applied ‘in-the-field’. However, the extent to which observers can accurately rate participant SA is questionable, and multiple SMEs are required. • Performance Measures. Performance measures involve measuring relevant aspects of participant performance during the task under analysis. Whilst performance measures are simple to obtain and are non-intrusive as they are generated through the natural flow of the task, they are beset by a number of problems concerning the relationship between SA and performance. For example, an expert participant may be able to achieve acceptable performance even when his SA is inadequate. Similarly, a novice participant may possess superior levels of SA but still achieve inferior performance, due to other factors such as inexperience.

o Verbal Protocol Analysis (VPA). VPA involves creating a written transcript of operator behaviour as they perform the task under analysis. The transcript is based upon the operator ‘thinking aloud’ as he conducts the task under analysis.

Page 151 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

VPA is used as a means of gaining an insight into the cognitive aspects of complex behaviours and is often used to indicate operator SA during task performance.

C.3.4.2.2 Workload

Workload refers to the total energy output a participant needs to perform a task. Workload may impact human performance through both overload and underload of work (Dubé, 2015). Operator (individual and team) workload analyses should be performed and compared with performance criteria. To avoid overloading or underloading, the degree to which the demands of any task or group of tasks tax the attention, capacities, and capabilities of system personnel (individually and as a team) and thus affect performance shall be evaluated (DND, 2011c).

There are three major types of workload measures: performance-based; subjective; and, physiological and biochemical. However, only the first two types of workload measures will be used for evaluations – performance-based and subjective.

Performance Based Measures of Workload Performance based measures of workload are subdivided into primary-task and secondary-task measures. The rationale is that performance on the secondary task will decline as a function of the demands on the primary task.

• Primary Task. The primary tasks are the tasks whose workload is under consideration. Following are example of primary task measurements:

o Reaction Time. The time between presentation of a stimulus and execution of a response.

o Accuracy. Expressed in the form of percentage or proportions of error. There is a speed/accuracy trade off so both should be measured and plotted.

o Root Mean Square (RMS) Error. For tasks comprising continuous movement (e.g., steering). Error in the form of the distance between the actual and desired position, is recorded at a suitable rate (e.g., 10 Hz). The use of RMS error rather than arithmetic mean penalizes inconsistency.

o Signal Detection Theory (SDT). For vigilance tasks, characterized by the presentation of weak, irregular signals (e.g., radar monitoring), SDT is often applied. SDT is of considerable value in safety-critical applications since it indicates whether poor performance is attributable to difficulty in detecting events or to an inappropriate response criterion. For example, when all signals must be detected and there is little penalty for false alarms, a very strict criterion should not be adopted. SDT responses are correct rejection, miss, false alarm and hit. The application of SDT yields two major measures: . d’ (d-prime). A measure of the discriminability of the signal from noise,

Page 152 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

and corresponds to the distance between the means of the noise and signal plus noise distribution . β. A measure of the individual’s response criterion (i.e., the amount of sensory evidence required before a decision is made that a signal has been presented). With a strict criterion, the individual will commit few false alarms but is likely to miss genuine signals. With a lax criterion, the individual will incur few misses but commit a considerable number of false alarms. In practice, it is not necessary to compute these scores directly from experimental data. Having recorded the proportion of hits and false alarms, the experimenter can consult published tables of d’ and β values.

o SME Ratings. Observer ratings of performance can be used where objective measures are not available. Behaviorally Anchored Rating Scales (BARS), where each point on the scale is assigned a verbal description, are recommended for ratings since scorers are reluctant to use extremes of the scale. • Secondary Task. A secondary task is a task that is artificially added to determine the amount of spare mental capacity available when the operator is performing the primary task. A secondary task should be chosen that is likely to interfere with the primary task. For example, both should require spatial processing since performance of a verbal secondary task may be insensitive to changing demands of the primary task, since it is not in contention with this task. Following are examples of widely used secondary task measures:

o Interval Production. The individual is asked to tap at a specified rate. As workload increases, the intervals between taps become increasingly variable.

o Time Estimation. The individual is asked to estimate how much time has elapsed (e.g., since the start of a session). In general, time intervals are progressively under-estimated as workload increases (Casali et al, 1983).

o Random Number Generation. As workload increases, the individual may resort to well-learned sequences (e.g., 1, 2, 3). This decrease in randomness can be quantified mathematically (Zeitlin et al, 1975).

o Probe Reaction Time. A stimulus unrelated to the primary task appears periodically; RT to this stimulus is assumed to reflect the demands of the primary task. • Psychophysiological Measures. There are a number of commercially available systems that interpret behavioural and physiological measures (e.g., electroencephalogram, galvanic skin response and heart rate) within a contextual model, and derive high order estimates of cognitive–affective status (Hou et al, 2014).These estimates are expressed as higher- state descriptors (i.e., raw data is interpreted to infer cognitive states rather than sole cardiac activity, for instance) of executive load, visual load, and alertness, as well as verbal and spatial load. This approach ensures that a fine-grained assessment of workload can be made alongside a consideration of SRT interactions, as opposed to a global subjective

Page 153 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

workload rating aggregated over a period of time. Despite the relative complexity of using such measures, it comprises a series of advantages that should be considered. Indeed, psychophysiological measures can provide a non-obtrusive measure of workload (as opposed to questionnaires, for instance). Moreover, the sampling rate of psychophysiological measures is much higher than any other type of measures, enabling the precise assessment of critical periods in the scenario and a more representative overall assessment of workload. In addition, psychophysiological measures of workload are not subject to judgmental biases that may compromise the validity of subjective assessment techniques. One cannot lie about his physiological response to an event. Finally, this type of measure being collected in real-time may eventually be used to trigger adaptions by the autonomous system to changing operator demands. Subjective Measures of Workload Subjective measures of workload include both uni- and multi-dimensional scales.

• Unidimensional Scales. Unidimensional scales attempt to elicit a measure of overall workload. Following are examples of widely used unidimensional scale measures of workload:

o Situation Awareness and Workload In-Flight Measure (SAW-IM). The SAW-IM is a self-rating technique, described above, that is used to assess both SA and workload (Banbury et al, 2004). Participants provide a subjective rating of workload during the evaluation. As for SA, this is a very effective technique to track the profile of crew workload throughout the scenario, as opposed to the NASA-TLX which provides an aggregate measure of workload over the entire scenario. Subjective ratings of workload “How do you feel about your overall workload during the last n minutes (or mission vignette)?” are probed at every n minutes during the session (or after each mission vignette) using a 5-point Likert scale where the ratings go from 1 to 5: (1) underload, (3) optimal, (5) overload.

o Modified Cooper-Harper Scale. Based on a scale that was originally used for aircraft handling (Cooper et al, 1969). The scale describes aircraft handling on a 1 to 10 scale, where low numbers represent desirable handling characteristics. It requires the pilot to make a series of binary decisions about the handling qualities of the aircraft he or she is evaluating in response to a certain control input or when performing a pre-defined task.

o Integrated Workload Scale (IWS). The Integrated Workload Scale (IWS) was developed and tested for signalers (Pickup et al, 2005). The scale describes mental workload on a nine-point scale from: (1) not demanding, to (9) work too demanding – complex or multiple problems to deal with and even with very high levels of effort it is unmanageable. • Multidimensional Scales. Multidimensional scales address individual components of workload and hence are potentially of some diagnostic value in determining the source of any workload problem. Following are workload assessment techniques using

Page 154 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

multidimensional scales:

o Subjective Workload Assessment Technique (SWAT). Combines ratings of 3 different scales to produce an interval scale of mental workload (Crabtree et al, 1984). These scales are: . Time load. Reflects the amount of spare time available in planning, executing, and monitoring a task; . Mental effort load. Assesses how much conscious mental effort and planning are required to perform a task; and . Physiological stress load. Measures the amounts of risk, confusion, frustration, and anxiety associated with task performance.

o NASA Task Load Index (TLX). Designed to obtain workload estimates from one or more operators while they are performing a task or immediately afterwards (Hart et al, 1988). NASA TLX comprises six subscales: . Mental Demand. How much mental demand and perceptual activity was required? Was the task easy or demanding, simple or complex, exacting or forgiving? . Physical Demand. How much physical activity was required? Was the task easy or demanding, slow or brisk, slack or strenuous, restful or laborious? . Temporal Demand. How much time pressure did you feel due to the rate or pace at which the task or task elements occurred? Was the pace slow and leisurely or rapid and frantic? . Effort. How hard did you have to work (mentally and physically) to accomplish your level of performance? . Performance. How successful do you think you were in accomplishing the goals of the task set by the analyst (or yourself)? How satisfied were you with your performance in accomplishing these goals? . Frustration level. How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task? For each of the subscales, the participants are asked to rate their workload for the preceding period based upon an interval scale divided into 20 intervals, ranging from low (1) to high (20), during and after the experimental trial. The second part of the NASA TLX results in an individual weighting for each subscale by asking subjects to select the subscale with the greatest impact on workload in a systematic pairwise consideration of the subscales (i.e., 15 pairwise comparisons). This procedure accounts for differences in the sources of workload between tasks and differences in workload definition between raters. o DRA Workload Scales (DRAWS). Developed by the Defence Research Agency

Page 155 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

(DRA), and based on factor analysis of a large body of performance and workload data and were subjected to validation studies during their development (Jordan et al, 1995). Involves verbally querying the participant for a subjective rating between 0 (low) to 100 (high) for each dimension during task performance. Provides information on the four dimensions of workload: . Input demand. Demand associated with the acquisition of information from any external sources; . Central demand. Demand associated with the operator’s cognitive processes involved in the task; . Output demand. Refers to the demand associated with any required responses involved in the task; and . Time pressure. Refers to the demand associated with any time constraints imposed upon the operator.

o Subjective Workload Dominance (SWORD) Technique. Uses relative comparisons whereby each task is individually compared to all other tasks (Vidulich et al, 1989). Furthermore, when using SWORD tasks are evaluated retrospectively after having completed all tasks. The SWORD method is based on a paired comparison and uses three main steps: collecting the raw judgement data; constructing the judgement matrices; and, calculating the SWORD ratings. As SWORD is based on paired comparison the number of tasks which can be compared is limited. The number of comparisons is 1/2n (n-1). When too many tasks have to be compared (e.g., 45 comparisons with 10 tasks) memory is overloaded.

C.3.4.2.3 Usability/Utility/User Acceptance

The ‘usability’ of a system is not an absolute concept. It is more of a general quality of the appropriateness to a purpose of a specific tool or system, which is dependent on the intended users of the system, the tasks to be performed, and the characteristics of the environment (physical, organisational and social) in which the system will be used (Brooke, 1996). The Technology Acceptance Model is an information systems theory that models how operators come to accept and use a technology (Davis, 1989). Two factors influence the perception of the system by the users:

• Perceived Usefulness. This is defined as ‘the degree to which a person believes that using a particular system would enhance his or her job performance’ (i.e., utility). For SRT experimentation, the military utility (i.e., impact on operational effectiveness) should be assessed as a measure of perceived usefulness (Bartik et al, 2019). • Perceived Ease of Use. This is defined as ‘the degree to which a person believes that using a particular system would be free from effort’ (i.e., usability).

Page 156 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Multiple measures and metrics exist to assess usability and can be classified into three categories:

• Effectiveness. The accuracy and completeness with which users achieve specified goals. These measures are closely related to MOPs and MOEs. Examples of these measures of usability include:

o Task Success. Scored as success or failure. Those measures need a clear end-state to be able to confirm task success. For example, reporting the percentage of success is a good measure of effectiveness of transaction (Albert et al, 2013).

o Errors. Refers to the number and frequency of errors committed during the test session (Albert et al, 2013).

o Completeness & Quality of Outcome. The extent to which tasks are solved to a satisfactory level. Quality of outcome comes into play in circumstances where the end result cannot be classified as a success or a failure (Hornbæk, 2006). • Efficiency. The resources expended in relation to accuracy and completeness with which users achieve goals. These measures are also closely related to MOPs and MOEs. Examples of these measures of usability include:

o Raw Efficiency. These measures are used when a user must complete the same transaction several times and a comparison can be made from one time to the other. For example, measuring task completion per unit of time (Albert et al, 2013).

o Lostness. This is an efficiency-based measure of usability where the number of steps it takes a user to complete a task is compare to the minimum number of steps possible to successfully accomplish this task (Albert et al, 2013).

o Learnability. Refers to the time or effort required to achieve maximum efficiency (Albert et al, 2013).

o Task Time. Defined as the time it takes a user to complete a set of core tasks. These measures reveal effort involved in achieving the task. They are helpful to compare task completion time to expert’s time (Albert et al, 2013).

o Mental Effort. Refers to the mental resources spent by the user to accomplish a task. This measure of usability is tightly linked to workload (Hornbæk, 2006).

o Usage Patterns. Refers to how the interface is used. This is indicative of the resources required to achieve the task. An example of usage pattern measure would be the number of keystrokes or mouse clicks required to perform a task (Hornbæk, 2006).

o Communication Effort. Defined as the number of communications between team members required to accomplish a collaborative task. This is tightly linked to communication analysis and anticipation ratio described in the following section (Hornbæk, 2006).

Page 157 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

• Satisfaction. The freedom from discomfort, and positive attitudes towards the user of the system. Examples of these measures of usability include:

o System Usability Scale (SUS). SUS is a ten-item subjective tool that provides a general assessment of usability. The following aspects of usability are assessed: need for support; training; and, complexity. The inclusion of various aspects of usability ensures a high level of face validity for measuring usability of a system (Brooke, 1996).

o Usefulness, Satisfaction, and Ease of Use (USE). The USE questionnaire was developed to measure usability of software, hardware, services, and user support materials across domains. USE measures usability on three dimensions: usefulness; satisfaction; and, ease of use with a set of 22 questions. USE was not intended to be a diagnostic tool, but rather treats the dimensions of usability as dependent variables (Lund, 2001).

C.3.4.3 Measures of SRT Effectiveness

The following measures are directly applicable to evaluating the effectiveness of autonomous systems to support SRT.

C.3.4.3.1 Trust

Trust can be defined as the extent to which an operator is confident in and willing to act on the basis of the recommendations, actions, and decisions of an automated aid (e.g., contact identification propositions). Trust is a multidimensional concept including correct trust and distrust, but also over and under trusting. Operator trust and acceptance of automation is both affect-base and cognitive based (Madsen et al, 2000). Trust is determined from the outcome of a comparison process between the perceived reliability of the automated aid (i.e., trust in aid) and the perceived reliability of manual control (i.e., trust in self). Trust is an important concept as it plays a role in how automation is used or planned to be used (Jian et al, 2000).

• Human-Computer Trust Questionnaire. This questionnaire is a subjective measure for assessing affect-based and cognitive based dimensions of trust. HCT includes five factors (Madsen et al, 2000):

o Perceived Reliability. Reliability of the system, in the usual sense of repeated, consistent functioning.

o Perceived Technical Competence. Technical Competence of the system meaning that the system is perceived to perform the tasks accurately and correctly based on the information that is input.

o Perceived Understandability. Understandability in the sense that the human supervisor or observer can form a mental model and predict future system behavior.

Page 158 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

o Faith. Faith meaning that the user has faith in the future ability of the system to perform even in situations in which it is untried.

o Personal Attachment. Personal Attachment to the system comprised of: liking meaning that the user finds using the system agreeable and it suits their taste and loving meaning that the user has a strong preference for the system, is partial to using it and has an attachment to it. • Autonomous Warrior 2018 System Trustworthiness Questionnaire. A trust questionnaire was developed to evaluate an operator’s trust using a software tool to automatically task and monitor multiple UVSs during the Autonomous Warrior 2018 exercise (Bartik, 2019). The questionnaire used a seven-item Likert scale to assess the following trust dimensions:

o Reliability. The ability of the system to operate on missions, perform tasks, and deliver effects as specified.

o Dependability. The ability of the system to be relied upon to operate on missions, perform tasks, and deliver effects as specified.

o Predictability. The ability of the system to respond to events and to operate, perform and deliver effects consistently and reliably as planned and anticipated.

o Availability. The ability of the system to operate on missions, perform tasks, and deliver effects when requested.

o Resilience. The ability of the system to transform, renew and recover in timely response to events.

o Safety. The ability of the system to operate without harmful states. o Security. The ability of the system to remain protected against accidental or deliberate attacks.

C.3.4.3.2 Teamwork

The use of multidisciplinary teams reduces individual workload, increases overall knowledge and expertise, as well as resilience – by the overlap of knowledge and expertise between team members (Loerger, 2003). Teamwork is known to increase efficiency, but also complexity as it requires functions and behaviours that are not observed in individual work (Loerger, 2003; Salas et al, 2005; Tremblay et al, 2010). These team functions and processes have a direct influence on team performance (McIntyre et al, 1995; Salas et al, 2004; Stout et al, 1999). The following techniques can be used to evaluate teamwork in manned-unmanned teams:

• Human-Machine Teaming. The Human-Machine Teaming questionnaire focuses on how well an autonomous system supports human-machine teaming in order to identify any gaps, deficiencies, and future opportunities for operator support. The questionnaire comprises the following factors that are known to effect human-machine teaming

Page 159 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

effectiveness (McDermott et al, 2017). Specifically:

o Observability. Ability of system to communicate to the human what it’s thinking, doing, and how far along it is in accomplishing joint work.

o Predictability. Ability of system to communicate with human about its’ intentions, goals, and future actions in various contexts.

o Directability. Ability of human to direct or redirect the tasks, resources, and priorities of an automated partner.

o Directing Attention. Ability of system to notify human and present time-critical or task-critical information in a helpful way, so that human can stay aware of what’s important and act proactively.

o Exploring Solution Space. Ability for system to help problem solve by suggesting things to consider and offering alternative suggestions.

o Adaptability. Ability for system to recognize and handle unplanned situations. o Calibrated Trust. Ability of system to be clear about what it is good at and what its shortcomings are so human can determine how much and when to trust it.

o Common Ground. Ability for human and the system to have a two-way street of communication about each other’s intentions, beliefs, and assumptions.

o Information Presentation. Ability of system to present information so that it is easily understandable and does not require unnecessary work to calculate or interpret. • Team Communication. Communication is the process by which task-relevant information (i.e., SA requirements) and task strategies are shared between team members. Communication analysis involves two measures: frequency and content analysis.

o Frequency. Communication frequency is a simple measure that has been linked to team performance and workload. The relation between communication frequency and performance is not linear, with studies showing both positive and negative correlations with between performance and communication frequency (Brannick et al, 1993; Cannon-Bowers et al, 1998; Demir et al, 2017; McNeese et al, 2018; Sexton et al, 2000). Communication frequency is calculated by counting the number of times each team member speaks (or send a message if this function is available). This can be done manually, by listening to the audio recording of an experimental task or simulation session, or automatically, using a speech recognition software. Communication frequency can be used in conjunction with content analysis, where the number of communications per categories is compared.

o Content Analysis. Content analysis of the communication refers to the assessment of the semantical content of the communications. It is an unobtrusive tool for making replicable and valid inferences from verbal, visual, or written data to the

Page 160 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

contexts of their use, involving a set of specialized procedures (Krippendorff, 2013). Content analysis involves manifest content (i.e., tangible or concrete surface content of the communication) and latent content (i.e., underlying meaning of the communication) (Krippendorff, 2013). Content analysis is time and resource intensive as it requires three or more analysts to listen to the audio recording of an experimental task or simulation session and categorize each parts of the communication using a predetermine coding scheme (Krippendorff, 2013).

o Anticipation Ratio. The anticipation ratio (AR) is a measure of communication efficiency within a team (MacMillan et al, 2004). AR is obtained by classifying communication into transfer of information or request for information. The number of information transfers is then divided by number of requests for information (Spradley, 1979). AR is usually calculated for the team as a whole. AR values higher than one demonstrate that information is "pushed" (sent) to the team more frequently than it is "pulled" (requested), showing that team members anticipate each other's needs for information before being asked for it (Sarcevic et al, 2008).

o Social Network Analysis. The social network analysis (SNA) is an approach for the study of social interactions that describes relationships between entities in terms of communications, relationships, or transactions. A matrix of relationships is used to construct a social network graph and run mathematical analyses (Houghton et al, 2006). SNA includes three measures (Benta, 2005): . Centrality. Degree of access to information of each team member. At the network level, this coefficient provides an insight into the distribution of information within a group. . Sociometric Status. Individual level of communicational activity inside the team. . Determination Degree. Index of the dominance that team members receive from their teammates. This index can be positive or negative. A positive value suggests that the team member is mostly dominated, i.e., is mostly a consumer of information. A negative value suggests that the member is a dominating agent (i.e., a producer of information).

o Cognitive Network Tracing. Cognitive Network Tracing (CNT) is a measure of the team responsiveness to stressors (Banbury et al, 2001). CNT involves the deliberate propagation of scenario events, or ‘seeds’, in the simulation, followed by the observation of their trajectory throughout the team. To be effective, a seed must be both critical enough to demand action by team members, and salient enough for the experimenter to observe their subsequent effect on team members’ behaviour. The team responsiveness to the seed is obtained by calculating the time it took the team to address the seed and change their course of action.

Page 161 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

• Coordination. Coordination is the process by which a task strategy is formulated in order to coordinate and prioritise the task-related activities of all team members (Stanton et al, 2017). There are two types of coordination: explicit coordination (i.e., achieved through communication, to articulate plans, actions and responsibilities, and the sequentialization tasks (Espinosa et al, 2004)), and implicit coordination (i.e., acting in concert without overt communication, based on a shared understanding of the tasks and situation (Blickensderfer et al, 2010)).

o Coordination Coding System. A coding system providing descriptive observation of team coordination. The coding system includes four categories of coordination (Kunzle et al, 2010): . Implicit Coordination. Team member A anticipates the information and resources needed by B and uses verbal communication or silent action to provide the information or help without being asked to do so. . Explicit Coordination. Team member A uses open verbal or non-verbal communication to exchange information with other team members, to clarify the situation and coordinate action. . Leadership Coordination. Team member A takes the lead by making plans, assigning tasks or instructing B as the need arises and is not met by others. The focus is on leadership activity exercised by any team member, and not necessarily by the formal lead. . Heedful Interrelating. Team member A works to help the team achieve good performance by carefully following the actions of other team members and continuously adjusting her own behaviour to theirs in tasks. She is alert to any timely, personal or medical consequences of actions taken.

o Shared Situation Awareness. Shared SA is defined as “the degree to which team members possess the same SA on shared SA requirements” (Endsley et al, 2001). Although often used interchangeably, team SA can take a slightly different meaning and be defined as “coordinated perceptions and coordinated actions” (Gorman et al, 2006). From these definitions, it is possible to conclude that shared SA is comprised of two dimensions (Saner et al, 2009): . Accuracy. How close to ground truth is one’s the understanding of the situation. This concept is highly related to mental model of the situation. It refers to the individual information requirements for SA. . Similarity. How close the understanding of the situation elements is between two participants. This refers to the overlap of information requirements among the team members.

o SAGAT for Teams. The measure of shared SA the most widely used is an adaptation of the individual SA measure SAGAT (Saner et al, 2009). SAGAT for teams is a knowledge-based objective measure assessing both the accuracy and

Page 162 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

the similarity of SA. SAGAT for teams is identical to the individual version; only the calculation of the score changes. SAGAT for Teams queries are scored individually in terms of the proportion correct answers to measure accuracy of SA. Each participant’s data are then paired. A similarity score is calculated for each individual pair. Finally, the mean of the similarity scores is calculated.

o Coordinated Awareness of Situation by Teams (CAST). The Coordinated Awareness of Situation by Teams (CAST) is an implicit performance-based measure focusing on adaptive behaviors to non-routine events (Gorman et al, 2006). CAST uses ‘roadblocks’ integrated in a simulation scenario. Roadblocks are trigger events forcing the team to change their course of actions.

o Transactive Memory System Scale. Transactive Memory System (TMS) refers to the cooperative division of labor for learning, remembering, and communicating relevant team knowledge (Wegner, 1987). TMS includes three dimensions: . Specialization. A differentiated structure of members’ knowledge; . Credibility. The members’ beliefs about the reliability of another members’ knowledge; and, . Coordination. An effective, orchestrated knowledge processing. The TMS questionnaire is comprised of fifteen items rated on a five-point scale, ranging from (1) totally disagree to (5) totally agree, where a higher score represents a better knowledge of each other’s expertise, trust, and coordination (Lewis, 2003). The questionnaire is task-independent, allowing for comparisons between different teams and tasks. Confirmatory factor analyses supported that three factors, corresponding to the three subscales of the questionnaire (i.e., specialization, credibility, and coordination), do underlie the construct. Convergent, discriminant, and criterion-related validity tests in all three samples suggested that the scale behaved as expected because it was related to similar constructs, and distinct from constructs it is not intended to measure.

C.3.4.4 Summary

Previous sections of this Annex have reviewed many objective and subjective techniques to assess SRT performance and effectiveness across several different perspectives – usability and utility, SA, workload, trust and teamwork.

As the experimentation roadmap is defined in more detail as the HAI project progresses, the selection of specific techniques will be made based on their applicability to the empirical objectives (e.g., experimental design and research hypotheses) and the limitations and constraints of the proposed study (e.g., facilities, access to participants, maturity of technologies to be assessed). However, at this point we can determine the applicability of the general classes of measures (i.e., usability and utility, SA, workload, trust and teamwork) to the main SRT experimentation themes outlined at the beginning of this section. As described in Table 26, all

Page 163 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

classes of measures of effectiveness and performance are directly relevant to all SRT HFE experimentation themes; however, more weight should be given to specific classes based on the aims and objectives of the study.

Table 26: Mapping SRT Measures of Effectiveness and Performance against SRT HFE Experimentation Themes

SRT HFE Usability and Situation Experimental Workload Trust Teamwork Utility Awareness Themes

Visual Perception of HIGH HIGH MEDIUM LOW LOW Remote Imagery

Human-Machine Interface Design HIGH HIGH HIGH HIGH HIGH Considerations

Human-Autonomy HIGH MEDIUM MEDIUM HIGH HIGH Teamwork

Operator Trust in Autonomous HIGH HIGH MEDIUM HIGH MEDIUM Systems

Single Operator Control of Multiple HIGH HIGH HIGH HIGH HIGH Autonomous Systems

C.3.5 Overview of SRT Research Facilities

The following data collection facilities are available to the project team to conduct SRT HFE evaluations.

C.3.5.1 DRDC Toronto Research Centre

The Testbed for Integrated Ground Control Station Experimentation and Rehearsal (TIGER) is a game-based system comprising commercial off-the-shelf technologies to enhance simulation fidelity and backbone architecture (see Figure 19). TIGER’s gaming software is capable of rendering extremely realistic environments, high fidelity entities with articulated parts, material specific infrared data, and highly detailed urban environments equivalent to commercial video games. TIGER uses X-Plane 9 for the flight models and renders terrain and ground features using CryEngine 3 (or other similar applications).

Page 164 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Figure 19: TIGER Facility – DRDC Toronto Research Centre

Each workstation consists of associated monitors and the software enabling realistic mission emulation (e.g. threat generation software, flight simulation software, simulated sensor controls, imagery analysis software, intelligence gathering, communications, chat and mapping). Computers are housed in a rack which is located away from the participants.

All workstations have the following standard GCS components:

1. Standard Communication Suite. This suite comprises: (i) a voice radio with hot mike, push to talk, footswitch and headset; (ii) a chat client; (iii) a keyboard and mouse; and, (iv) a monitor for chat. 2. Situation Awareness Displays. These displays comprise: (i) sensor imagery; (ii) tactical map; and (iii) Sensor C2 Planning Software. 3. Planning and Reporting Tools. These tools comprise: (i) portable flight planning software; and, (ii) Microsoft Office. In addition, the following role-specific functionality is provided for each of the following workstations:

1. Air Vehicle Operator Workstation. This workstation also comprises flight controls and flight instrument display. 2. Payload Operator Workstation. This workstation also comprises a joystick to control the payload (i.e., visual sensor orientation, wavelength, zoom, firing of laser designator), and an interface for sensor configuration and control. 3. Image Analyst Workstation. This workstation comprises the Multi-INT Analysis and

Page 165 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Archive System (MAAS). 4. Image Reporter Workstation. This workstation comprises MAAS. 5. Electronic Warfare Intelligence Analyst and Reporter Workstations. A suite of bespoke Electronic Warfare tools was developed specifically for TIGER. These include a separate window with five frequency scanners and a waterfall display. In addition, lines of bearing are plotted on the map display. 6. Experimenter and White Force Workstations. These workstations also comprise TIGER simulator control software and Computer-Generated Forces (CGF) control software, as well as data collection and analysis tools.

C.3.5.2 DRDC Suffield

Research areas at the Centre focus on military engineering, mobility and autonomous systems, weapons system evaluation and chemical-biological defence. These scientific and technological activities are supported by meteorological, photographic, information, design and development, materiel management and field support services.

The DRDC Autonomous Systems Operations groups are the primary research groups for land robotics and autonomous systems within DRDC. Their current research focus involves defeating autonomous systems, improving the dismounted soldier's effectiveness with robotic systems, and exploring new capability for small UVS CBRNE detection and identification. Past research programs have looked at mapping, localization and navigation, intelligent mobility, and large UGV-based CBRNE standoff detection (Collier et al, 2019).

DRDC Suffield has several autonomous assets available for SRT HFE evaluations, including multi- copters, rotorcraft, and fixed wing UAVs. Many of these UAVs are capable of the following autonomous functions:

1. Precision hover without Global Positioning System; 2. Automatic take-off and landing; 3. Waypoint navigation; 4. Visual target tracking; 5. Swarming; and, 6. Obstacle avoidance.

C.3.5.3 University of Toronto Institute for Aerospace Studies

The University of Toronto Institute for Aerospace Studies (UTIAS) is an advanced research facility for aeronautics and aerospace engineering, located in the Downsview district of Toronto,

Page 166 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Ontario, Canada. Established in 1949 by founding Director Gordon N. Patterson, the institute is managed by the University of Toronto Faculty of Applied Science and Engineering and mainly receives funding from governmental agencies such as the National Research Council, the Department of National Defence and the Canadian Space Agency. Notable international sponsors include the European Space Agency, Lawrence Livermore National Laboratory, Max- Planck-Institut für Plasmaphysik, NASA Ames Research Center and the United Kingdom Atomic Energy Authority.

C.3.6 Data Analysis

This section describes best practices for analysing data. Qualitative data analysis methods and quantitative data analysis methods are described below.

C.3.6.1 Qualitative Data Analysis

Qualitative data are non-numerical data, typically collected through observations, interviews, or post-experiment de-briefing sessions. Qualitative data are used to approximate and characterize users’ traits, characteristics and behaviors (Dey, 2003).

C.3.6.1.1 Classification of Qualitative Data

Analysis of qualitative data begins with the development of a conceptual framework under which data will be organised. The conceptual framework is based on theoretical concepts and on relevant empirical material. In other worlds, the categories that constitutes the conceptual framework: “must have two aspects, an internal aspect—they must be meaningful in relation to the data—and an external aspect— they must be meaningful in relation to the other categories” (Dey, 2003). Each category of a conceptual framework reflects a criterion against which the data will be assessed. The categories permit to grouping similar observations and differentiate dissimilar observations.

C.3.6.1.2 Formatting Data

The choice of a format depends on the goal of the analysis, and on the conceptual framework against which data will be classified. There are two main formats used in qualitative data analysis (Miles et al, 1994):

• Matrices. These are a tabular representation of the data, with defined rows and columns. Matrices collect all the data in one place, under a format that is easy to view, and facilitate comparison between categories. The data listed in a matrix are abstractions and do not include quotes. • Networks. These are a collection of nodes interconnected by links illustrating the relations between users’ actions, events, and processes. Networks are useful to illustrate complex interrelationships between variables and analyze trends.

Page 167 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

C.3.6.1.3 Qualitative Analysis Methods

There are various methods for to analyse qualitative data. The selection of a technique depends on the objective of the analysis and the nature of the data collected. The following methods can be applied to SRT evaluations.

• Constant Comparative Analysis. Constant comparative analysis consists of taking one piece of data and comparing it with all others to determine how similar or different they are. Constant comparative analysis is used to conceptualise the relationship existing between pieces of data. It is used when the analysis is supported with a solid theoretical background. This technique produces descriptive knowledge from a set of data (Thorne, 2000). • Word Counts. Word counts suppose that each user has a distinctive vocabulary and word usage pattern referred to as “linguistic fingerprints” (Leech et al, 2007). By counting the number of times some key words are pronounced by a user, researchers can better understand its perspective on a specific subject, as well as what is important for them. Word counts can be used to:

o Identify patterns more easily; o Verify a hypothesis; and, o Maintain analytic integrity. Word counts improve the rigor of a qualitative analysis by providing an objective view on the data-set, reducing the likelihood of overweighting or underweighting the importance of a theme (Sandelowski, 2001). However, word counts do not take into consideration the context in which a word is being used, which may reduce the value of the analysis (Miles et al, 1994). Word counts are particularly useful in a group setting, where it can be used to identify which participants contributed the most, and who contributed the least (Lewis, 2003). • Taxonomic Analysis. A ‘taxonomy’ is a classification system cataloguing subjects, or domains, in the form of a flowchart or diagram illustrating the relationships between them (Spradley, 1979). Taxonomic analysis, therefore, is the classification of information, or observation, in an organisation of interrelated categories. Eight steps are required to perform a taxonomic analysis (Lewis, 2003): 1. Select a domain for taxonomic analysis; 2. Identify the appropriate substitution frame for analysis; 3. Search for possible subsets among the included terms; 4. Search for larger, more inclusive domains; 5. Construct a tentative taxonomy; 6. Formulate structural questions to verify taxonomic relationships;

Page 168 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

7. Conduct additional structural interviews; and, 8. Construct a completed taxonomy. C.3.6.2 Quantitative Data Analysis

Quantitative data refers to any data that is presented in the form of counts (or numbers), and where each data-set is associated to a specific numerical value (Surendran, 2019). Typically, quantitative data is analysed using statistical analyses. There are many statistical analysis techniques; each having their own advantages and disadvantages. There are two broad categories of statistical analyses:

• Parametric Tests. These can be conducted when the dataset respects the following conditions:

o The level of measurement is of equal interval or ratio scaling; o The distribution of the population scores is normal; and, o The variances of both variables are equal or homogeneous. • Non-Parametric Tests. These do not depend on assumptions about the precise form of the distribution of the sampled populations. The choice of a statistical analysis is guided by the type of data present in the dataset (Bryman et al, 2012):

• Categorical Data. The values of the data refer to the number or frequency of cases that fall within specific categories, where there are two or more categories, but there is no intrinsic ordering to the categories. In this case, only non-parametric test can be used. • Ordinal Data. The values of the data refer to the number or frequency of cases that fall within specific categories, where there is a clear ordering of the variables. In this case, both parametric and non-parametric tests can be used. • Interval Data. The values of the data refer to the number or frequency of cases that fall within specific categories, where there is a clear ordering of the variables, and the intervals between the values of the interval variable are equally spaced. In this case, both parametric and non-parametric tests can be used. • Ratio Data. The values of the data refer to the number or frequency of cases that fall within specific categories, where there is a clear ordering of the variables, and the intervals between the values of the interval variable are equally spaced, and there is an absolute “zero” being a treated as a point of origin so it is impossible to have negative value. In this case, both parametric and non-parametric tests can be used.

C.3.6.2.1 Univariate Analyses

Page 169 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Univariate analyses focus on information relating to a single variable. There are two broad categories of univariate analyses:

• Central Tendency. Central tendencies show where the values in a distribution tend to concentrate. There are three major measures of central tendencies (Bryman et al, 2012):

o Arithmetic Mean. Consists of adding up all of the values and dividing by the number of values in the dataset. The arithmetic mean is easy to understand and to interpret but is highly influenced by extreme values.

o Median. Refers to the mid-point in a distribution of values. It splits a distribution of values in half. It is less influenced by the extreme values.

o Mode. Refers to the value that occurs most frequently in a distribution. • Dispersion. Measures of dispersion refer to the amount of variation present in the dataset. There are four main measures of dispersion (Bryman et al, 2012):

o Range. Corresponds to the difference between the highest value and the lowest value of a dataset. Range is highly influenced by extreme values.

o Inter-Quartile Range. Refers to the division of the dataset into quartiles. This is obtained by dividing the dataset into four equal portions; with the lowest 25 per cent in the first portion and the highest 25 per cent in the last portion. Inter- quartile range is useful to eliminate extreme values. However, information is lost as 50 per cent of the dataset is ignored.

o Decile Range. Refers to the division of the dataset into deciles. This is similar to the inter-quartile range, except that the dataset here is divided in ten portions. The advantage here is that only 20 per cent of the dataset is ignored.

o Standard Deviation. Corresponds to the degree to which the values in a dataset differ from the arithmetic mean of this same dataset. The standard deviation is interpreted in relation to the arithmetic mean.

C.3.6.2.2 Bivariate Analyses

Bivariate analyses refer to statistical methods that examine the connections between two variables. There are a relatively large number of statistical tests to determine whether a difference between two or more experimental groups is significant (Bryman et al, 2012). Only the bivariate analyses deemed relevant to this SRT evaluations are presented here.

• Binomial Test. Compares the frequency of cases actually found in the two categories of a dichotomous variable with those that are expected on some basis. • Chi-Square Test for One Sample. Compares the observed frequencies of a group with the expected frequencies of that same group for variables with more than two categories. • t-Test for One Sample. Determines if the mean of a sample is similar to that of the

Page 170 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

population. • t-Test for Two Unrelated Means. Determines if the means of two unrelated samples differ. This is done by comparing the difference between the means of each sample with the standard error of the difference in the means of different samples. • One-Way Analysis of Variance. Compares the means of three or more unrelated samples with an F-test. An F-test is a comparison of an estimate of the between-groups variance and an estimate of the within-groups variance, where the former is divided with the latter. The total amount of variance is comprised of the variance due to the independent variable and the variance due to other factors, (i.e., the error or residual variance). If the between-groups estimated variance is considerably larger than that within-groups, then the value of the F ratio will be higher, which implies that the differences between the means is unlikely to be due to chance.

C.3.6.2.3 Multivariate Analyses

Multivariate analysis is an extension of bivariate regression in which two or more independent variables are taken into consideration simultaneously to predict a value of a dependent variable for each subject” (Miles et al, 1994). There are three types of multivariate analyses relevant to SRT evaluations:

• Hierarchical or Sequential Approach (Type I). Determines the order of the effects observed, when one factor is thought to precede another (based on existing theory or body of research). This approach should be used in non-experimental designs where the factors can be ordered in some sequential manner. • Classic Experimental or Least Squares Approach (Type II). Used with an unequal number of participants, where the cells (conditions) having a larger number of cases are thought to be more important. • Regression, Unweighted Means or Unique Approach (Type III). Used with an equal number of cases in each cell, or where the design is balanced in that the cell frequencies are proportional in terms of their marginal distributions, and where all cells are thought to be equally important.

C.3.6.2.4 Statistical Power

Statistical power refers to the ability of the statistical methods to detect significant differences between the experimental conditions (e.g., decision support tools versus no decision support tools); the greater the statistical power, the more reliable the statistics are able to detect differences between the experimental conditions. For example, a within-subjects design (in which participants receive all the experimental treatments) is more powerful than between- subjects design (in which one group of participants receives treatment A, another separate

Page 171 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01 group of participants receives treatment B, etc.). Furthermore, larger sample sizes (i.e., a greater number of participants) also yield higher statistical power than small sample sizes.

Page 172 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

C.4 CONCLUSIONS AND RECOMMENDATIONS

The SRT HFE experimentation plan for the HAI TAC is meant to guide, and be integrated with, the development and procurement of future autonomous system capabilities under the DRDC Human System Performance project. Along with the operational and system requirements process, the SRT HFE experimentation plan is an essential part in determining the perceptual, cognitive, and behavioural issues that are critical to SRT operations. In addition, the SRT HFE experimentation plan maps directly to the evaluation component of the CD&E process.

The solid theoretical and experimental bases of the research on which this test plan is grounded, combined with the rigorous and well-defined evaluation plan from which to generate a wealth of data, will thus support the procurement, implementation, and training related to the adoption of SRT technologies with the CAF.

Finally, the evaluation planning process described in the previous sections raise several issues for the development of the evaluation methodology that will need to be considered during the development of HFE evaluation plans to address specific SRT concepts. These issues are discussed below:

1. Identify Suitable Benchmarks for Comparison. In order to assess the impact of a specific experimental manipulation on SRT performance, a suitable benchmark must be identified for comparison (i.e., a ‘control’ condition). The difficulty faced, however, is that the future SRT concepts identified by stakeholders represent a completely new capability within the CAF, and as such there is no existing training or system that can be used as a comparison. Critically, the objective of the first evaluation will be to establish baseline performance for current operations similar to those outlined by the SRT use cases, against which SRT performance using autonomous systems can be compared. In addition, the determination of performance ‘benchmarks’ for the SRT concepts would mitigate the requirement to conduct ‘side-by-side’ comparison of ‘old’ versus ‘new’ systems. For example, quality and timeliness benchmarks (e.g., no more than x identification errors per y number of contacts, or the generation of targeting information within z seconds of initial contact with a hostile force) could be determined from the CAF’s current operational experience. These performance benchmarks are mostly constructed in terms of observable behaviours, which are suitable for measurement using many of the techniques described earlier in this annex. 2. Speed-Accuracy Trade-Off. One of the objectives of future evaluations is likely to be the assessment of the impact of SRT concepts on the timeliness (i.e., speed) and quality (i.e., accuracy) of contact detection, identification, and engagement. Inevitably, the impact of these enhancements might be manifest in only one of these two measures of performance. For example, the image processing enhancements available might increase the time an operator takes to identify a contact; however, the accuracy of the identification will be significantly improved. In this case, time has been traded-off against accuracy (for a similar study see Hou et al (2009)). A multi-method approach using speed

Page 173 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

and accuracy measures (as well as workload, SA, etc.) allows the experimenter to examine the behavioural consequences of these enhancements. Alternatively, we might also seek to control the speed-accuracy trade-off using the instructions that are given to participants undertaking the evaluation. For example, participants can be instructed to be ‘as quick as you can’ (controlling for speed to observe differences in accuracy) or ‘as accurate as you can’ (controlling for accuracy to observe differences in speed). For example, tests of typing competence normally set a threshold level of accuracy over which applicants can then be differentiated in terms of their speed. A multi-method measurement approach also allows the objective assessment of design trade-offs. For example, the provision of automated functionality (e.g., automatic target cueing) might afford a reduction in operator workload, but at the expense of reduced operator SA and trust in the event of automation failure. 3. Optimising Experimental Validity. Validity is the ability of a test to measure what it was designed to measure, and the degree to which the results of an experimental method lead to clear-cut conclusions (i.e., internal validity) and how far those can be generalized (i.e., external validity). It is therefore imperative that the adopted experimental method, as well as the measures applied to the results, have both internal and external validity to support the overall CD&E process: a. Internal Validity. Internal validity is a form of experimental validity. An experiment is said to possess internal validity if it properly demonstrates a causal relation between two variables. An experiment can demonstrate a causal relation by satisfying three criteria: i. The ‘cause’ precedes the ‘effect’ in time (i.e., temporal precedence); ii. The ‘cause’ and the ‘effect’ are related (i.e., covariation); and iii. There are no plausible alternative explanations for the observed covariation (i.e., non-spuriousness). In the SRT evaluations a number of variables (i.e., independent variables) will be manipulated to see what effect they have on other variables (i.e., dependent variables, or MOPs). For example, the level of autonomy might be manipulated to see what effect it has on the soldier’s workload (e.g., the level of support provided by a decision aid). If the evaluation allows the researcher to conclude that different levels of autonomy caused a change in the soldier’s workload (by satisfying the above criteria), then the study possesses internal validity. In other words, an experiment possesses internal validity if the observed changes in the dependent variable were caused by the manipulation of the independent variable. b. External Validity. An evaluation is said to possess external validity if the evaluation’s results hold across different evaluation settings, procedures, and participants. If a study possesses external validity, its results will generalize from the tested sample to the larger population. External validity should not be confused with ecological validity. While external validity deals with the ability of

Page 174 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

evaluation results to generalize to the “real-world” population, ecological validity is a criterion ascertaining whether the experimental procedures resemble real- world conditions. For an evaluation to possess ecological validity, the methods, materials, and setting of the experiment must approximate the real-life situation that is under study. Unlike internal and external validity, ecological validity is not necessary to the overall validity of an evaluation. One difficulty associated with the development of an appropriate evaluation methodology is the compromise that inevitably has to be made between internal and external validity. For example, on one hand efforts to increase internal validity (e.g., by removing confounding variables and increasing the validity of the measures used) often have negative consequences on the applicability of the results to the target domain (i.e., external validity). On the other hand, evaluations which have high external validity (e.g., realistic field studies) often have confounding variables, which, in turn, reduce the internal validity of the study. Finding a suitable compromise between the internal and external validity of a single evaluation is not easy. The best evaluation strategy, therefore, would be to use a combination of a laboratory-based approach (i.e., high internal validity achieved with some risks of compromising external validity) and a field-based approach (i.e., high external validity achieved with some risks of compromising internal validity). 4. Operational Realism. The evaluations must be sufficiently realistic and representative of the future roles of the autonomous systems described in the SRT use case storyboards (i.e., ecological validity). The levels of workload normally experienced during CA dismounted infantry operations must be reproduced during the evaluations whenever possible. For example, the evaluation scenarios should be designed in such a way so that realistic levels of operator workload (i.e., long periods of relatively low workload, interspersed with short periods of high activity), and fatigue (i.e., increasing levels of fatigue during sustained operations) can be reproduced. The requirement to reproduce realistic levels of operator workload and fatigue necessitates relatively long evaluation sessions. For example, evaluation sessions should last at least two to three hours to reflect the duration of a mission segment that a dismounted infantry section or platoon typically undertakes. 5. Statistical Power. The experimental design adopted for the future SRT concept evaluations should be as powerful as possible in order to increase the likelihood of finding reliable significant differences between the experimental conditions. Due to the limited number of participants, it is recommended that a within-subjects design is utilised for the evaluations (i.e., participants receive all experimental treatments) whenever possible. In this case, participants will take part in all experimental conditions (one after the other, and the order will be counter-balanced) during all scenarios.

Page 175 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

C.5 REFERENCES

Albert, W., & Tullis, T. (2013). Measuring the user experience: collecting, analyzing, and presenting usability metrics. Newnes.

Banbury, S., & Howes, A. (2001). Development of generic methodologies for the evaluation of collaborative technologies. DERA Technical Report (Contract CU005-2927).

Banbury, S., Dudfield, H. & Hoermann, H.J. (2004). Development of novel measures to assess the effectiveness of commercial airline pilot Situation Awareness training. Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting, New Orleans, USA.

Bartik, J., et al. (2019). Autonomy Strategic Challenge Allied IMPACT Final Report. TTCP Technical Report, TR-ASC-01-2020.

Behymer, K. J., Mersch, E. M., Ruff, H. A., Calhoun, G. L., & Spriggs, S. E. (2015). Unmanned vehicle plan comparison visualizations for effective human-autonomy teaming (pp. 1022-1029). Procedia Manufacturing, 3.

Benta, M. I. (2005). Studying communication networks with AGNA 2.1. Cognition, Brain, Behavior, 9(3), 567-574.

Blickensderfer, E. L., Reynolds, R., Salas, E., & Cannon-Bowers, J. A. (2010). Shared expectations and implicit coordination in tennis doubles teams. Journal of Applied Sport Psychology, 22(4), 486-499.

Brandt, S. L., Lachter, J., Russell, R., & Shively, R. J. (2017). A human-autonomy teaming approach for a flight-following task. In International Conference on Applied Human Factors and Ergonomics (pp. 12-22). Springer, Cham.

Brannick, M. T., Roach, R. M., & Salas, E. (1993). Understanding team performance: A multimethod study. Human Performance, 6(4), 287-308.

Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), 4-7.

Brown, M., Herdman, C., Banbury, S., & Baker, K. (2005). INCOMMANDS TDP: Demonstration and Experimentation Plan. Defence R&D Canada – Valcartier Contract Report. SP 2005-999.

Bryman, A., & Cramer, D. (2012). Quantitative data analysis with IBM SPSS 17, 18 & 19: A guide for social scientists. Routledge.

Calhoun, G. L., Ruff, H. A., Behymer, K. J., & Mersch, E. M. (2017). Operator-autonomy teaming interfaces to support multi-unmanned vehicle missions. In Advances in Human Factors in Robots and Unmanned Systems (pp. 113-126). Springer, Cham.

Page 176 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Calhoun, G. L., Ruff, H. A., Behymer, K. J., & Frost, E. M. (2018). Human-autonomy teaming interface design considerations for multi-unmanned vehicle control (pp. 321-352). Theoretical Issues in Ergonomics Science, 19(3).

Cannon-Bowers, J. A., Salas, E., Blickensderfer, E., & Bowers, C. A. (1998). The impact of cross- training and workload on team functioning: A replication and extension of initial findings. Human Factors, 40(1), 92-101.

Casali, J. G., & Wierwille, W. W. (1983). A comparison of rating scale, secondary-task, physiological, and primary-task workload estimation techniques in a simulated flight task emphasizing communications load. Human Factors, 25(6), 623-641.

Chadwick, R. A. (2008). Considerations for use of aerial views in remote unmanned ground vehicle operations. Proceedings of the 52nd Human Factors and Ergonomics Society Annual Meeting (pp. 252–256). Santa Monica, CA: Human Factors and Ergonomics Society.

Chen, J. C., Haas, E. C., & Barnes, M. J. (2007). Human performance issues and user interface design for teleoperated robots (pp. 1231–1245). IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37.

Clare, A. S., Maere, P. C. P., & Cummings, M. L. (2012). Assessing operator strategies for real-time replanning of multiple unmanned vehicles. Intelligent Decision Technologies, 6, 221–231. doi:10.3233/IDT-2012-0138.

Collier, J., & Hou, M. (2019). DRDC Human Robotic Teaming Interests. Internal DRDC technical note, April 2019.

Cooper, G. & Harper, R. (1969). The use of pilot rating in the evaluation of aircraft handling qualities. Technical Report TN D-5153, NASA, April 1969.

Crabtree, M. S., Bateman, R. P., & Acton, W. H. (1984, October). Benefits of using objective and subjective workload measures. In Proceedings of the Human Factors Society Annual Meeting (Vol. 28, No. 11, pp. 950-953). Los Angeles, CA: SAGE Publications.

Cummings, M. L., & Mitchell, P. J. (2005). Managing multiple UAVs through a timeline display. In American Institute of Aeronautics and Astronautics (pp. 1–13). Reston, Virginia: American Institute of Aeronautics and Astronautics.

Cummings, M.L., Bruni, S., Mercier, S., & Mitchell, P.J. (2007). Automation Architecture for Single Operator, Multiple UAV Command and Control. The International C2 Journal, 1(2), 1-24.

Cummings, M. L., & Quimby, P. W. (2018). The power of collective intelligence in a signal detection task (pp. 375-388). Theoretical issues in ergonomics science, 19(3).

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information

Page 177 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

technology. MIS quarterly, 319-340.

De Greef, T. E., & Neerincx, M. A. (2010). Adaptive automation based on an object-oriented task model : Implementation and evaluation in a realistic C2 environment. Journal of Cognitive Engineering and Decision Making, 4, 152–182.

Demir, M., McNeese, N. J., & Cooke, N. J. (2017). Team situation awareness within the context of human-autonomy teaming (pp. 3-12). Cognitive Systems Research, 46.

Demir, M., Likens, A. D., Cooke, N. J., Amazeen, P. G., & McNeese, N. J. (2018a). Team coordination and effectiveness in human-autonomy teaming (pp. 150-159). IEEE Transactions on Human-Machine Systems, 49(2).

Demir, M., Cooke, N. J., & Amazeen, P. G. (2018b). A conceptual model of team dynamical behaviors and performance in human-autonomy teaming (pp. 497-507). Cognitive Systems Research, 52.

Department of Defense (2011c). Department of Defense Standard Practice: Human engineering requirements for military systems, equipment, and facilities (MIL-STD-46855A).

Dey, I. (2003). Qualitative data analysis: A user friendly guide for social scientists. Routledge.

Donmez, B., Nehme, C., & Cummings, M. L. (2010). Modeling workload impact in multiple unmanned vehicle supervisory control. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 40, 1180–1190.

Dubé, G. (2015). Fonctionnement des équipes de réponse d'urgence: examen de la dynamique d'un modèle d'efficacité.

Dubé, G. & Lamoureux, T.M. (2017). HALIFAX Class Modernization Automation Impact on Crew (3) Experimental Plan - SEAWOLF. CAE Contract Report 6010-006-01.

Durso, F. T., Hackworth, C. A., Truitt, T. R., Crutchfield, J., Nikolic, D., & Manning, C. A. (1998). Situation awareness as a predictor of performance for en route air traffic controllers. Air Traffic Control Quarterly, 6(1), 1-20.

Durso, F. T., Dattel, A. R., Banbury, S., & Tremblay, S. (2004). SPAM: The real-time assessment of SA. A cognitive approach to situation awareness: Theory and application, 1, 137-154.

Endsley, M. R. (1993). A survey of situation awareness requirements in air-to-air combat fighters. The International Journal of Aviation Psychology, 3(2), 157-168.

Endsley, M. R. (1995a). A taxonomy of situation awareness errors. Human factors in aviation operations, 3(2), 287-292.

Endsley, M. R. (1995b). Measurement of situation awareness in dynamic systems. Human factors,

Page 178 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

37(1), 65-84.

Endsley, M. R., & Jones, W. M. (2001). A model of inter- and intrateam situation awareness: Implications for design, training, and measurement. In M. McNeese, E. Salas, & M. Endsley (Eds.), New trends in cooperative activities: Understanding system dynamics in complex environments (pp. 46–67). Santa Monica, CA: Human Factors and Ergonomics Society.

Espinosa, A., Lerch, F. J., Kraut, R. E., Salas, E., & Fiore, S. M. (2004). Explicit vs. implicit coordination mechanisms and task dependencies: one size does not fit all. Team cognition: understanding the factors that drive process and performance. American Psychological Association, Washington, DC, 107-129.

Giles, K., & Giammarco, K. (2017). Mission-based Architecture for Swarm Composability (MASC). Procedia Computer Science, 114, (pp. 57-64).

Gorman, J. C., Cooke, N. J., & Winner, J. L. (2006). Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49(12-13), 1312-1325.

Greenley, M., & Boyle, K. (2003). Future Armoured Vehicle Systems Virtual Immersive Environment/Vehicle Integration Operator Machine Interface: Lab Evaluation Plan. For General Dynamics. Document No 974229

Haring, K. S., Finomore, V., Muramato, D., Tenhundfeld, N. L., Redd, M., Wen, J., & Tidball, B. (2018). Analysis of Using Virtual Reality (VR) for Command and Control Applications of Multi-Robot Systems. In Proceedings of the 1st International Workshop on Virtual, Augmented, and Mixed Reality for HRI (VAM-HRI).

Hart, S.G., & Staveland, L.E. (1988). Development of the NASA TLX (Task Load Index): results of empirical and theoretical research. In P.A. Hancock and N. Meshkati (Eds.), Human Mental Workload (pp. 139-183). Amsterdam: Elsevier Science.

Hendrick, H. (2004). Handbook of Human Factors and Ergonomics Methods. Taylor and Francis CRC.

Hochmitz, I., & Yuviler-Gavish, N. (2011). Physical fidelity versus cognitive fidelity training in procedural skills acquisition. Human factors, 53(5), 489-501.

Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International journal of human-computer studies, 64(2), 79-102.

Hou, M., Hollands, J. G., Scipione, A., Magee, L., & Greenley, M. (2009). Comparative evaluation of display technologies for collaborative design review. PRESENCE: Teleoperators and Virtual Environments, 18(2), 125-138.

Hou, M., Zhu, H., Zhou, M., & Arrabito, G. R. (2010). Optimizing operator–agent interaction in

Page 179 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

intelligent adaptive interface design: A conceptual framework. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 41(2), 161-178.

Hou, M., Banbury, S., & Burns, C. (2014). Intelligent adaptive systems: An interaction-centered design perspective. CRC Press.

Hou, M. (in press). IMPACT: A Trust Model for Human-Agent Teaming. Proceedings of the 1st IEEE Int'l Conf on Human-Machine Systems, Rome, April 2020.

Houghton, R. J., Baber, C., McMaster, R., Stanton, N. A., Salmon, P., Stewart, R., & Walker, G. (2006). Command and control in emergency services operations: a social network analysis. Ergonomics, 49(12-13), 1204-1225.

Hsia, P., Samuel, J., Gao, J., Kung, D., Toyoshima, Y., & Chen, C. (1994). Formal approach to scenario analysis. IEEE Software, 11(2), 33-41.

Jian, J. Y., Bisantz, A. M., & Drury, C. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53-71.

Jordan, C. S., Farmer, E. W., & Belyavin, A. J. (1995). The DRA workload scales (DRAWS): A validated workload assessment technique. In International Symposium on Aviation Psychology, 8th, Columbus, OH (pp. 1013-1018).

Kelly, C. (2003). Guidelines for trust in future ATM systems. European Organisation for the Safety of Air Navigation Technical Report HRS/HSP-005-GUI-03.

Kirk, R.E. (1998). Experimental Design: Procedures for the behavioral sciences 3rd Ed. Brooks/Cole Publishing Company.

Krippendorff, K. (2013). Content Analysis: An Introduction to its Methodology, Third Edition. Sage.

Kunzle, B., Xiao, Y., Miller, A.M., & Mackenzie, C. (2010). Survey of Healthcare Teamwork Rating Tools: Reliability, Validity, Ease of Use and Diagnostic Efficacy. In E. S. Patterson & J. E. Miller (Eds.), Macrocognition Metrics and Scenarios: Design and Evaluation for Real- World Teams. (pp. 123-135). Farnham, Ashgate.

Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: a call for data analysis triangulation. School psychology quarterly, 22(4), 557.

Levulis, S. J., Kim, S. Y., & DeLucia, P. R. (2016). Effects of Touch, Voice, and Multimodal Input on Multiple-UAV Monitoring During Simulated Manned-Unmanned Teaming in a Military Helicopter. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 60, No. 1, pp. 132-132). Sage CA: Los Angeles, CA: SAGE Publications.

Lewis, K. (2003). Measuring transactive memory systems in the field: Scale development and validation. Journal of applied psychology, 88(4), 587.

Page 180 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Loerger, T. R. (2003). Literature Review: Modeling Teamwork as Part of Human Behavior Representation. Technical report TSSTI-TR-10-03, Training System Science and Technology Initiative, Texas A&M University.

Lund, A. M. (2001). Measuring usability with the USE questionnaire. Usability interface, 8(2), 3-6.

MacMillan, J., Entin, E. E., & Serfaty, D. (2004). Communication Overhead: The Hidden Cost of Team Cognition. In E. Salas, & S. M. Fiore (Eds.), Team Cognition: Understanding the Factors that Drive Process and Performance (pp. 61D82). Washington, DC: APA.

Madsen, M. & Gregor, S. (2000). Measuring human-computer trust. In Proceedings of Eleventh Australasian Conference on Information Systems (pp. 6-8).

Matthews, M.L., Webb, R.D.G., & Keeble, A.R. (2002). Assessing the Impact of Multi-Sensor Data Fusion on Command and Control Operations in the Halifax Class Frigate: Recommendations for Measures of Performance and Detailed Test Plan. DRDC Toronto CR-2002-018.

McDermott, P. L., Walker, K. E., Dominguez, C. O., Nelson, A., & Kasdaglis, N. (2017). Quenching the thirst for human-machine teaming guidance: helping military systems acquisition leverage cognitive engineering research (pp. 236-240). In 13th International Conference on Naturalistic Decision Making, Bath, UK.

McGuinness, B., & Foy, L. (2000). A subjective measure of SA: The Crew Awareness Rating Scale (CARS). In Proceedings of the first human performance, situation awareness, and automation conference, Savannah, Georgia (Vol. 16, pp. 286-291).

McIntyre, R., & Salas, E. (1995). Team performance in complex environments: What we have learned so far. In R. Guzzo & E. Salas (Eds.), Team effectiveness and decision-making in organizations (pp.9-45). San Francisco: Jossey-Bass.

McNeese, N. J., Demir, M., Cooke, N. J., & Myers, C. (2018). Teaming with a synthetic teammate: Insights into human-autonomy teaming (pp. 262-273). Human factors, 60(2).

Miles, M. B., Huberman, A. M., Huberman, M. A., & Huberman, M. (1994). Qualitative data analysis: An expanded sourcebook. Los Angeles, CA: SAGE Publications.

Miller, C., Funk, H., Wu, P., Goldman, R., Meisner, J., & Chapman, M. (2005). The Playbook™ approach to adaptive automation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 49, no.1, (pp. 15-19). New York: SAGE Publications.

Miller, C., & Parasuraman, R. (2007). Designing for flexible interaction between humans and automation: delegation interfaces for supervisory control. Human Factors, 49, 57–75.

Parasuraman, R., Cosenzo, K. a., & De Visser, E. (2009). Adaptive automation for human

Page 181 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

supervision of multiple uninhabited vehicles: Effects on change detection, situation awareness, and mental workload. Military Psychology, 21, 270–297.

Pickup, L., Wilson, J. R., Norris, B. J., Mitchell, L., & Morrisroe, G. (2005). The Integrated Workload Scale (IWS): a new self-report tool to assess railway signaller workload. Applied Ergonomics, 36(6), 681-693.

Prewett, M. S., Johnson, R. C., Saboe, K. N., Elliott, L. R., & Coovert, M. D. (2010). Managing workload in human–robot interaction: A review of empirical studies (pp. 840–856). Computers in Human Behavior, 26.

Roldán, J. J., Peña-Tapia, E., Martín-Barrio, A., Olivares-Méndez, M. A., Del Cerro, J., & Barrientos, A. (2017). Multi-robot interfaces and operator situational awareness: Study of the impact of immersion and prediction (pp. 1720(. Sensors, 17(8).

Rosenfeld, A., Agmon, N., Maksimov, O., & Kraus, S. (2017). Intelligent agent supporting human– multi-robot team collaboration. Artificial Intelligence, 252, 211-231.

Salas, E., & Fiore, S. M. (2004). Team Cognition: Understanding the Factors that Drive Process and Performance. Washington, DC: American Psychological Association.

Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a "big five" in teamwork? Small Group Research, 36 (5), 555-599.

Salmon, P., Stanton, N., Walker, G., & Green, D. (2006). Situation awareness measurement: A review of applicability for C4i environments. Applied ergonomics, 37(2), 225-238.

Sandelowski, M. (2001). Real qualitative researchers don’t count: The use of numbers in qualitative research. Research in Nursing & Health, 24, 230 –240.

Saner, L. D., Bolstad, C. A., Gonzalez, C., & Cuevas, H. M. (2009). Measuring and predicting shared situation awareness in teams. Journal of Cognitive Engineering and Decision Making, 3(3), 280-308.

Sarcevic, A., Marsic, I., Lesk, M. E., & Burd, R. S. (2008). Transactive memory in trauma resuscitation. In Proceedings of the 2008 ACM conference on Computer supported cooperative work (pp. 215-224). ACM.

Sexton, J. B., & Helmreich, R. L. (2000). Analyzing cockpit communications: the links between language, performance, error, and workload. Human Performance in Extreme Environments, 5(1), 63-68.

Page 182 Operational Requirements for Soldier-Robot Teaming Calian Report DND-1144.1.1-01

Surendran, A. (n.d.). Quantitative data: Definition, types, analysis and examples. Retrieved September 9, 2019 from https://www.questionpro.com/blog/quantitative-data.

Spradley, J. P. (1979). The ethnographic interview. Fort Worth, TX: Holt, Rinehart and Winston.

Stanton, N. A. (Ed.). (1996). Human factors in nuclear safety. CRC Press.

Stanton, N. A., Salmon, P. M., Rafferty, L. A., Walker, G. H., Baber, C., & Jenkins, D. P. (2017). Human factors methods: a practical guide for engineering and design. CRC Press.

Stout, R. J., Cannon-Bowers, J. A., Salas, E., & Milanovich, D. M. (1999). Planning, shared mental models, and coordinated performance: An empirical link is established. Human Factors, 41, 61-71.

Taylor, R. M. (1990). Situational Awareness Rating Technique (SART): The development of a tool for aircrew systems design. AGARD. Situational Awareness in Aerospace, 23-53.

Thorne, S. (2000). Data analysis in qualitative research. Evidence-based nursing, 3(3), 68-70.

Tremblay, S., Vachon, F., Lafond, D., & Hodgetts, H. (2010). Does teaming up make you less vulnerable to task interruption? Proceedings of the 54th annual meeting of the Human Factor and Ergonomics Society. Santa Monica, CA: Human Factor Society.

Vidulich, M. A. (1989). The use of judgment matrices in subjective workload assessment: The subjective workload dominance (SWORD) technique. In Proceedings of the Human Factors Society Annual Meeting (Vol. 33, No. 20, pp. 1406-1410). Sage CA: Los Angeles, CA.

Wegner, D. M. (1987). Transactive memory: A contemporary analysis of the group mind. In Theories of group behavior (pp. 185-208). Springer, New York, NY.

Wickens, C. D., Dixon, S. R., & Ambinder, M. S. (2006). Automation reliability in unmanned air vehicles. In N. J. Cooke, H. L. Pringle, H. K. Pedersen, and O. Connor (Eds.), Human Factors of Remotely Operated Vehicles (Advances in Human Performance and Cognitive Engineering Research, Volume 7 (Vol. 7, pp. 209–222). Emerald Group Publishing Limited.

Zeitlin, L. R., & Finkelman, J. M. (1975). Research note: Subsidiary task techniques of digit generation and digit recall as indirect measures of operator loading. Human Factors, 17(2), 218-220.

Page 183

DOCUMENT CONTROL DATA *Security markings for the title, authors, abstract and keywords must be entered when the document is sensitive 1. ORIGINATOR (Name and address of the organization preparing the document. 2 a . SECURITY MARKING A DRDC Centre sponsoring a contractor's report, or tasking agency, is entered (Overall security marking of the document including in Section 8.) special supplemental markings if applicable.)

CAN UNCLASSIFIED Calian Ltd. 770 Palladium Drive Ottawa, Canada 2b. CONTROLLED GOODS K2V 1C8 NON-CONTROLLED GOODS

DMC A

3. TITLE (The document title and sub-title as indicated on the title page.)

Operational Requirements for Soldier-Robot Teaming

4. AUTHORS (Last name, followed by initials – ranks, titles, etc., not to be used)

Banbury, S.; Heffner, K.; Liu, H.; Pelletier, S.

5. DATE OF PUBLICATION 6a. NO. OF PAGES 6b. NO. OF REFS (Month and year of publication of document.) (Total pages, including (Total references cited.) Annexes, excluding DCD, covering and verso pages.) August 2020 191 145

7. DOCUMENT CATEGORY (e.g., Scientific Report, Contract Report, Scientific Letter.)

Contract Report

8. SPONSORING CENTRE (The name and address of the department project office or laboratory sponsoring the research and development.)

DRDC – Toronto Research Centre Defence Research and Development Canada 1133 Sheppard Avenue West Toronto, Ontario M3K 2C9 Canada

9a. PROJECT OR GRANT NO. (If appropriate, the applicable 9b. CONTRACT NO. (If appropriate, the applicable number under research and development project or grant number under which which the document was written.) the document was written. Please specify whether project or grant.) W7719-185397/001/TOR 02ac - Human Systems Performance (HSP)

10a. DRDC PUBLICATION NUMBER (The official document number 10b. OTHER DOCUMENT NO(s). (Any other numbers which may be by which the document is identified by the originating assigned this document either by the originator or by the sponsor.) activity. This number must be unique to this document.)

DRDC-RDDC-2020-C172 DND-1144.1.1-01

11a. FUTURE DISTRIBUTION WITHIN CANADA (Approval for further dissemination of the document. Security classification must also be considered.)

Public release

11b. FUTURE DISTRIBUTION OUTSIDE CANADA (Approval for further dissemination of the document. Security classification must also be considered.)

12. KEYWORDS, DESCRIPTORS or IDENTIFIERS (Use semi-colon as a delimiter.)

Human-Autonomy Teaming; Teamwork/Collaboration; Trust (Human/Teams); Human/Soldier Performance; Dismounted Soldier System; Network-Enabled Soldier; Soldier Burden (Physical & Cognitive)

13. ABSTRACT/RÉSUMÉ (When available in the document, the French version of the abstract must be included here.)

The concept of Soldier-Robot Teaming (SRT) has been identified by the Canadian Armed Forces (CAF) as a way of improving mission effectiveness by leveraging enabling technologies such as robotics and autonomous systems. To support this endeavor, a series of Human Factors Engineering (HFE) research activities have been conducted at the Defence Research and Development Canada (DRDC) Toronto Research Center (TRC). In this report, key stakeholders within the Department of National Defence (DND) and the DRDC are first engaged to identify future CAF SRT concepts for further developments. Based on the results of the stakeholder analysis and a brief review of state of-the-art SRT concepts from allied partners, industry, and academic networks, five SRT storyboard use cases are then developed to capture the intended concept of operations, SRT interactions, operational contexts, as well as expected mission performance. With these use cases, a preliminary mission function task analysis is also performed in the report to identify key SRT capability featuresthat would need to be developed to support the envisioned improvements to future operational effectiveness of the CAF. As such, SRT operational requirements and related HFE experimentation plans are therefore developed and recommended in the report to guide future DRDC/DND SRT research and experimentation activities.

Les Forces armées canadiennes (FAC) ont porté leur attention sur le concept d’équipe soldat-robot (ESR), qui pourrait rendre les missions plus efficaces par l’exploitation de technologies habilitantes comme les robots et les systèmes autonomes. Pour appuyer cela, le Centre de recherches de Toronto (CRT), de Recherche et développement pour la défense Canada (RDDC) a mené une série de recherches en ingénierie des facteurs humains (IFH). Pour le rapport, des parties intéressées du ministère de la Défense nationale (MDN) et de RDDC ont cerné plusieurs concepts d’ESR pour les FAC en vue de recherches futures. L’analyse des parties intéressées et une étude succincte des concepts de pointe en ESR de nos partenaires alliés, de l’industrie et des réseaux universitaires a permis de dégager cinq scénarios et cas d’utilisation décrits dans le rapport : concept d’opération prévu, interactions des ESR, contextes opérationnels et rendement prévu des missions. En fonction de ces cas d’utilisation, une analyse de la mission, des fonctions et des tâches est faite afin de relever les « fonctions » d’ESR importantes qu’il faudrait créer pour prendre en charge les améliorations prévues à l’efficacité opérationnelle des FAC. Le rapport se conclure par un exposé des besoins opérationnels en ESR et des plans d’expérimentation en IFH, qui sont recommandés afin d’orienter les recherches et les expérimentations à venir du MDN et de RDDC sur les ESR.