DEPARTMENT OF NATIONAL DEFENCE CANADA
OPERATIONAL RESEARCH DIVISION
DIRECTORATE OF OPERATIONAL RESEARCH (JOINT)
DOR(JOINT) RESEARCH NOTE RN 2003/06
PACIFIC LITTORAL ISR EXPERIMENT 1 DESIGN
BY
G. H. Van Bavel
SEPTEMBER 2003
OTTAWA, CANADA National Défense Defence nationale
OPERATIONAL RESEARCH DIVISION
CATEGORIES OF PUBLICATION
ORD Reports are the most authoritative and most carefully considered publications of the DGOR scientific community. They normally embody the results of major research activities or are significant works of lasting value or provide a comprehensive view on major defence research initiatives. ORD Reports are approved personally by DGOR, and are subject to peer review.
ORD Project Reports record the analysis and results of studies conducted for specific sponsors. This Category is the main vehicle to report completed research to the sponsors and may also describe a significant milestone in ongoing work. They are approved by DGOR and are subject to peer review. They are released initially to sponsors and may, with sponsor approval, be released to other agencies having an interest in the material.
Directorate Research Notes are issued by directorates. They are intended to outline, develop or document proposals, ideas, analysis or models which do not warrant more formal publication. They may record development work done in support of sponsored projects which could be applied elsewhere in the future. As such they help serve as the corporate scientific memory of the directorates.
ORD Journal Reprints provide readily available copies of articles published with DGOR approval, by OR researchers in learned journals, open technical publications, proceedings, etc.
ORD Contractor Reports document research done under contract of DGOR agencies by industrial concerns, universities, consultants, other government departments or agencies, etc. The scientific content is the responsibility of the originator but has been reviewed by the scientific authority for the contract and approved for release by DGOR.
DEPARTMENT OF NATIONAL DEFENCE
CANADA
OPERATIONAL RESEARCH DIVISION
DIRECTORATE OPERATIONAL RESEARCH (JOINT)
DOR(JOINT) RESEARCH NOTE RN 2003/06
PACIFIC LITTORAL ISR EXPERIMENT 1 DESIGN
by
G. H. Van Bavel
Recommended by: Approved by: P. Comeau R.G. Dickinson EXORT/TL DOR(Joint)
Directorate Research Notes are written to document material which does not warrant or require more formal publication. The contents do not necessarily reflect the views of ORD or the Canadian Department of National Defence.
OTTAWA, ONTARIO SEPTEMBER 2003
ABSTRACT
The Canadian Forces Experimentation Centre is tasked with Concept Development and Experimentation (CD&E), which is contributing to the transformation of the forces. Strategy 2020 indicates the goal of the transformation, and the Canadian Joint Task List specifies the capability areas that require CD&E. Information and Intelligence (I2) is the capability area of interest to the Pacific Littoral ISR Experiment 1 (PLIX-1). This experiment attempts to enhance I2 capabilities pertaining to a selected operations area in a littoral environment. An uninhabited aerial vehicle shall survey the littoral operations area with its sensors. Level-1 analysts shall use the radar and camera data to generate a tactical- level operating picture and post selected information and images to a website. Higher-level analysts shall fuse the tactical-level operating picture with the Recognised Maritime Picture (RMP) to generate an Experimental RMP (XRMP). The measure of effectiveness of this I2 experiment shall be based upon the completeness of the XRMP relative to the ordinary RMP. The completeness will be measured by a count of the number of positively identified contacts against the null hypothesis that the XRMP will not be improved. The measure of effectiveness is quantifiable and the hypothesis is falsifiable. The I2 operation shall also be assessed with a measure of force effectiveness. During PLIX-1, two command teams will generate plans for four assigned missions; one command team shall use the ordinary RMP, the other the XRMP. The indicator for the measure of force effectiveness shall be savings in CF resources and improved timeliness to execute the mission-plan under the XRMP relative to the ordinary RMP.
i
RESUME
Le Centre d'expérimentation des Forces canadiennes assure l’élaboration de concepts et d'expérimentation (ECE) et contribue ainsi à la transformation des Forces. La Stratégie 2020 définit le but de cette transformation tandis que la Liste canadienne des tâches interarmées précise quels sont les domaines de capacité qui requièrent l’ECE. L’information et le renseignement (I2) constituent le domaine de capacité visé par l’Essai no 1 de RSR sur le littoral du Pacifique (ERLP-1). Cet essai aura pour but d’améliorer des capacités de type I2 propres à une zone d’opérations déterminée située en milieu littoral. Un véhicule aérien piloté à distance devra faire le lever de cette zone au moyen de ses détecteurs. Grâce aux données recueillies par le radar et la caméra, des analystes de niveau 1 produiront une image de fonctionnement de niveau tactique ainsi que des renseignements et des images post-sélectionnés pour un site Web. Des analystes chevronnés devront ensuite fusionner cette image de fonctionnement de niveau tactique avec l’image maritime reconnue (IMR) afin de produire une IMR expérimentale (IMRE). La mesure de l’efficacité de cet essai de type I2 dépendra du niveau de complétude de l’IMRE comparativement à l’IMR habituelle. On mesurera ce niveau de complétude en comptant les points de contact repérés avec certitude en fonction de l'hypothèse nulle selon laquelle l’IMRE ne sera pas améliorée. La mesure de l’efficacité est quantifiable et l’hypothèse, falsifiable. L’opération de type I2 devra également être évaluée selon l'efficacité des forces. Durant l’ERLP-1, deux équipes de commande produiront des plans pour quatre missions assignées : une équipe utilisera l’IMR ordinaire et l’autre, l’IMRE. Un indicateur pour la mesure de l'efficacité des forces devra se traduire par une économie de ressources des FC et par une opportunité accrue d’exécution du plan de mission avec l’IMRE plutôt qu’avec l’IMR.
ii
TABLE OF CONTENTS
PAGE ABSTRACT...... i RESUME ...... ii TABLE OF CONTENTS...... iii LIST OF FIGURES ...... v LIST OF TABLES...... vi LIST OF ABBREVIATIONS/GLOSSARY...... vii ACKNOWLEDGEMENTS...... xii I - INTRODUCTION...... 1 II - EXPERIMENT DESIGN...... 2 CONCEPT OF OPERATIONS...... 3 CAPABILITY MEASUREMENT DEFINITIONS ...... 10 VARIABLES...... 11 EXPERIMENT OPERATIONS...... 15 ASSUMPTIONS...... 21 RISKS ...... 22 SCHEDULE...... 23 ADMINISTRATIVE SUPPORT ...... 24 III - ANALYSIS PROPOSAL...... 24 CAPABILITY MEASUREMENT FORMULAS...... 24 FURTHER EVALUATION ...... 26 IV - CONCLUSIONS...... 28 V - RECOMMENDATIONS...... 29 VI - REFERNCES ...... 31 ANNEX A . HYPOTHESIS GENERATION...... A-1 ANNEX B . DATA COLLECTION PLAN ...... B-1 CRITICAL DATA SET...... B-2 THE ORDINARY RECOGNISED MARITIME PICTURE ...... B-2 THE EXPERIMENTAL RECOGNISED MARITIME PICTURE...... B-3 THE ORDINARY COMMAND TEAM MISSION PLAN...... B-5 THE EXPERIMENTAL COMMAND TEAM MISSION PLAN ...... B-6 PATROL START AND FINISH TIMES...... B-7 OPERATIONS AREA COORDINATES ...... B-9 SUPPORTING DATA SET ...... B-10
iii
ISR MISSION PLAN ...... B-10 TRAFFIC HISTORY IN OPERATIONS AREA...... B-12 PHYSICAL ENVIRONMENT STATE ...... B-13 AERIAL VEHICLE AND SENSORS ...... B-14 LEVEL-1 ANALYSIS IT SYSTEMS...... B-17 NETWORK CAPACITY AND RELIABILITY...... B-20 X-CELL IT SYSTEMS ...... B-22 VESSEL OF INTEREST...... B-25 AERIAL VEHICLE AND SENSOR CREW ...... B-28 LEVEL-1 ANALYSTS ...... B-30 X-CELL ANALYSTS ...... B-33 COMMAND TEAMS AND MISSION PLANNING...... B-36 VESSEL OF INTEREST CREW ...... B-38 THE AERIAL VEHICLE RECOGNISED MARITIME PICTURE...... B-40 REFERENCE VESSELS...... B-42 ANNEX C . A SIMPLE MODEL...... C-1 BASIC INTERACTIONS...... C-1 INTRODUCTION OF ERRORS...... C-4 CORRECTION OF ERRORS ...... C-5 THE RATE FUNCTIONS ...... C-7 AN EXAMPLE SOLUTION...... C-7
iv
LIST OF FIGURES
Figure 1: Area CYR 106...... 6 Figure 2: PLIX-1 OPAREA...... 7 Figure 3: Estimated Sensor Ranges at Approximate Altitude of 4,000 m...... 7 Figure 4: PLIX-1 Design Schematic...... 15
Figure C-1: ARMP Example Solution...... C-9 Figure C-2: XRMP Example Solution...... C-9 Figure C-3: XRMP Example Solution if the PLIX-1 Hypothesis Holds...... C-10
v
LIST OF TABLES
Table 1: PLIX-1 OPAREA Vertex Coordinates...... 6 Table 2: PLIX-1 Data Generation and Collection ...... 18 Table 3: The PLIX-1 Schedule...... 23
Table C-1: Example Parameters ...... C-8 Table C-2: Example Initial Conditions...... C-8
vi
LIST OF ABBREVIATIONS/GLOSSARY
ADIZ Air Defense Identification Zone
ALIX Atlantic Littoral ISR Experiment
ARMP Aerial Vehicle Recognized Maritime Picture, which is constructed at the GCS
Athena The name of the MOSIC that is located at CFB Esquimalt, which is part of the OSCP.
CD&E Concept Development and Experimentation
CD-ROM Compact Disk – Read Only Memory
CF Canadian Forces
CFB Canadian Forces Base
CFEC Canadian Forces Experimentation Centre
contact 1. Any discrete airborne, surface or subsurface object detected by electronic, acoustic, and/or visual sensors (from CF/DND Defence Terminology Bank online) 2. An establishing of communication with someone or an observing or receiving of a significant signal from a person or object
excursion 1. A departure of PLIX-1 UAV from its assigned patrol in order to close on a contact and acquire sufficient sensor data for positive identification 2. Deviation from a direct, definite, or proper course (from the Merriam-Webster Dictionary)
FPS Force Planning Scenario
GCS Ground Control Station
vii
I2 Information and Intelligence
identification The highest level of identification in CF maritime ISR doctrine (i.e. the name of the vessel)
IISRA Integrated ISR Architecture
independent variables Quantities or qualities that may change independently of the value of other variables (from Reference [8])
intervening variables Quantities or qualities that that affect the relationship between the independent and dependent variables (from Reference [8])
intruder The target vessel of interest that poses a threat to Canadian sovereignty and/or interests.
ISR Intelligence, Surveillance, and Reconnaissance
IXT Integrated Experimentation Team
latency The time delay between the initial detection of a target and its identification
MALE Medium Altitude Long Endurance, a class of UAV
MARPAC Maritime Forces Pacific
MC Mission Commander measures of effectiveness Tools used to measure results achieved in the overall mission and execution of assigned tasks. Measures of effectiveness are a prerequisite to the performance of combat assessment. Also called MOEs. (from the DoD Dictionary of Military Terms, Jan. 9, 2003)
mole 1. The intruder’s clandestine shore support 2. A spy (as a double agent) who establishes a cover long before beginning espionage (from Merriam-Webster Dictionary)
MOSIC Maritime Operational Surveillance and Information Centre
MPMC Maritime Patrol Mission Commander
NDCC National Defence Command Centre
viii
NOTAM Notice to Airmen objective function The function to be maximised or minimised in a linear programming problem (from The Concise Oxford Dictionary of Mathematics)
O-Cell The Ordinary Cell of analysts who construct the ORMP (i.e. the regular OSCP personnel performing their regular function)
OCOM The Ordinary Command team, who use the ORMP to plan their assigned mission.
OPAREA Operations Area, which is designated PLIX-1 OPAREA for PLIX-1
O-Plan The Ordinary Plan, which is generated by the OCOM syndicate based upon the ORMP
ORMP Ordinary Recognized Maritime Picture
OSCP Operational Support Centre Pacific
PDT Pacific Daylight Time, the local time zone for PLIX-1
PLIX Pacific Littoral ISR Experiment (campaign)
PLIX-1 OPAREA The MALE UAV littoral operations area, near Vancouver Island PLIX-1 UAV The uninhabited aerial vehicle planned for PLIX-1, which is to be a MALE UAV equipped with multiple-sensors: synthetic aperture radar, and electro-optical/infrared camera
PLIX-N Pacific Littoral ISR Experiment N, where N = 1, 2, 3, 4, 5, and 6
pouncer A responding CF unit that executes the O-Plan or X-Plan to intercept the intruder
rescuer A responding CF, RCMP, or Coast Guard unit that executes the O-Plan or X-Plan to rescue the vessel-in- distress
RMP Recognized Maritime Picture
ix
SAR 1. Search and Rescue 2. Synthetic Aperture Radar
seeker the CF unit that searches for the intruder
surveil to subject to surveillance (Merriam-Webster Dictionary)
TBD To Be Determined
TPPU Task Post Process Use is an intelligence cycle or procedure under concept development for network-centric operations.
UAV Uninhabited Aerial Vehicle (also Unmanned Aerial Vehicle)
UT Universal Time vessel-in-distress The object of a Search and Rescue mission
VOI Vessel of Interest represents either a vessel-in-distress or intruder
XADMIN Experiment Administrator
X-Cell The Experimental Cell of analysts who construct the XRMP
XCOM The Experimental Command team, who use the XRMP to plan their assigned mission
XDE Experiment Design and Evaluation (person or function)
XMAN Experiment Manager
XOBS(N) Experiment Observer Lambda-N, where N = One, Two, Three, Four, Five. XOBS(N) is dedicated to observing the physical process that the state vector N describes.
XOBS(N) Experiment Observer Psi-N, where N = One, Two, Three, Four, Five. XOBS(N) is dedicated to observing the human process that the state vector N describes.
XOPS Experiment Operations (person or function)
X-Plan The Experimental Plan, which is generated by the XCOM syndicate based upon the XRMP
x
XRMP Experimental Recognized Maritime Picture
XT Experimentation Team
xi
ACKNOWLEDGEMENTS
P. J. Comeau (CFEC, OR Team Lead), for reviewing this experiment design and implementing of the data collection plan.
L. Col. S. J. Newton (CFEC), for reviewing this experiment design.
K. R. Wheaton (CFEC), for supporting and guiding the author’s first experiment design.
xii
PACIFIC LITTORAL ISR EXPERIMENT 1 DESIGN
I - INTRODUCTION
1. The Canadian Forces Experimentation Centre (CFEC) is engaged in Concept Development and Experimentation described in Reference [1] (Plan Pegasus). Strategy 2020 (Reference [2]) provides the capstone, or over-arching concept, which is the transformation of the Canadian Forces into a responsive organization with modern capabilities. An integrating concept under development at CFEC is Information and Intelligence. At CFEC, an integrating concept corresponds to one of the eight capability areas specified in the Canadian Joint Task List. Intelligence, Surveillance and Reconnaissance (ISR) capabilities are the focus of the Pacific Littoral ISR Experiment 1 (PLIX-1), which in accordance with Reference [1] shall involve the functional concept of Integrated ISR Architecture and the employment concept of Uninhabited Aerial Vehicles (UAV), also known as Unmanned Aerial Vehicles.
2. PLIX-1 is a live experiment in which a UAV patrols the littoral waters near Tofino, British Columbia. Specifically, PLIX-1 involves a Medium Altitude Long Endurance (MALE) UAV equipped with multiple sensors, which are a maritime patrol radar and an Electro-Optical/Infrared (EO/IR) camera. The data and information from the sensors shall be fused with the ordinary Recognised Maritime Picture (RMP), which is normally available on a continuous basis from the Operations Support Centre Pacific (OSCP).
3. At Canadian Forces Base (CFB) Esquimalt, the fusion of data and information shall yield the Experimental RMP (XRMP). Two command teams will generate plans for assigned missions; one command team shall use the ordinary RMP, the other the XRMP. The measure of force effectiveness shall be savings in notional resources and improved timeliness to execute the mission-plan under the XRMP relative to the ordinary RMP. The XRMP is the observable that yields the test of experiment hypothesis, which can be encapsulated as: all contacts in the UAV patrol area will be positively identified in the XRMP.
4. The foundation of this experiment design rests upon testing the hypothesis. In other words, testing the hypothesis requires operations and resources (personnel and materiel), which are estimated. Lists of assumptions and risks are provided, and these are related to critical requirements and operations. The PLIX-1 design concentrates on experimental data, particularly its collection and analysis. This design is a component of a larger experimentation plan, which includes the documentation necessary for the execution of a - 2 - military experiment (i.e. tasking orders, administrative directives, etc.).
5. The purpose of a CFEC experimentation plan, as outlined in Reference [3], is to provide the experimental conditions that rigorously and sufficiently tests hypotheses that are described in “Detailed Concept Development” documents. However, these documents do not exist. Since CFEC is presently at initial operating capability, no resources are available for the production of “Detailed Concept Development” documents. Therefore, hypothesis generation was added to the requirements listed in Reference [3] and is discussed further in Annex A.
6. A Concept of Operations for PLIX-1 was attempted in Reference [4], but the effort to correct for the lack of “Detailed Concept Development” documents drew the focus away from the specification of the activities to be studied at PLIX-1. However, Reference [4] facilitated the identification of critical problems, some of which are discussed in Reference [5], along with recommended solutions. This design implements some of the solutions.
7. Reference [6] discussed the problem of “Ground Truth”, and reached the conclusion that absolute ground truth cannot be attained. This is consistent with the fact that the Canadian Forces (CF) has an Information and Intelligence (I2) deficiency (see Ref [1]). In other words, if the CF were capable of providing absolute ground truth, then there would be no I2 deficiency. The unattainable nature of absolute ground truth necessitates the PLIX-1 solution: relative measurements. This practical solution involves observing the performance of CF operations with and without a UAV. As a result, the PLIX-1 measure of effectiveness quantifies the complementary nature of the UAV as an ISR asset and the measure is relative to current I2 capabilities.
8. Reference [7] identified the UAV as an aerospace ISR asset capable of patrolling all areas of interest to the Maritime Forces. However, UAV is not among the ISR assets that Ref. [7] recommended for meeting Canada’s maritime ISR requirements to 2010 and beyond. The recommendations emphasized the need to “enable the effective integration of operational information”. The scope of the recommendations in Ref. [7] was consistent with the fact that the CF has an I2 deficiency, whereas a UAV was a sensor-bearer that presented an I2 integration problem rather than solved it. Similarly, the PLIX-1 design emphasizes that the critical operational task is to integrate the UAV data into the ISR architecture and measure its effect upon dependent operations.
II - EXPERIMENT DESIGN
9. In this section the activities subject to experimentation are described. First, the
- 3 - objective is identified, and the related hypothesis is stated. The measures of performance and effectiveness are indicated, followed by a list of the data to be collected. An outline is provided for the activities needed to meet the objective. Variables, assumptions, and risks are identified. Finally, indications of the required human and material resources are given.
10. In the following sections, the UAV is denoted PLIX-1 UAV (Aerial Vehicle for PLIX-1). PLIX-1 UAV data and information shall be fused with the Ordinary Recognised Maritime Picture (ORMP), which is continuously available from the Operations Support Centre Pacific (OSCP), also known as Athena. The fusion of PLIX-1 UAV data and information into the ORMP shall yield the Experimental RMP (XRMP). The purpose of this experiment is summarized as follows:
a. PLIX-1 Objective. Assess the utility of PLIX-1 UAV to support the construction of the XRMP within a specific littoral operations area; and
b. PLIX-1 Hypothesis. If PLIX-1 UAV patrols a designated operations area of littoral waters, then all surface contacts are continuously tracked, and positively identified in the XRMP of the operations area before the end of the patrol.
CONCEPT OF OPERATIONS
11. The military activity to be studied is an ISR operation with the following concept of operations:
a. Objective. Classify, identify, and track all contacts in the operations area (OPAREA) designated PLIX-1 OPAREA; and
b. Operations. The objective is achieved by performing the following operations:
(1) Monitor PLIX-1 OPAREA with current ISR assets;
(2) Fuse data and information from current ISR assets into the Ordinary RMP (ORMP) for PLIX-1 OPAREA;
(3) Patrol PLIX-1 OPAREA with PLIX-1 UAV, which follows a pre- planned mission, unless:
(a) An excursion is required to identify a contact with shorter-
- 4 -
range sensors; and
(b) The course-of-events (triggered by the real-world or experi- menters) entails a new tasking.
(4) Fuse data and information from PLIX-1 UAV sensors with the ORMP to create the XRMP for PLIX-1 OPAREA;
(5) Two operational-level command teams plan the same specified mission (see paragraph 11.c below) for one or more units from the CF or Other Government Departments (OGD) as follows:
(a) The Ordinary Command (OCOM) team bases their plan upon the ORMP; and
(b) The Experimental Command (XCOM) team bases their plan upon the XRMP. c. Mission Options.
(1) Littoral ISR;
(2) A search and rescue operation (e.g. overdue ship);
(3) Locate, intercept, and track a vessel of interest; and
(4) Locate, intercept, track, and engage a vessel of interest that has shore support.
- 5 -
12. The Pacific coast of Canada and the military restricted airspace CYR 106 are shown in Figure 1. The coordinates given in TABLE 1 define the PLIX-1 OPAREA and its geographical extent is shown in Figure 2. PLIX-1 OPAREA is composed of a subsection of CYR 106 and an extension that shall be affected through a Notice To Airmen (NOTAM). 13. Figure 3 provides an estimate of the unobstructed ranges of the Radar and Electro- Optical/Infrared (EO/IR) sensors. It is emphasized that the estimated ranges in Figure 3 neglect obstructions such as mountains, clouds, and other environmental impediments to the propagation of electromagnetic radiation.
14. Three generic scenarios provide the context for the activity in PLIX-1. This multiple-scenario approach is recommended by Reference [8]. The scenarios are related to Force Planning Scenarios (FPS), which are detailed in Reference [9]. The three FPS of interest are as follows:
a. FPS 1: Search and Rescue in Canada;
b. FPS 4: Surveillance/Control of Canadian Territory and Approaches; and
c. FPS 8: National Sovereignty/Interests Enforcement.
- 6 -
Figure 1: Area CYR 106
TABLE 1: PLIX-1 OPAREA VERTEX COORDINATES
Vertex Latitude Longitude North West
1 49 44.0’ 127 30.0’ 2 49 20.0’ 126 30.0’ 3 48 45.0’ 125 45.0’ 4 48 40.0’ 125 20.0’
5 48 30.0’ 125 20.0’ 6 48 25.0’ 126 30.0’ 7 48 22.0’ 127 30.0’
Note: Vertices are given clockwise starting from the northern-most vertex.
- 7 -
Figure 2: PLIX-1 OPAREA
Figure 3: Estimated Sensor Ranges at Approximate Altitude of 4,000 m
- 8 -
15. The main players in PLIX-1 are:
a. Seeker. This role is composed of the following:
(1) PLIX-1 UAV;
(2) Level 1 sensor-data analysts (constructing ARMP); and
(3) Higher-level Intelligence analysts (constructing ORMP and XRMP).
b. Rescuer. The notional responding unit or units (e.g. CF ships, Canadian Coast Guard Helicopters) that perform the rescue operations;
c. Vessel-in-Distress. A specific vessel that is overdue and is the object of the Search and Rescue mission;
d. Intruder. A specific vessel that is a threat to Canadian sovereignty and/or interests, which ISR assets must detect and identify and notional CF units must intercept, track and/or disable;
e. Pouncer. The notional responding CF unit or units (on paper only) that intercepts, tracks, and/or disables the intruder (e.g. Frigates, Maritime Patrol Aircraft, Maritime Helicopters, etc.); and
f. Mole. A shore-based source of counter-intelligence to help the intruder evade detection and/or identification by the seeker.
16. In PLIX-1, the specific activity is played out in four vignettes:
a. Seeker On Patrol. PLIX-1 UAV executes its pre-planned patrol with excursions, thus PLIX-1 UAV monitors normal littoral traffic, and attempts to meet its objective (paragraph 11.a);
b. Seeker and Rescuer. PLIX-1 UAV executes its pre-planned patrol with excursions, attempts to meet its objective, and supports the planning of a Search and Rescue mission by locating the vessel-in-distress;
- 9 -
c. Intruder Versus Seeker and Pouncer. The intruder is embedded within normal traffic while the PLIX-1 UAV executes its pre-planned patrol with excursions, attempts to meet its objective, and PLIX-1 UAV sensor data shall be used to plan a notional mission to covertly intercept and track the intruder; and
d. Intruder and Mole Versus Seeker and Pouncer. The intruder uses shore- based counter-intelligence to evade detection and identification, while the PLIX-1 UAV executes its pre-planned patrol with excursions, attempts to meet its objective, and supports a notional mission to intercept, track, and disable the intruder.
17. The required CF support for the operations under investigation in PLIX-1 is as follows:
a. Tofino Airport.
(1) A Maritime Patrol Mission Commander (MPMC) and staff plan the PLIX-1 UAV mission, and issue updates as necessary during the mission;
(2) A Navy Liaison Officer communicate with personnel in Esquimalt; and
(3) PLIX-1 UAV crewmembers analyse sensor data and construct the local ARMP.
b. CFB Esquimalt.
(1) Two analysts are required to construct the XRMP (i.e. fuse the PLIX-1 UAV data and information into the ORMP); and
(2) Two command teams to plan the assigned mission:
(a) OCOM. The Ordinary Command team uses the ORMP to plan the assigned mission; and
(b) XCOM. The Experimental Command team uses the XRMP to plan the assigned mission.
- 10 -
CAPABILITY MEASUREMENT DEFINITIONS
18. The I2 capability provided by the use of PLIX-1 UAV as an ISR asset shall be assessed with the following measurements:
a. Measures of Performance.
(1) Accuracy of PLIX-1 UAV execution of the patrol;
(2) Sensor availability: the fraction of time that PLIX-1 UAV sensors are active and functioning properly;
(3) Fraction of PLIX-1 UAV excursions from the patrol that yield a positively identified target;
(4) Fraction of PLIX-1 UAV detections that are identified as false alarms;
(5) Observed PLIX-1 UAV sensor performance relative to manufacturer’s specifications and military requirements;
(6) Timeliness (or latency) of contact classification in the ARMP, ORMP, and XRMP;
(7) Timeliness (or latency) of contact identification in the ARMP, ORMP, and XRMP; and
(8) Effort (person-hours) to plan the assigned mission using the ORMP and XRMP.
b. Measure of Effectiveness. The number of identified targets (ISR coverage) in the XRMP relative to the ORMP; and
c. Measure of Force Effectiveness. The written estimate of the time (person- hours and unit-hours) to execute the XCOM plan relative to the OCOM plan.
- 11 -
VARIABLES
19. The variables are identified as follows (see the Glossary, page vi, for definitions):
a. Independent Variables - Controllable.
(1) t is the time of all events of interest, and the relevant time zone is always explicitly specified (e.g. Universal Time (UT) or Pacific Daylight Time (PDT)) ;
(2) ts the time when PLIX-1 UAV starts its patrol, which is the first instant (after launch) when PLIX-1 UAV sensors surveil PLIX-1 OPAREA;
(3) tf the time when PLIX-1 UAV finishes its patrol, which is the last instant (prior to landing) when PLIX-1 UAV sensors survey PLIX-1 OPAREA;
(4) R is the planned position of PLIX-1 UAV (pre-planned patrol and intentional excursions);
(5) S is the state of PLIX-1 UAV sensors as specified by the manufacturer and mission commander (i.e. mode of operation, on/off, range, resolution, etc);
b. Independent Variables - Uncontrollable.
(1) N is the total number of targets (regardless of class) in PLIX-1 OPAREA;
(2) W is the environmental state (e.g. weather, sea state, etc).
c. Intervening Variables.
(1) 1 is the reliability state of PLIX-1 UAV and its sensors;
(2) 2 is the capacity and reliability of the devices at Tofino that are used to
(a) Construct the ARMP; and
- 12 -
(b) Post data and information for fusion into the XRMP;
(3) 3 is the capacity and reliability of the network connecting the PLIX-1 UAV website and X-Cell analysts;
(4) 4 is the capacity and reliability of the devices at CFB Esquimalt used to construct the XRMP;
(5) 5 is the capability state of the VOI in its role as vessel-in-distress and intruder (i.e. speed, radar and EO/IR signatures, organic sensor counter-detection ranges, etc.);
(6) 1 is the capability state of the PLIX-1 UAV pilot and sensor operator;
(7) 2 is the collective capability state of the Level 1 Analysts at Tofino to construct the ARMP and post data for fusion into the XRMP;
(8) 3 is the collective capability state of the X-Cell Analysts to construct the XRMP;
(9) 4 is the state of enforcement mission planners regarding:
(a) Confidence in the XRMP or ORMP; and
(b) Mission planning capabilities.
(10) 5 is the VOI crew state vector, which indicates the collective capability of the VOI crew to perform the following:
(a) Act as the unfortunate crew of a vessel-in-distress; and
(b) Act as the radical/outlaw crew of an intruder. d. Dependent Variables. All dependent variables are functions of time t:
(1) C is the observed sensor capability state (e.g. range, resolution, functioning/inoperative, etc.);
(2) r is the observed position of PLIX-1 UAV;
- 13 -
(3) v is the observed velocity of PLIX-1 UAV;
(4) is the attitude of PLIX-1 UAV;
(5) NE is the number of excursions attempted for target identification;
(6) Gi is the excursion success indicator for i = 1, 2, …, NE:
(a) Gi = 1 if the excursion identifies a target; and
(b) Gi = 0 if the excursion fails to identify a target.
(k) (7) q is the observed state of reference vessel k = 1, 2, …, NRV, where
NRV is the number of reference vessels. For each observation, the time-of-observation and following components of the state vector q(k) are recorded:
(k ) (a) qLat the latitude of reference vessel k;
(k ) (b) qLon the longitude of reference vessel k;
(k ) (c) qHdg the heading of reference vessel k; and
(k ) (d) qSpd the speed of reference vessel k.
(8) HA is the number of contacts (including false alarms) in the ARMP;
(9) HO is the number of contacts in the ORMP;
(10) HX is the number of contacts in the XRMP;
(11) KA is the number of classified contacts (including errors) in the ARMP;
(12) KO is the number of classified contacts in the ORMP;
(13) KX is the number of classified contacts in the XRMP;
(14) LA is the contact-identification latency (the length of time from initial contact to identification) in the ARMP;
- 14 -
(15) LO is the contact-identification latency in the ORMP;
(16) LX is the contact-identification latency in the XRMP;
(17) IA is the number of identified contacts in the ARMP;
(18) IO is the number of identified contacts in the ORMP;
(19) IX is the number of identified contacts in the XRMP;
(20) JA is the number of false contacts (no surface vessel present) in the
ARMP since t = ts (the mission start time);
(21) JO is the number of false contacts in the ORMP since t = ts;
(22) JX is the number of false contacts in the XRMP since t = ts;
(23) EO is the effort (person-hours) that OCOM required to generate the O-Plan using the ORMP;
(24) EX is the effort (person-hours) that XCOM required to generate the X-Plan using the XRMP;
(25) TO is the time (e.g. rescuer-hours) to execute the O-Plan; and
(26) TX is the time (e.g. pouncer-hours) to execute the X-Plan.
- 15 -
EXPERIMENT OPERATIONS
20. The PLIX-1 design schematic is shown in Figure 4. Note that in the absence of an experiment, the boxes labelled ARMP, ORMP, XRMP, O-Plan, and X-Plan signify military outputs. The boxes labelled IA, IO, IX, TO, and TX represent the related experiment data, which are quantitative measurements of the military outputs.
PLIX-1 I2 Application UAV Existing ISR OCOM O-Plan T Assets O
XCOM X-Plan Level 1 TX Analysis & Post Data O-Cell I ARMP Analysis ORMP O
X-Cell Analysis XRMP IX IA I2 Source I2 Fusion
Figure 4: PLIX-1 Design Schematic
21. Some infrastructure and personnel require attention prior to the commencement of experiment operations and afterward:
a. Airfield. Tofino is an unmanned airfield. Some infrastructure (e.g. air control radar, supporting equipment, local area network, etc.) required for air operations shall have to be established;
b. Information Technology. The network and software to enable the flow of UAV sensor data and information from Tofino to OSCP Athena;
- 16 -
c. Ground Control Station. The UAV contractor must assemble and integrate the GCS with the aerial vehicle, sensors and ground exploitation system in order to perform system tests;
d. PLIX-1 UAV. All components must be transported to and from the airfield:
(1) The aerial vehicle needs assembly, initial testing, and flight- functional;
(2) The sensors require calibration and tests, and may require installation and removal from the aerial vehicle.
e. Radio Communications. Radio hardware and procedures must be set up and tested prior to the execution of the experiment;
f. Temporary Accommodation.
(1) Weather-proof shelters, including a hanger for the aerial vehicle;
(2) Personnel (including VIP guests for one day) need housing; and
(3) Sufficient number of backup electric generators.
g. CFB Esquimalt.
(1) A temporary work area for the Ordinary Command (OCOM) team;
(2) Adequate IT equipment and support for a temporary work-area in which the Experimental Cell (X-Cell) of analysts construct the XRMP; and
(3) A temporary work area for the Experimental Command (XCOM) team.
22. The Human Resources (HR) required for experiment operations are as follows (training, if any, is indicated):
a. Airfield.
(1) Personnel to set up, test, and maintain the equipment;
- 17 -
(2) Qualified aerospace-controllers; and
(3) Security. b. Information Technology. Personnel to construct and maintain the network and software that supports the posting of PLIX-1 UAV data and information to a designated website, which is accessible to X-Cell analysts at OSCP Athena (as required to construct the XRMP). c. Ground Control Station.
(1) Mission Commander and Navy Liaison Officer;
(2) Contracted UAV pilot and sensor operator,
(3) Level-1 image analysts to prepare PLIX-1 UAV contact data and information for transmission to X-Cell analysts;
(4) The level-1 image analysts may require training in the analysis of data PLIX-1 UAV sensor payloads; and
(5) Three observers to collect data from the mission commander, the UAV crew (including sensor payload operators) and the level-1 analysts. d. PLIX-1 UAV. Contractors to maintain PLIX-1 UAV and its sensors; e. Radio Communications. Personnel to maintain and operate radio equipment; f. Accommodations.
(1) CFEC Contractor to act as Tofino site manager;
(2) Local administrative staff; and
(3) CF maintenance personnel. g. RMP Construction.
(1) The O-Cell analysts do not represent an extra requirement, because OSCP analysts construct the ORMP on a continuous basis;
- 18 -
(2) The X-Cell requires two qualified analysts to construct the XRMP;
(3) The X-Cell analysts may require training in the fusion of the PLIX-1 UAV data and information into the ORMP in order to construct the XRMP; and
(4) One observer to collect data from the X-Cell.
h. Mission Planning.
(1) Three qualified CF members for the Ordinary Command (OCOM) team;
(2) Three qualified CF members for the Experimental Command (XCOM) team;
(3) The XCOM team may require training in the interpretation of the ARMP and XRMP; and
(4) Two observers to collect data from each command team.
23. Experiment operations are subject to the following critical constraints:
a. Experiment operations must not interfere with normal (real-world) operations at CFB Esquimalt, and OSCP (Athena) in particular;
b. PLIX-1 UAV must launch and land at the Tofino airfield when the local air traffic is light;
c. PLIX-1 UAV must adhere to airspace arrangements and restrictions while in flight;
d. A line of sight link must be maintained between PLIX-1 UAV and its GCS; and
e. Observers and data collectors must have adequate security clearances.
24. TABLE 2 summarises how the data is generated and collected. All geographic coordinates are to be given in degrees, minutes, and decimal minutes (see TABLE 1 for examples). Details of the data collection are given in Annex B.
TABLE 2: PLIX-1 DATA GENERATION AND COLLECTION
- 19 -
Variable(s) Generation Collection
Automated or manual records must state Clock in specified the time of events of interest and t time zone (e.g. UT explicitly specify the relevant time zone or PDT). (e.g. UT or PDT). The PLIX-1 UAV crew records patrol The PLIX-1 UAV start time: the first instant (after launch) t pilot and sensor s when AVX- sensors surveil PLIX-1 operator OPAREA The PLIX-1 UAV crew records the The PLIX-1 UAV patrol finish time: the last instant (prior t pilot and sensor f to landing) that PLIX-1 UAV sensors operator surveil PLIX-1 OPAREA Mission Commander records the Mission planning R intended patrol and the procedure for any and retasking excursions (e.g. target identification). PLIX-1 UAV The PLIX-1 UAV GCS records all sensor S sensors parameters. OSCP Athena electronically records the history of the ORMP and XRMP at ORMP, XRMP, and N regular intervals (at least10 times per historical records hour). The XT obtains historical records prior to experiment. Environmental data obtained by the XT W Weather services from appropriate weather service providers The UAV and its The performance is summarised by an 1 sensors independent observer at the GCS The capacity and The performance is summarised by an reliability of the independent observer who monitors the 2 devices used by construction of the ARMP and the analysts at Tofino posting of data for fusion into the XRMP Capacity and reliability of the The software application records network connecting measurement directly to XT electronic 3 the PLIX-1 UAV archives at/near the GCS GCS and OSCP Athena Performance mea- An independent observer or device surements of the records the measurements in a standard- 4 devices that support format at XRMP sites the XRMP
- 20 -
Variable(s) Generation Collection
The crew records their vessel’s Perfomance performance exceptions in a written log 5 assessment of the and (if time permits) the VOI signatures VOI in radar and EO/IR are acquired by the UAV sensors and electronically recorded An independent observer or device The UAV pilot and records observations at the PLIX-1 UAV 1 sensor operator GCS The Level 1 An independent observer records Analysts construct- observations and administers surveys at 2 ing the ARMP and the ARMP site posting data The OSCP personnel An independent observer records 3 constructing the observations and administers surveys at XRMP the XRMP site The mission An independent observer records 4 planners in the observations and administers a survey at XCOM and OCOM the mission-planning sites The crew self-reports observations of 5 The VOI crew their performance in a written log that is submitted to CFEC Recorded in a standard format at the PLIX-1 UAV GCS (this data shall In-flight calibration include special radar and EO/IR C and tests of the observations of the intruder where aspect sensors capability angle, range, and elevation shall be varied) Electronic records of the ORMP saved in HO, IO, JO, KO, The O-Cell and a standard format such that a time- LO ORMP resolution of 6 minutes or less is supported HX, IX, JX, KX, The X-Cell and Same as the ORMP collection LX XRMP specifications given above The GCS records the PLIX-1 UAV The UAV guidance r, v, position, velocity and attitude in a system standard format After each excursion, the level 1 analysts NE, Gi The level 1 analysts record whether the target was identified or not. Every six minutes or less, each reference The reference vessel will automatically record the time, q(k) vessels and the vessel’s geographic position, heading and speed.
- 21 -
Variable(s) Generation Collection
OCOM records their effort (person- hours) to generate the O-Plan, estimates T , E OCOM O O the time to execute the O-Plan, and delivers these to the XT XCOM records their effort (person- hours) to generate the X-Plan, estimates T , E XCOM X X the time to execute the X-Plan, and delivers these to the XT
ASSUMPTIONS
25. These are the main assumptions:
a. Flight Approval. Canadian authorities shall grant the leased UAV flight approval for PLIX-1;
b. UAV Availability. An affordable UAV contract is finalised in time;
c. Fair Weather. The weather does not prevent more than four UAV flights;
d. Air Defence Radar. A mobile CF air defence radar shall be acceptable to the flight approval authorities;
e. Normal Shipping Traffic. The background of shipping traffic will not be exceptionally low or high;
- 22 -
f. Target Vessel of Interest. A suitable intruder shall be available; and
g. XRMP Construction. Analysts shall be able to cope with the novel challenge of pulling PLIX-1 UAV sensor data and information from a website and then fusing that data and information into the ORMP in order to construct the XRMP.
RISKS
26. These are the risks that were identified in the conduct of this experiment:
a. Time Limitations. PLIX-1 is a live experiment that is not associated with a military exercise. Reference [10] indicates that on March 17, 2003, the decision was made to abandon other live experiments (see Ref. [5]) and proceed with PLIX-1 as the only live UAV component of the Pacific Littoral ISR Experiment; therefore, a little more than three months were allowed for detailed planning. This limitation degrades the following efforts:
(1) Coherent and comprehensible communication with stakeholders;
(2) Contracting;
(3) Acquisition of CF support personnel and materiel; and
(4) Sensible planning.
b. Real-World Operations.
(1) A search and rescue operation in or near PLIX-1 OPAREA may result in a request for support, in which case the experiment operations would be suspended while PLIX-1 UAV and its support personnel participate in the search and rescue; and
(2) CFB Esquimalt must continue to support ongoing operations, which take precedence over experiment operations.
c. Isolated Site. The Tofino airport is non-military, isolated geographically, and not materially supported on a regular basis; therefore; the XT shall be responsible for resolving its own emergencies.
- 23 -
SCHEDULE
27. TABLE 3 shows the schedule for the experiment.
TABLE 3: THE PLIX-1 SCHEDULE
Flight Time Date Activity (July 2003) (PDT)
2 No Flights Initial Deployment 3 No Flights
4 TBD Possible Test Flight(s) Assemble and Test Experiment 5 No Flights No UAV Activity Infrastructure
6 TBD Possible Test Flight(s)
7 10:00 to 16:00 Scenario 1: Patrol PLIX-1 OPAREA, identify all contacts
8 09:00 to 17:00 Scenario 2: Support SAR mission, overdue ship
9 08:00 to 14:00 Scenario 3: Detect, identify, covertly track, and intercept intruder
10 09:00 to 17:00 Scenario 4: CF engagement of a mole-supported intruder
12:00 to 14:00 Visitor Day and Demonstration Flight 11 TBD Contingency Flight 1, if necessary
12 No Flights No UAV Activity
13 TBD Contingency Flight 2 or Disassemble Experiment Infrastructure
14 TBD Contingency Flight 3 or Disassemble Experiment Infrastructure
15 TBD Contingency Flight 4 or Disassemble Experiment Infrastructure
Disassemble Experiment Infrastructure (if all Contingency Flights were No Flights 16 required)
- 24 -
ADMINISTRATIVE SUPPORT
28. The administration of PLIX-1 requires the following support:
a. Property Manager for the Tofino airport;
b. Contracts Manager; and
c. Site Manager at Tofino for food, shelter, and other provisions.
III - ANALYSIS PROPOSAL
29. The analysis of PLIX-1 data is proposed to be a parallel effort of multiple disciplines. Each discipline shall be free to report those aspects of the experiment and results that are of greatest interest to their peers. The manner in which each discipline reports their results is left to the discretion of the authors, who should be guided primarily by the common practices of their peers and, most importantly, by the requirements and expectations of their intended readers.
30. Experiment Design and Evaluation (XDE) shall coordinate the multi-disciplinary analysis and report generation. Although data shall be shared, different analyses will focus on different subsets of the data. Therefore, XDE needs to ensure consistency and be prepared to direct the resolution of disagreements. If a problem arises that cannot be solved within the Experimentation Team, then external reviewers (i.e. subject matter experts) shall be engaged to resolve the issue.
CAPABILITY MEASUREMENT FORMULAS
31. The definitions of the capability measurements are given in paragraph 18, and variables are defined in paragraph 19. These definitions can now be used to construct formulas for the capability measurements:
a. Measures of Performance.
(1) The accuracy APat of the patrol execution is given by
1 N WP APat ri Ri NWP i1
- 25 -
where NWP is the number of way points used in the calculation, i is the way point index.
(2) Fraction of time fsens that the PLIX-1 UAV sensors are available is given by
tsens S,W,ȁ1,Ȍ1,C fsens tf ts
where tsens is the period of time that the PLIX-1 UAV sensors are
active and functioning properly, and tsens depends upon the state
vectors S, W, 1, 1, and C.
(3) The fraction of excursions fE that yield a positively identified target is given by
1 N E fE Gi NE i1
(4) In the ARMP, the fraction of PLIX-1 UAV detections that are
identified as false alarms, fFA, is given by
JA fFA NHA
where NHA is the total number of PLIX-1 UAV detections since t = ts.
(5) The observed PLIX-1 UAV sensor performance Sobs is given by
ˆ S obs G(S, ȁ1 ,C)
where Gˆ denotes a grading function that yields “pass” or “fail”
through a comparison of 1 and C (actual performance) with S (specified performance).
(6) The latency of the three operating pictures shall be determined
through the calculation of the LA, LO, and LX distributions in two ways:
- 26 -
(a) Direct construction of histograms of the detection-to- identification delay time for all successful identification events; and
(b) Use Fourier Transform methods to deconvolve the latency (i.e. response function) from the observed detection and identification time series; for example
t H (t) ds I (s) L (t s). X X X
(7) The effort EO and EX in person-hours to formulate a specified mission based upon the ORMP and XRMP, respectively (note that this quantity shall be recorded directly in the mission statement; therefore, it shall not require calculation).
b. Measure of Effectiveness. The objective of the Information and Intelligence operation is to maximize the number of identified contacts (i.e. identify all contacts) in the XRMP; hence the measure of effectiveness is the increase in the number of identifications relative to the ORMP
M1 IX IO ; and
c. Measures of Force Effectiveness. The objective of the force-level operation that uses the XRMP to plan a mission is to minimize the time-to-completion; hence measure of force effectiveness is the decrease in the time-to- completion relative to the mission plan based upon the ORMP
F1 TO TX .
FURTHER EVALUATION
32. Examine observed time series operationally significant features and trends:
a. HX(t), HO(t)
b. KX(t), KO(t)
c. IX(t), IO(t)
- 27 -
33. Fit observations to a dynamical model, which is derived in Annex C:
dH f H, K, I dt H dK f H, K, I dt K dI f H, K, I dt I where the model applies separately to the ARMP, ORMP, and XRMP. The observed time evolution of the number of detections, identifications and classifications are the solution of the above autonomous system of differential equations. The model’s constant parameters, which are defined in Annex C, can be determined by fitting (e.g. least-squares best fit) the dynamical model to the data.
34. Test null hypotheses as follows:
a. RMP Construction:
(1) HX(t) – HO(t) = 0
(2) KX(t) – KO(t) = 0
(3) IX(t) – IO(t) = 0
(4) LX(t) – LO(t) = 0
(5) M1(t) = 0
b. Rescuer and pouncer missions:
(1) EX(t) – EO(t) = 0
(2) F1(t) = 0
- 28 -
IV - CONCLUSIONS
35. The Pacific Littoral ISR Experiment 1 (PLIX-1) requires the patrol of a littoral operations area by an uninhabited aerial vehicle. The decision to proceed with PLIX-1 was based upon contractors’ assurances that the aerial vehicle operations were feasible. Although the aerial vehicle and sensors are contractually required to be technically and operationally mature, the operations area is unfamiliar to any of the UAV contractors under consideration, and specialized personnel and materiel (in addition to the contractor) must be deployed to the Tofino airport, which is geographically isolated and not materially supported on a regular basis. Test flights prior to the experiment can demonstrate that PLIX-1 UAV operations in Tofino are feasible before the entire experiment operation is conducted.
36. The Information and Intelligence (I2) operations depend upon an Integrated ISR Architecture (IISRA). The transmission of the ARMP to Esquimalt, and the pulling of information and imagery by X-Cell analysts cannot occur, unless the IISRA functions properly. Although many components of the IISRA are proven, the particular configuration is not. Without the Tofino-Esquimalt connections, the experiment fails. Therefore, the tests of the IISRA to be conducted prior to deployment are critical.
37. Experiment operations may interfere with the UAV operations at Tofino airport. Data collection shall be particularly challenging. For example, the presence of a human observer in the Ground Control Station (GCS) may pose a problem, because of the confined space. At CFB Esquimalt, interference with I2 operations and command team activities is a problem, although intrusive activity at Esquimalt would not jeopardise the experiment to the same extent as interference with UAV operations at Tofino. The effort to minimise interference shall make the coordination of experiment operations in Tofino and Esquimalt a serious challenge. However, a rehearsal of the data collection in conjunction with a test flight should help alleviate this risk.
38. PLIX-1 has an abbreviated planning period. Presenting a comprehensible picture to stakeholders and supporters shall be challenging. As mentioned above, the critical operational task is to integrate the UAV data and information into an experimental ISR architecture, but this may be jeopardised not by technical issues, rather the IISRA may be compromised by time limits.
- 29 -
39. The proposed analysis is flexible enough to allow for incomplete data collection. Live experiments present unforeseen circumstances and accidents occur. That is why the critical data has been explicitly identified. Allowing for parallel multi-disciplinary analyses should alleviate the problem of how the experiment should be analysed, reported, and reviewed. The Experiment Design and Evaluation shall coordinate the overall analysis effort.
V - RECOMMENDATIONS
40. It is recommended that the following preparatory activity begins immediately (June, 2003):
a. Finalise UAV contract;
b. Finalise Tofino airbase infrastructure and support;
c. Begin Recognised Maritime Picture (RMP) familiarisation;
d. Commence Ottawa-Tofino testing of the Integrated ISR Architecture (IISRA);
e. Identify mission-critical CF personnel for the Information and Intelligence (I2) operations;
f. Model the I2 operations;
g. Identify data collectors;
h. Finalise data collection forms and surveys;
i. Generate CONOPS for MPMC; and
j. Generate master events lists for scenarios.
- 30 -
41. It is recommended that the following analytical activity begins immediately after completion of PLIX-1:
a. Coordinate the multi-disciplinary reporting effort;
b. Obtain RMP analysis tools; and
c. Update the I2 operations model.
- 31 -
VI - REFERENCES
[1] Canadian Forces Experimentation Centre (CFEC), Canadian Forces Joint Concept Development and Experimentation Plan (CF JCD&E Plan): Plan Pegasus, Department of National Defence (DND), May 2002.
[2] Deputy Minister of National Defence and the Chief of Defence Staff, Shaping the Future of the Canadian Forces: A Strategy for 2020, DND, June 1999.
[3] CFEC, Unit Operating Procedures, DND, September 2002.
[4] S. J. Newton, Pacific Littoral Experiment: Concept of Operations, DND CFEC, January 17, 2003.
[5] G. H. Van Bavel and K. R. Wheaton, Pacific Littoral Experiment Design: Review of the Draft Working Paper for the Concept of Operations, CFEC, February 14, 2003.
[6] Steve Dore and Van Fong, Operational Evaluation of the Cape Race High Frequency Surface Wave Radar Technology Demonstrator, DND Directorate of Operational Research (Maritime, Land & Air), Project Report PR 2002/10, August, 2002.
[7] James Kraft, The Canadian Navy’s Operational Intelligence Surveillance and Reconnaissance (ISR) Blueprint to 2010, Canadian Forces Chief of Maritime Staff, 2003.
[8] D. S. Alberts and R. E. Hayes, Code of Best Practice for Experimentation, United States Department of Defense, 2002.
[9] Director Defence Analysis, Descriptions - Departmental Force Planning Scenarios (FPS), DND, January 2000.
[10] Minutes of the Third UAV Joint Littoral ISR Experiment 2003 Integrated Experimentation Team Meeting, Chairman LCol S. J. Newton, Secretary R. K. Bowes, CFEC, 17 March 2003.
[11] C. S. Peirce, Pholosophical Writings of Peirce, edited by Justus Buchler, Dover Publications, New York, 1955.
[12] K. R. Popper, Objective Knowledge: An Evolutionary Approach, Revised Edition,
- 32 -
Oxford University Press, Oxford, 1991.
[13] H. Pospesel, Introduction to Logic: Propositional Logic, Prentice-Hall, Englewood Cliffs, 1974.
[14] D. Hume, An Inquiry Concerning Human Understanding, Edited by C. W. Hendel, Bobbs-Merrill Co. Inc., Indianapolis, 1955 (originally published 1748).
[15] R. E. Giffin, Superstitious rituals - Naïve inductivism in command and control doctrine: Its causes, consequences and cures, Proceedings of the Seventh International Command and Control Research and Technology Symposium (ICCRTS), Evidence Based Research Inc., Vienna, Virginia, 2002.
[16] R. K. Nagle and E. B. Saff, Fundamentals of Differential Equations, Benjamin/Cummings Pub. Co. Inc., Menlo Park CA, 1986.
ANNEX A DOR(JOINT) RN 2003/06 SEPTEMBER 2003
ANNEX A. HYPOTHESIS GENERATION
1. As mentioned in paragraph 2 on page 1 of the main report, there are no detailed concept development documents related to the PLIX objectives. Therefore, hypotheses were generated, as they were needed. In the sense that a preferred hypothesis was selected from a set of possible candidates, the procedure was similar to the “abduction” described in Reference [11]. The preferred hypothesis focused on a measurable ISR capability, and was falsifiable, which Reference [12] concludes is a necessary property.
2. Reference [8] recommends that a hypothesis is stated in the form of a conditional:
if A, then B, whenever {C1 & C2 & … & CN} (1) where A is the principal antecedent, B is the consequent, and {C1 & C2 & … & CN} are a set of N subsidiary antecedents. Reference [8] argues that unless a hypothesis is stated in the form of (1), the “research issue cannot be converted into falsifiable propositions and the experiment results will not be clear.” On the issue of falsifiable statements, this agrees with Reference [12].
3. However, Reference [8] does not offer a clear explanation as to why the hypothesis must take the form of (1). At the introductory-level, Reference [13] proves that “not A, or B” is equivalent to “if A, then B”. Perhaps the conditional provides a clearer indication of the dependency of B on A. The utility of stating the hypothesis in the form of a conditional remains to be assessed.
4. The set of subsidiary antecedents {C1 & C2 & … & CN} are discussed in the main report, following the statement of the hypothesis. Some examples of subsidiary antecedents are:
a. Assumptions to be satisfied;
b. Risks to be mitigated;
c. Data to be generated and collected; and
A-1
d. Human and materiel resources to be acquired.
5. Before proceeding, a warning regarding the possible states of a hypothesis is worth considering. Inductivism was a school of thought that held that a hypothesis could be proven true through experimentation. Reference [14] conclusively demonstrated that this aspect of inductivism is irrational. Inductivist tendencies have been identified in the Canadian Forces command and control doctrine in Reference [15]. Given these indications, it is worth emphasizing that any hypothesis is either provisionally accepted, or rejected as false. When a hypothesis is tested, it is sometimes falsified, but it is never confirmed (or validated) as truth.
A-2
ANNEX B DOR(JOINT) RN 2003/06 SEPTEMBER 2003
ANNEX B. DATA COLLECTION PLAN
1. A summary of the data collection plan is given in TABLE 2 on page 18. This Annex shall specify the data collection in greater detail. The set of data to be collected is divided into two major sets:
a. Critical Data Set. The critical data set is required for the following analytical activities:
(1) Test the hypothesis; and
(2) Calculate measures of effectiveness.
b. Supporting Data Set. The supporting data set does not directly affect the test of the hypothesis, rather the supporting data set is required for the following analytical activities:
(1) Specification of the conditions under which the hypothesis was tested; and
(2) Calculation of measures of performance.
2. The relevant time zone of any observations must be explicitly noted. Records should be in Universal Time or Local Time, which shall be Pacific Daylight Time (PDT). The importance of this directive cannot be emphasised too much, and is repeated throughout this document.
3. Geographic coordinates are to be recorded in degrees, minutes, and decimal minutes. For example, the latitude and longitude of a surface contact would be 48 30.019 North and 126 28.854 West. Any exceptions, such as software-generated data files, to this manner of recording geographic coordinates must be explicitly noted on the data record.
B-1
4. Separate paragraphs describe the collection of each data subset, thus, there is some repetition in the data collection plan. However, a specific paragraph may be extracted and given to those who shall collect or record that specific data. Repeated abbreviations are not defined within a data-collection paragraph; therefore, the List of Abbreviations and Glossary (page vii to xi) must be distributed with each data-collection paragraph.
5. A data collection rehearsal is recommended. A comprehensive rehearsal should coincide with a test flight, but not necessarily. Rather, the rehearsal of the collection of different data sets should be executed whenever an opportunity arises. This includes prior to the experiment and in situations and sites far removed from PLIX-1.
CRITICAL DATA SET
6. The critical data set is required to test the hypothesis and to calculate the measures of effectiveness, including the measure of force effectiveness.
THE ORDINARY RECOGNISED MARITIME PICTURE
7. The Ordinary Recognised Maritime Picture (ORMP) for the operations area PLIX-1 OPAREA (see Figure 2, on page 7) must be recorded for the duration of the experiment according to the following specifications:
a. Output Data. The following shall be extracted from the ORMP database with dedicated software applications:
(1) HO , IO, JO , and KO at regular time intervals (every six minutes or less) where:
(a) HO : the number of detected contacts in the ORMP;
(b) IO : the number of identified contacts in the ORMP;
(c) JO : the number of false detections in the ORMP since the start of the patrol; and
(d) KO : the number of classified contacts in the ORMP.
(2) LO for all contacts that were detected and identified during the experiment (with a time-resolution of six minutes or less), where:
(a) LO : the contact-identification latency in the ORMP.
B-2
b. Source. The ORMP;
c. Collection Mechanism. Automated (computer database);
d. Location.
(1) X-Cell analysis laboratory at CFB Esquimalt; or
(2) Alternate: NDCC in Ottawa.
e. Storage. Save three copies of the ORMP database on CD-ROM disks, one copy remains at OSCP Athena, and two copies are shipped to CFEC for analysis;
f. Human Resources. Computer support personnel capable of the following:
(1) Initiate and terminate automatic recording;
(2) Produce three copies on CD-ROM; and
(3) No training required.
g. Standards. The database must be in a standard format that is portable to common computer operating systems;
h. Quality Control. XDE to monitor the process, and apply a test to all three copies of the ORMP database before original is destroyed; and
i. Data Management. XDE to notify XMAN if the data was successfully collected or not, and where it is archived.
THE EXPERIMENTAL RECOGNISED MARITIME PICTURE
8. The Experimental Recognised Maritime Picture (XRMP) data collection is similar to that of the ORMP discussed above, but for the sake of completeness, the details of the XRMP data collection are provided. The XRMP for the operations area PLIX-1 OPAREA (see Figure 2, on page 7) must be recorded for the duration of the experiment according to the following specifications:
a. Output Data. The following shall be extracted from the XRMP database with dedicated software applications:
B-3
(1) HX , IX, JX , and KX at regular time intervals (every six minutes or less) where:
(a) HX : the number of detected contacts in the XRMP;
(b) IX : the number of identified contacts in the XRMP;
(c) JX : the number of false detections in the XRMP since the start of the patrol; and
(d) KX : the number of classified contacts in the XRMP;
(2) LX for all contacts that were detected and identified during the experiment (with a time-resolution of six minutes or less), where:
(a) LX : the contact-identification latency in the XRMP. b. Source. The XRMP; c. Collection Mechanism. Automated (computer database); d. Location.
(1) X-Cell analysis laboratory at CFB Esquimalt; or
(2) Alternate: NDCC in Ottawa. e. Storage. Save three copies of the XRMP database on CD-ROM disks, one copy remains at OSCP Athena, and two copies are shipped to CFEC for analysis; f. Human Resources. Computer support personnel capable of the following:
(1) Initiate and terminate automatic recording;
(2) Produce three copies on CD-ROM; and
(3) No training required. g. Standards. The database must be in a standard format that is portable to common computer operating systems;
B-4
h. Quality Control. XDE to monitor the process, and apply a test to all three copies of the XRMP database before original is destroyed; and
i. Data Management. XDE to notify XMAN if the data was successfully collected or not, and where it is archived.
THE ORDINARY COMMAND TEAM MISSION PLAN
9. The Ordinary Command (OCOM) team shall use the ORMP to formulate the Ordinary Plan (O-Plan) for the assigned mission. The O-Plan must be recorded by OCOM according to the following specifications:
a. Output Data. The following shall be extracted from the O-Plan documents by XDE:
(1) EO : the effort (in person-hours) that OCOM required to generate the O-Plan using the ORMP;
(2) TO : the time (in unit-hours) for the responding unit(s) to execute the O-Plan;
b. Source. OCOM;
c. Collection Mechanism. Mission planning documentation (paper and/or portable electronic document) issued by XOPS;
d. Location. CFB Esquimalt;
e. Storage. One copy to remain at CFB Esquimalt for safe keeping, and the original and at least one other copy are shipped to CFEC for analysis:
(1) If paper document, make two photocopies; and
(2) If electronic document, make two copies in a portable format.
f. Human Resources. CFB Esquimalt administrative support to provide paper and/or suitable computer to OCOM;
g. Standards.
(1) The assigned mission, planning procedures and documentation
B-5
formats shall be specified by XOPS; and
(2) If the documentation is recorded in electronic files, they must be in a machine-portable format.
h. Quality Control.
(1) XOPS to monitor OCOM activity and inspect all planning documents upon completion; and
(2) Before accepting the O-Plan, XOPS requests a verbal quote of the estimates of the required output data (see paragraph 9.a, page B-5).
i. Data Management. XOPS to notify XMAN if the data was successfully collected or not, and where the originals and copies are archived.
THE EXPERIMENTAL COMMAND TEAM MISSION PLAN
10. The Experimental Plan (X-Plan) data collection is similar to that of the O-Plan discussed above, but for the sake of completeness, the details of the X-Plan data collection are provided. The Experimental Command (XCOM) team shall use the XRMP to formulate the X-Plan for the assigned mission. The X-Plan must be recorded by XCOM according to the following specifications:
a. Output Data. The following shall be extracted from the X-Plan documents by XDE:
(1) EX : the effort (in person-hours) that XCOM required to generate the X-Plan using the XRMP;
(2) TX : the time (in unit-hours) for the responding unit(s) to execute the X-Plan;
b. Source. XCOM;
c. Collection Mechanism. Mission planning documentation (paper and/or portable electronic document) issued by XOPS;
d. Location. CFB Esquimalt;
e. Storage. One copy to remain at MARPAC for safe keeping, and the original
B-6
and at least one other copy are shipped to CFEC for analysis:
(1) If paper document, make two photocopies; and
(2) If electronic document, make two copies in a portable format.
f. Human Resources. MARPAC administrative support to provide paper and/or suitable computer to XCOM;
g. Standards.
(1) The assigned mission, planning procedures and documentation formats shall be specified by XOPS; and
(2) If the documentation is recorded in electronic files, they must be in a machine-portable format.
h. Quality Control.
(1) XOPS to monitor XCOM activity and inspect all planning documents upon completion; and
(2) Before accepting the X-Plan, XOPS requests a verbal quote of the estimates of the required output data (see paragraph 10.a, page B-6).
i. Data Management. XOPS to notify XMAN if the data was successfully collected or not, and where the originals and copies are archived; and
PATROL START AND FINISH TIMES
11. The test of the PLIX-1 hypothesis (see main report paragraph 10.b, page 3) requires the precise times of the start and finish of the PLIX-1 UAV patrol. For this reason, the patrol start and finish times are part of the critical data set:
a. Output Data. The following data, for each flight, is required to test the hypothesis:
B-7
(1) tf : the time when PLIX-1 UAV finishes its patrol, which is the last instant (prior to landing) when PLIX-1 UAV sensors surveil PLIX-1 OPAREA;
(2) ts : the time when PLIX-1 UAV starts its patrol, which is the first instant (after launch) when PLIX-1 UAV sensors surveil PLIX-1 OPAREA. b. Source. The GCS crew; c. Collection Mechanism. Paper and/or electronic message; d. Location. Tofino airport; e. Storage. The original message is retained by the GCS crew, and two copies are sent to the XMAN; f. Human Resources. If an electronic message is not transmitted directly to XMAN, then a courier shall be required to deliver the message to XMAN representative at Tofino; g. Standards.
(1) Times are to be recorded in to the nearest minute, the relevant time zone is explicitly stated (e.g. UT or PDT);
(2) Patrol start time is the first instant (after launch) when PLIX-1 UAV sensors surveil PLIX-1 OPAREA; and
(3) Patrol finish time is the last instant (prior to landing) when PLIX-1 UAV sensors surveil PLIX-1 OPAREA. h. Quality Control.
(1) Message regarding patrol start and finish times to be forwarded to XDE at OSCP Athena for review; and
(2) X-Cell analysts to forward estimate of patrol start and finish times to XDE for comparison with values from PLIX-1 UAV GCS crew.
B-8
i. Data Management. PLIX-1 UAV GCS crew to notify XMAN where original is archived, and XDE is to notify XMAN of the quality assessment.
OPERATIONS AREA COORDINATES
12. In order to test the PLIX-1 hypothesis (see main report paragraph 10.b, page 3), the precise coordinates of the Operations Area (OPAREA) are necessary. The OPAREA for PLIX-1 is designated PLIX-1 OPAREA. These critical data are set prior to the experiment; however, to provide for the contingency that the OPAREA coordinates must be changed, a collection plan follows:
a. Output Data. The definition of the designated operations area of littoral waters (i.e. PLIX-1 OPAREA in Figure 2, page 7, of the main report);
b. Source. XOPS;
c. Collection Mechanism.
(1) The coordinates of the PLIX-1 OPAREA vertices shall be set and published prior to the experiment; and
(2) In the event changes must be made, XOPS records and promulgates the new coordinates to the entire XT.
d. Location.
(1) CFEC shall retain the PLIX-1 OPAREA coordinates prior to the experiment; and
(2) If changes are required during the experiment, then XOPS records and retains the original copy of the message regarding the updated PLIX-1 OPAREA coordinates at CFB Esquimalt.
e. Storage.
(1) The PLIX-1 OPAREA coordinates that are set prior to the experiment shall be in published material to be distributed to the XT; and
B-9
(2) If changes are required during the experiment, then XOPS retains the original record of the updated PLIX-1 OPAREA coordinates at CFB Esquimalt and XMAN retains the copy promulgated to the XT.
f. Human Resources. The following should be identified as a contingency if changes are necessary (most should be available at CFB Esquimalt):
(1) Maritime cartographer;
(2) Specialist in airspace management/coordination; and
(3) A liaison with the approving authorities.
g. Standards. Identify the latitude and longitude of all PLIX-1 OPAREA vertices, which are connected by great-circle segments;
h. Quality Control. XOPS to confirm updates, if any, with the Human Resources identified in paragraph 12.f, and notifies XMAN of the quality assessment; and
i. Data Management. XOPS notifies XMAN of changes, if any, which XMAN promulgates and retains.
SUPPORTING DATA SET
13. The supporting data set is required to complete the description of the specific conditions under which the experiment was conducted and provide data to calculate measures of performance.
ISR MISSION PLAN
14. The ISR Mission Plan specifies the PLIX-1 UAV patrol of PLIX-1 OPAREA, the procedure for performing excursions, and sensor requirements. The ISR Mission Commander (MC) generates ISR mission plan.
a. Output Data. The following data shall be extracted from the ISR mission plan:
(1) R is the planned patrol of PLIX-1 UAV consisting of the following:
(a) Ordered list of patrol waypoint coordinates; and
B-10
(b) Procedures and general flight-profiles of excursions (as required for target identification).
(2) S is the state of PLIX-1 UAV sensors, which is determined as follows:
(a) MC specifies the modes of operation under different patrol phases, weather conditions, and excursions (i.e. on/off, filters, etc.); and
(b) Manufacturer specifies capabilities and limits (range, resolution, data rates, etc.). b. Source. MC; c. Collection Mechanism. Documentation written by the MC, in paper and/or portable electronic format; d. Location. Tofino; e. Storage. Original and one copy to be shipped to CFEC, and one copy to be retained by XMAN at Tofino; f. Human Resources. Tofino detachment administration and IT support to provide MC with required expendables; g. Standards.
(1) The ISR task, planning procedures and documentation formats shall be specified by XOPS; and
(2) If the documentation is recorded in electronic files, they must be in a machine-portable format. h. Quanlity Control.
(1) MC-Observer to monitor MC activity, and inspect all planning documents upon completion; and
(2) MC-Observer retains the ISR mission plan until MC briefs the Tofino detachment.
B-11
i. Data Management. MC-Observer provides XMAN one copy of the ISR mission plan, and sends original and one copy to CFEC.
TRAFFIC HISTORY IN OPERATIONS AREA
15. The surface vessel traffic in PLIX-1 OPAREA has been observed for the last century. Records of the last five years would provide a lower bound on the total number of surface- vessel contacts to be expected in PLIX-1 OPAREA. This data should be obtained prior to experiment operations.
a. Output Data. The following shall be extracted from the historical records of the surface vessel traffic in PLIX-1 OPAREA:
(1) NLB the mean value of the lower-bound on the total number of contacts in the PLIX-1 OPAREA (directly comparable to the
maximum values of HA, HO, and HX); and
(2) The limits of the 90% confidence interval about NLB .
b. Source. OSCP;
c. Collection Mechanism. A formal request by XDE to OSCP for pertinent reports and consultations with subject-matter experts;
d. Location. CFEC;
e. Storage. Normal CF/DND storage of classified documents and electronic media;
f. Human Resources. Subject matter experts and library services;
g. Standards. The mean and standard deviation of the number of contacts in the PLIX-1 OPAREA in the first fifteen days of July for the years 1998, 1999, 2000, 2001, and 2002;
h. Quality Control. The XDE shall submit results of the historical data analysis to subject matter experts for review; and
i. Data Management. Upon completion of data analysis, XDE to notify XMAN of where results are archived.
B-12
PHYSICAL ENVIRONMENT STATE
16. Observations of the weather and the sea state in PLIX-1 OPAREA are required to specify the physical conditions under which the hypothesis was tested. These data are collected as follows:.
a. Output Data. The following shall be extracted from the weather data archive:
(1) W the environmental state (e.g. weather, sea state, etc).
b. Source. These data are available from:
(1) Environment Canada;
(2) Meteorlogical and Oceanographic (METOC) services at CFB Esquimalt;
(3) PLIX-1 UAV sensors; and
(4) Surface vessels participating in PLIX-1.
c. Collection Mechanism. A formal request shall be submitted by XOPS to some source(s) listed in paragraph 16.a (page B-13) to provide hourly weather observations, which were acquired in or near PLIX-1 OPAREA;
d. Location.
(1) Tofino; and
(2) CFB Esquimalt.
e. Storage. The data shall be stored by XOPS at CFEC as follows:
(1) Electronic forms are preferred; or
(2) Paper copies are acceptable.
f. Human Resources. XOPS Administrative Assistant to receive data and create archive.
g. Standards. The data must conform to the following:
B-13
(1) Standard Meteorological data, such as temperature, humidity, precipitation, winds, etc.
(2) Standard Oceanographic surface data, such as water temperature, sea state, currents, etc.;
(3) If electronic files, then they must be portable to common operating systems.
h. Quality Control.
(1) XOPS Administrative Assistant to provide XOPS and/or XDE with a sample of the data for review; and
(2) Determine error correction procedure with the data source(s) (see paragraph 16.a, page B-13).
i. Data Management. XOPS Administrative Assistant to notify XMAN if the data was successfully collected or not, and where it is archived.
AERIAL VEHICLE AND SENSORS
17. The PLIX-1 UAV (i.e. the aerial vehicle and its sensors) shall be subject to its nominal operating environment and conditions. MC shall set the PLIX-1 UAV patrol, but excursions may be needed in order to acquire the sensor data necessary for the positive identification of some contacts. Thus, the excursions are critical to the mission, and will be specifically monitored. Complete vehicle or sensor failure has an obvious impact on experiment results. However, sub-optimal function of any PLIX-1 UAV component may be difficult to notice, but can seriously affect subsequent analysis. For this reason, a close watch on the PLIX-1 UAV performance is needed and the data shall be collected as follows:
a. Output Data. The following shall be extracted from the data:
(1) 1 is the state vector of PLIX-1 UAV. 1 indicates the capacity and reliability of PLIX-1 UAV during the performance of its mission, which includes the aerial vehicle on patrol and multiple sensors providing data from which analysts may detect and/or identify contacts.
(2) The components of 1 that shall be extracted for special analysis are:
B-14
(a) C is the observed sensor capability state (e.g. range, resolution, etc.). This shall be determined through in-flight calibration and diagnostics, and through post-experiment analysis.
(b) r is the observed position of PLIX-1 UAV;
(c) v is the observed velocity of PLIX-1 UAV;
(d) is the observed attitude of PLIX-1 UAV; and
(e) NE is the number of excursions attempted for target identification. b. Source. The primary source is Experiment Observer Lambda-One,
XOBS(1), and the secondary sources are:
(1) PLIX-1 UAV GCS (vehicle and sensor data); and
(2) Level-1 Analysts. c. Collection Mechanism.
(1) All written logs (paper or electronic) are to be retained by their authors until the end of PLIX-1, after which time all logs are submitted to XMAN;
(2) XOBS(1) notes the time and nature of any exceptions to nominal PLIX-1 UAV performance (especially during excursions) in a written log;
(3) The PLIX-1 UAV crew records (automatically, whenever possible) the following PLIX-1 UAV data every six minutes (or less):
(a) Number and time of each excursion attempted to identify a contact;
(b) Aerial Vehicle position, speed, and attitude; and
(c) Sensor states, diagnostics, and calibration results.
B-15
(4) Level-1 Analysts note the time and nature of any exceptions to the expected PLIX-1 UAV performance in a written log. d. Location. Tofino; e. Storage.
(1) XOBS(1) retains all original paper and electronic records on their original media until completion of the experiment;
(2) PLIX-1 UAV crew retains one electronic copy of their PLIX-1 UAV data record; and
(3) Level-1 analysts retain a copy of their logs. f. Human Resources. XOBS(1) and the Level-1 Analysts may need training regarding:
(1) The interpretation of PLIX-1 UAV data (especially diagnostics); and
(2) The expected performance of PLIX-1 UAV. g. Standards.
(1) All data regarding the aerial vehicle and sensor states should be in standard units (i.e. SI) and electronic data formats;
(2) All data records and notes regarding PLIX-1 UAV performance exceptions and excursions are to be recorded in to the nearest minute, the relevant time zone is explicitly stated (e.g. UT or PDT); and
(3) Electronic records must be portable to common operating systems. h. Quality Control. XOBS(1) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries PLIX-1 UAV crew regarding the following:
(a) The number of excursions attempted;
(b) If they noticed any PLIX-1 UAV performance exceptions (especially during excursions); and
B-16
(c) If they captured the PLIX-1 UAV data specified on page B- 15, paragraph 17.c(3).
(2) Attempts to read or display the PLIX-1 UAV diagnostic records; and
(3) Queries the Level-1 Analysts if they noticed any PLIX-1 UAV performance exceptions.
i. Data Management. XOBS(1) manages the data as follows:
(1) Generates and retains his reports of PLIX-1 UAV performance exceptions until the end of PLIX-1, and then submits the data to XMAN; and
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If an PLIX-1 UAV performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-14, paragraph 17.a) regarding the interpretation of an PLIX-1 UAV performance exception.
LEVEL-1 ANALYSIS IT SYSTEMS
18. The generation of information and intelligence at Tofino is affected by the capacity and reliability of the IT systems that are used to construct the ARMP and post data and information for fusion into the XRMP. Since the IT system enables the Level-1 analysis activities, it is labelled L1SYS. The required data shall be collected as follows:
a. Output Data. The following shall be extracted from the data:
(1) 2 is the state vector of L1SYS. 2 indicates the capacity and reliability of the devices at Tofino that are used to construct the ARMP and post data and information for fusion into the XRMP.
B-17
b. Source. The primary source is Experiment Observer Lambda-Two,
XOBS(2), and the secondary sources are:
(1) MC;
(2) Tofino IT support; and
(3) X-Cell Analysts. c. Collection Mechanism.
(1) All written logs (paper or electronic) are to be retained by their authors until the end of PLIX-1, after which time all logs are submitted to XMAN;
(2) XOBS(2) notes the time and nature of any exceptions to nominal L1SYS performance in a written log;
(3) MC notes time and nature of any exceptions to the construction and maintenance of the ARMP;
(4) Tofino IT support automatically records L1SYS diagnostics, and
submits a portable copy to XOBS(2); and
(5) X-Cell Analysts note the time and nature of any exceptions to the expected L1SYS performance in a written log. d. Location.
(1) Tofino: XOBS(2), MC and detactment IT support; and
(2) CFB Esquimalt: X-Cell analysts. e. Storage.
(1) XOBS(2) retains all paper and electronic records on their associated media until completion of PLIX-1;
(2) MC retains a copy of his logs;
B-18
(3) Tofino IT support retains one electronic copy of the L1SYS diagnostics record; and
(4) X-Cell Analysts retain a copy of their logs. f. Human Resources. XOBS(2), MC, and the X-Cell Analysts may need training regarding:
(1) The interpretation of L1SYS diagnostics;
(2) Expected performance of the ARMP; and
(3) The expected performance of the L1SYS. g. Standards.
(1) All notes regarding L1SYS performance exceptions are to be recorded in to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic records must be portable to common operating systems. h. Quality Control. XOBS(2) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries the MC if he noticed any ARMP performance exceptions;
(2) Queries Tofino IT support if they noticed any L1SYS performance exceptions;
(3) Attempts to read or display the L1SYS diagnostic records; and
(4) Queries the X-Cell Analysts if they noticed any performance exceptions that might be attributable to L1SYS. i. Data Management. XOBS(2) manages the data as follows:
(1) Generates and retains the reports of L1SYS performance exceptions until the end of PLIX-1, and then submits the data to XMAN; and
(2) Immediately notifies XMAN if any of the following conditions arise:
B-19
(a) Loss of data;
(b) If an L1SYS performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-17, paragraph 18.a) regarding the interpretation of a L1SYS performance exception.
NETWORK CAPACITY AND RELIABILITY
19. The flow of information and intelligence from Tofino to CFB Esquimalt is affected by the capacity and reliability of the network that connects the PLIX-1 UAV GCS and the Tofino detachment to the X-Cell and XCOM. Since the network directly enables the X-Cell and XCOM activities, it is labelled XNET. The required XNET data shall be collected as follows:
a. Output Data. The following shall be extracted from the data:
(1) 3 is the capacity and reliability of the network connecting the PLIX- 1 UAV GCS, the Level-1 analysis laboratory, the X-Cell analysis laboratory, and XCOM HQ.
b. Source. The primary source is Experiment Observer Lambda-Three,
XOBS(3), and the secondary sources are:
(1) Tofino IT support;
(2) Level-1 Analysts;
(3) X-Cell Analysts; and
(4) CFB Esquimalt IT support
c. Collection Mechanism.
(1) All written logs (paper or electronic) are to be retained by their authors until the end of PLIX-1, after which time all logs are submitted to XMAN;
(2) XOBS(3) notes the time and nature of any exceptions to nominal XNET performance in a written log;
B-20
(3) IT support at Tofino and CFB Esquimalt automatically record XNET
diagnostics, and submit a portable copy to XOBS(3); and
(4) Level-1 and X-Cell Analysts note the time and nature of any exceptions to the expected XNET performance in a written log. d. Location.
(1) Tofino: XOBS(3), Level-1 Analysts and detactment IT support; and
(2) CFB Esquimalt: X-Cell analysts and Base IT support. e. Storage.
(1) XOBS(3) retains all paper and electronic records on their associated media until completion of PLIX-1;
(2) IT support at Tofino and CFB Esquimalt retain one electronic copy of the XNET diagnostics record; and
(3) Level-1 and X-Cell Analysts retain a copy of their logs. f. Human Resources. XOBS(3), Level-1 Analysts, and the X-Cell Analysts may need training regarding:
(1) The interpretation of XNET diagnostics;
(2) The expected performance of the XNET. g. Standards.
(1) All notes regarding XNET performance exceptions are to be recorded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic records must be portable to common operating systems. h. Quality Control. XOBS(3) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
B-21
(1) Queries IT support at Tofino and CFB Esquimalt if they noticed any XNET performance exceptions;
(2) Attempts to read or display the XNET diagnostic records; and
(3) Queries the Level-1 Analysts and X-Cell Analysts if they noticed any performance exceptions.
i. Data Management. XOBS(3) manages the data as follows:
(1) Generates and retains the reports of XNET performance exceptions until the end of PLIX-1, and then submits the reports to XMAN; and
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If an XNET performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-20, paragraph 19.a) regarding the interpretation of an XNET performance exception.
X-CELL IT SYSTEMS
20. The fusion of information and intelligence at CFB Esquimalt is affected by the capacity and reliability of the IT systems that are used to construct the XRMP and pull data and information for fusion from the Level-1 website. Since the IT system enables the X-Cell analysis activities, it is labelled XCELSYS. The required data shall be collected as follows:
a. Output Data. The following shall be extracted from the data:
(1) 4 is the state vector of XCELSYS. 4 indicates the capacity and reliability of the devices in the X-Cell analysis laboratory, where XRMP is constructed by fusing Level-1 analysis results from PLIX-1 UAV into the ORMP.
b. Source. The primary source is Experiment Observer Lambda-Four,
XOBS(4), and the secondary sources are:
(1) X-Cell Analysts;
B-22
(2) CFB Esquimalt IT support; and
(3) XCOM. c. Collection Mechanism.
(1) All written logs (paper or electronic) are to be retained by their authors until the end of PLIX-1, after which time all logs are submitted to XMAN;
(2) XOBS(4) notes the time and nature of any exceptions to nominal XCELSYS performance in a written log;
(3) CFB Esquimalt IT support automatically records XCELSYS
diagnostics, and submits a portable copy to XOBS(4);
(4) X-Cell Analysts note the time and nature of any exceptions to the expected XCELSYS performance in a written log; and
(5) XCOM notes time and nature of any exceptions to the construction and maintenance of the XRMP in a written log. d. Location. CFB Esquimalt; e. Storage.
(1) XOBS(4) retains all paper and electronic records on their associated media until completion of PLIX-1;
(2) X-Cell Analysts retain a copy of their logs;
(3) CFB Esquimalt IT support retains one electronic copy of the XCELSYS diagnostics record; and
(4) XCOM retains a copy of thier logs. f. Human Resources. XOBS(4), the X-Cell Analysts, and XCOM may need training regarding:
(1) The interpretation of XCELSYS diagnostics;
(2) Expected performance of the XRMP; and
B-23
(3) The expected performance of the XCELSYS. g. Standards.
(1) All notes regarding XCELSYS performance exceptions are recorded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic records must be portable to common operating systems. h. Quality Control. XOBS(4) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries the X-Cell Analysts if they noticed any XCELSYS performance exceptions;
(2) Queries CFB Esquimalt IT support if they noticed any XCELSYS performance exceptions;
(3) Attempts to read or display the XCELSYS diagnostic records; and
(4) Queries XCOM if they noticed any XRMP performance exceptions. i. Data Management. XOBS(4) manages the data as follows:
(1) Generates and retains the reports of XCELSYS performance exceptions until the end of PLIX-1, and then submits the reports to XMAN; and
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If an XCELSYS performance exception occurs; and
B-24
(c) If there is discrepancy among the data sources (page B-22, paragraph 20.a) regarding the interpretation of an XCELSYS performance exception.
VESSEL OF INTEREST
21. A Vessel of Interest (VOI) shall represent the object of a search and rescue mission (vessel-in-distress) and represent a threat to Canada (intruder). The capability and characteristics of the VOI affects how well PLIX-1 UAV can detect, identify, and track the VOI. This is important whether the VOI is the vessel-in-distress or the intruder. Therefore, data regarding these aspects of the VOI shall be collected as follows:
a. Output Data. 5, the state vector of the VOI, shall be extracted from the
data. 5 indicates the capacity and reliability of the ship and its systems, insofar as they affect the VOI representation of a vessel-in-distress and an intruder, and the PLIX-1 UAV ability to detect, identify, and track the VOI;
b. Source. The primary source is Experiment Observer Lambda-Five,
XOBS(5), and the secondary sources are:
(1) The PLIX-1 UAV sensor operator;
(2) The Level-1 Analysts; and
(3) The VOI crew.
c. Collection Mechanism.
(1) All written logs (paper or electronic) are to be retained by their authors until the end of PLIX-1, after which time all logs are submitted to XMAN;
(2) XOBS(5) enters the following events into a written log:
(a) Whenever the VOI is positively identified, note the time and apparent VOI position;
(b) Whenever the VOI is tracked by PLIX-1 UAV, summarize the progress and outcome of the tracking operation;
B-25
(3) The PLIX-1 UAV sensor operator shall engage in the following:
(a) Note any difficulties or anomalies regarding the detection, identification and tracking of the VOI in a written log; and
(b) If time permits, record extensive multiple sensor observations of VOI at various aspect angles, elevations and ranges, and
then submit a copy of the records to XOBS(5).
(4) Level-1 Analysts to note any difficulties or anomalies regarding identification and tracking of the VOI in a written log; and
(5) The VOI crew to note any anomalous performance or malfunction of ship-systems in a written log. d. Location.
(1) Tofino: XOBS(5), PLIX-1 UAV sensor operator, and Level-1 Analysts; and
(2) Onboard the VOI: ship’s crew. e. Storage.
(1) XOBS(5) retains paper and electronic records of VOI performance exceptions on their associated media until completion of PLIX-1;
(2) PLIX-1 UAV sensor operator, Level-1 Analysts, and VOI crew retain a copy of their logs; and
(3) XMAN archives all submitted records and logs after the completion of PLIX-1. f. Human Resources. XOBS(5) may need training regarding:
(1) The interpretation of observed VOI activity;
(2) The expected performance of the VOI under various sea states and weather; and
(3) The ranges at which PLIX-1 UAV sensors are expected to:
B-26
(a) Detect the VOI; and
(b) Identify the VOI
(4) How to perform (if time permits) multiple-sensor observations of ships’ signatures (in radar and EO/IR). g. Standards.
(1) All notes regarding VOI performance exceptions are recorded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT);
(2) Observations of the VOI signatures (if time permits) must be in standard (e.g. SI) units; and
(3) Electronic records must be portable to common operating systems. h. Quality Control. XOBS(5) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries the PLIX-1 UAV sensor operator if he noticed any VOI performance exceptions;
(2) Queries the Level-1 Analysts if they noticed any VOI performance exceptions; and
(3) Queries the VOI crew if there were any VOI performance exceptions. i. Data Management. XOBS(5) manages the data as follows:
(1) Generates and retains the reports of VOI performance exceptions and (if time permits) of VOI signature data until the end of PLIX-1, and then submits the data to XMAN; and
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If a VOI performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-25,
B-27
paragraph 21.a) regarding the interpretation of a VOI performance exception.
AERIAL VEHICLE AND SENSOR CREW
22. The PLIX-1 UAV crew shall be operating in an unfamiliar location. The ability of the PLIX-1 UAV crew to cope with the assigned tasks, especially excursions from the patrol, shall be subject to self-assessment and the scrutiny of an independent observer. The data shall be collected as follows:
a. Output Data. 1 is the state vector of the PLIX-1 UAV pilot and sensor
operator. 1 indicates the capacity and reliability of PLIX-1 UAV crew to perform their mission.
b. Source. The primary source is Experiment Observer Psi-One, XOBS(1), and the secondary sources are:
(1) The PLIX-1 UAV crew; and
(2) The MC.
c. Collection Mechanism.
(1) All written logs and surveys (paper or electronic) are to be retained by their owners until the end of PLIX-1, after which time all logs and surveys are submitted to XMAN;
(2) XOBS(1) performs the following data-collection functions:
(a) Notes the time and nature of any exceptions to nominal PLIX-1 UAV crew performance (especially during excursions) in a written log; and
(b) Conducts a daily end-of-mission survey (written or verbal) of the crew.
(3) The PLIX-1 UAV crew notes any crew-performance exceptions in a written log; and
(4) MC notes any crew-performance exceptions and summarizes the crew performance of the mission in a written log.
B-28
d. Location. Tofino; e. Storage.
(1) XOBS(1) retains all original reports and surveys (paper and electronic ) until completion of the experiment;
(2) PLIX-1 UAV crew retains one copy of the PLIX-1 UAV crew log; and
(3) MC retains a copy of his log. f. Human Resources. XOBS(1) and MC may need training regarding:
(1) The interpretation of PLIX-1 UAV crew activity/behaviour; and
(2) The expected activity/behaviour of the PLIX-1 UAV crew. g. Standards.
(1) All times regarding PLIX-1 UAV crew-performance exceptions are recorded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic reports and surveys must be portable to common operating systems. h. Quality Control. XOBS(1) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries PLIX-1 UAV crew if they noted any PLIX-1 UAV crew- performance exceptions;
(2) Queries MC if he noticed any PLIX-1 UAV crew-performance exceptions, and if the overall performance of the mission was satisfactory. i. Data Management. XOBS(1) manages the data as follows:
(1) Retains the following items until the end of PLIX-1, and then submits
B-29
the data to XMAN:
(a) Reports of PLIX-1 UAV crew-performance exceptions; and
(b) End-of-mission surveys of the PLIX-1 UAV crew.
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If an PLIX-1 UAV crew-performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-28, paragraph 22.a) regarding the interpretation of an PLIX-1 UAV crew-performance exception.
LEVEL-1 ANALYSTS
23. The Level-1 Analysts at Tofino shall construct the ARMP and post information for fusion into the XRMP. The critical task that the Level-1 Analysts shall perform is the positive identification of ARMP detections. The PLIX-1 UAV may need to perform excursions in order to provide sensor data to support Level-1 identifications, thus the Level-1 Analysts shall be close observers of the excursion operations. Given these novel tasks, any difficulties (i.e performance exceptions) must be identified. The ability of the Level-1 Analysts to cope with novel tasks shall be subject to self-assessment and the scrutiny of an independent observer. The required data shall be collected as follows:
a. Output Data. The following shall be extracted from the data:
(1) 2 is the state vector of the Level-1 Analysts at Tofino. 2 indicates the collective capacity and reliability of the Level-1 Analysts to construct the ARMP and post information for fusion into the XRMP.
(2) Gi is the excursion-success indicator for i = 1, 2, …, NE, where NE is
the number of excursions attempted. Gi is a critical component of 2, because it indicates the success of the Level-1 analysis of PLIX-1 UAV excursions:
(a) Gi = 1 if the excursion identifies a contact; and
(b) Gi = 0 if the excursion fails to identify a contact.
B-30
b. Source. The primary source is Experiment Observer Psi-Two, XOBS(2), and the secondary sources are:
(1) Level-1 Analysts;
(2) MC; and
(3) X-Cell Analysts. c. Collection Mechanism.
(1) All written logs and surveys (paper or electronic) are to be retained by their owners until the end of PLIX-1, after which time all logs and surveys are submitted to XMAN;
(2) XOBS(2) performs the following data-collection functions:
(a) Notes the time and nature of any exceptions to nominal Level-1 Analysts performance in a written log; and
(b) Conducts a daily end-of-mission survey (written or verbal) of the Level-1 Analysts.
(3) The Level-1 Analysts note the following in a written log:
(a) Any performance exceptions in their analysis; and
(b) For each PLIX-1 UAV excursion, the success or failure to identify the contact.
(4) MC notes any anomalies (e.g. spurious contacts, tracking errors, etc.) in the ARMP constructed by the Level-1 Analysts; and
(5) The X-Cell Analysts note any anomalies (e.g. spurious content, identification errors, etc.) in the information and intelligence posted by the Level-1 Analysts.
B-31
d. Location.
(1) Tofino: XOBS(2), MC, and Level-1 Analysts; and
(2) CFB Esquimalt: X-Cell Analysts. e. Storage.
(1) XOBS(2) retains all original reports and surveys (paper and electronic ) until completion of the experiment; and
(2) Level-1 Analysts, MC, and X-Cell Analysts retain one copy of their logs. f. Human Resources. XOBS(2), MC, and X-Cell Analysts may need training regarding:
(1) The interpretation of Level-1 Analyst activity/behaviour; and
(2) The expected activity/behaviour of the Level-1 Analyst. g. Standards.
(1) All times regarding Level-1 Analyst performance exceptions are recorded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic reports and surveys must be portable to common operating systems. h. Quality Control. XOBS(2) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries Level-1 Analysts regarding the following:
(a) If they noted any Level-1 Analysis performance exceptions;
(b) How many PLIX-1 UAV excursions succeeded to identify the contact; and
B-32
(c) How many PLIX-1 UAV excursions failed to identify the contact.
(2) Queries MC if he noticed any anomalies in the ARMP constructed by the Level-1 Analysts, and if the tactical utility and content of the ARMP was satisfactory; and
(3) Queries X-Cell if they noticed any anomalies in the information and intelligence posted by the Level-1 Analysts, and if the overall content of the Level-1 Analysis was satisfactory.
i. Data Management. XOBS(2) manages the data as follows:
(1) Retains the following items until the end of PLIX-1, and then submits the data to XMAN:
(a) Reports of Level-1 Analyst performance exceptions; and
(b) End-of-mission surveys of the Level-1 Analysts.
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If a Level-1 Analyst performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-30, paragraph 23.a) regarding the interpretation of a Level-1 Analyst performance exception.
X-CELL ANALYSTS
24. The X-Cell Analysts at CFB Esquimalt shall construct the XRMP by pulling Level-1 Analysis information and intelligence (as required) for fusion into the ORMP. Given that these shall be novel tasks, any difficulties (i.e performance exceptions) must be identified. The ability of the X-Cell Analysts to adapt shall be subject to self-assessment and the scrutiny of an independent observer. The required data shall be collected as follows:
a. Output Data. 3 is the state vector of the X-Cell Analysts. 3 indicates the collective ability of the X-Cell analysts to construct the XRMP.
B-33
b. Source. The primary source is Experiment Observer Psi-Three, XOBS(3), and the secondary sources are:
(1) X-Cell Analysts; and
(2) XCOM. c. Collection Mechanism.
(1) All written logs and surveys (paper or electronic) are to be retained by their owners until the end of PLIX-1, after which time all logs and surveys are submitted to XMAN;
(2) XOBS(3) performs the following data-collection functions:
(a) Notes the time and nature of any exceptions to nominal X- Cell Analysts performance in a written log; and
(b) Conducts a daily end-of-mission survey (written or verbal) of the X-Cell Analysts.
(3) The X-Cell Analysts note any performance exceptions in a written log; and
(4) XCOM notes, in a written log, any anomalies (e.g. spurious contacts, tracking errors, etc.) in the XRMP, and summarises the operational utility (i.e. for mission planning) of the XRMP. d. Location. CFB Esquimalt; e. Storage.
(1) XOBS(3) retains all original reports and surveys (paper and electronic ) until completion of the experiment; and
(2) X-Cell Analysts and XCOM retain one copy of their logs. f. Human Resources. XOBS(3) and XCOM may need training regarding:
(1) The interpretation of X-Cell Analyst activity/behaviour; and
(2) The expected activity/behaviour of an X-Cell Analyst.
B-34
g. Standards.
(1) All notes regarding X-Cell Analyst performance exceptions are recorded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic reports and surveys must be portable to common operating systems. h. Quality Control. XOBS(3) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries X-Cell Analysts if they noted any X-Cell Analysis performance exceptions;
(2) Queries XCOM if they noticed any anomalies in the XRMP constructed by the X-Cell Analysts, and if the operational utility and content of the XRMP was satisfactory for their mission-planning purposes. i. Data Management. XOBS(3) manages the data as follows:
(1) Retains the following items until the end of PLIX-1, and then submits the data to XMAN:
(a) Reports of X-Cell Analyst performance exceptions; and
(b) End-of-mission surveys of the X-Cell Analysts.
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If an X-Cell Analyst performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-33, paragraph 24.a) regarding the interpretation of an X-Cell Analyst performance exception.
B-35
COMMAND TEAMS AND MISSION PLANNING
25. OCOM and XCOM shall be assigned the same mission on each day of PLIX-1. Since OCOM shall use the ORMP to generate the O-Plan and XCOM shall use the XRMP to generate the X-Plan, then a relative evaluation of the mission-planning abilities and confidence can be performed. The relative evaluation shall be accomplished through self- assessment and the scrutiny of an independent observer. The required data shall be collected as follows:
a. Output Data. 4 is the state vector of OCOM and XCOM. 4 indicates command-team confidence in the operating picture (XRMP or ORMP) and demonstrated mission planning capabilities.
b. Source. The primary source is Experiment Observer Psi-Four, XOBS(4), and the secondary sources are:
(1) OCOM; and
(2) XCOM.
c. Collection Mechanism.
(1) All written logs and surveys (paper or electronic) are to be retained by their owners until the end of PLIX-1, after which time all logs and surveys are submitted to XMAN;
(2) XOBS(4) performs the following data-collection functions:
(a) Notes the time and nature of any exceptions to nominal OCOM or XCOM performance in a written log; and
(b) Conducts a daily end-of-mission survey (written or verbal) of OCOM and XCOM.
(3) OCOM and XCOM, in a written log, note their own performance exceptions and summarize their overall mission-planning performance.
d. Location. CFB Esquimalt;
e. Storage.
B-36
(1) XOBS(4) retains all original reports and surveys (paper and electronic ) until completion of the experiment; and
(2) OCOM and XCOM retain one copy of their logs. f. Human Resources. XOBS(4), OCOM, and XCOM may need training regarding:
(1) The interpretation of CF command team activity/behaviour; and
(2) The expected activity/behaviour of a CF command team. g. Standards.
(1) All times regarding OCOM and XCOM performance exceptions are recoreded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic reports and surveys must be portable to common operating systems. h. Quality Control. XOBS(4) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries OCOM and XCOM if they noted any performance exceptions (within their own teams); and
(2) Queries OCOM and XCOM if their mission-planning efforts were satisfactory. i. Data Management. XOBS(4) manages the data as follows:
(1) Retains the following items until the end of PLIX-1, and then submits the data to XMAN:
(a) Reports of OCOM and XCOM performance exceptions; and
(b) End-of-mission surveys of OCOM and XCOM.
(2) Immediately notifies XMAN if any of the following conditions arise:
B-37
(a) Loss of data;
(b) If an OCOM or XCOM performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-36, paragraph 25.a) regarding the interpretation of an OCOM or XCOM performance exception.
VESSEL OF INTEREST CREW
26. The crew of the Vessel of Interest (VOI) shall play the part of the object of a search and rescue mission (vessel-in-distress crew) and assume the role of a threat to Canada (intruder crew). The abilities of the VOI crew affects how well PLIX-1 UAV can detect, identify, and track the VOI. This is important when the VOI is the intruder, and especially so when a shore-based mole supports the intruder. Therefore, data regarding the VOI crew shall be collected as follows:
a. Output Data. 5 is the state vector of the VOI crew. 5 indicates the capability to perform their missions as the unfortunate crew of a vessel-in- distress and the radical/outlaw crew of an intruder.
b. Source. The primary source is Experiment Observer Psi-Five, XOBS(5), and the secondary sources are:
(1) The Level-1 Analysts; and
(2) The VOI crew.
c. Collection Mechanism.
(1) All written logs (paper or electronic) are to be retained by their owners until the end of PLIX-1, after which time all logs are submitted to XMAN;
(2) XOBS(5) performs the following data-collection functions:
(a) Notes the time and nature of any exceptions to nominal VOI crew performance in a written log; and
(b) Conducts a daily end-of-mission survey (written or verbal) of the VOI crew.
B-38
(3) The X The Level-1 Analysts note any VOI crew performance exceptions in a written log; and
(4) The VOI crew, in a written log, self-report their performance exceptions and summarize their overall mission-execution performance. d. Location.
(1) Tofino: XOBS(5), and Level-1 Analysts; and
(2) Onboard the VOI: ship’s crew. e. Storage.
(1) XOBS(5) retains all original reports and surveys (paper and electronic ) until completion of the experiment; and
(2) Level-1 Analysts and VOI crew retain one copy of their logs. f. Human Resources. XOBS(5), and the Level-1 Analysts may need training regarding:
(1) The interpretation of VOI activity/behaviour due to crew capabilities and limitations; and
(2) The expected activity/behaviour of the VOI crew. g. Standards.
(1) All times regarding VOI crew performance exceptions are recoreded to the nearest minute and the relevant time zone is explicitly stated (e.g. UT or PDT); and
(2) Electronic reports and surveys must be portable to common operating systems.
B-39
h. Quality Control. XOBS(5) performs the following quality-control functions on a daily basis, immediately upon termination of PLIX-1 UAV operations:
(1) Queries Level-1 Analysts if they noted any evidence of VOI crew performance exceptions;
(2) Queries VOI crew if they noted any performance exceptions (within their own members); and
(3) Queries VOI crew if their mission-execution efforts were satisfactory.
i. Data Management. XOBS(5) manages the data as follows:
(1) Retains the following items until the end of PLIX-1, and then submits the data to XMAN:
(a) Reports of VOI crew performance exceptions; and
(b) End-of-mission surveys of the VOI crew.
(2) Immediately notifies XMAN if any of the following conditions arise:
(a) Loss of data;
(b) If a VOI crew performance exception occurs; and
(c) If there is discrepancy among the data sources (page B-38, paragraph 26.a) regarding the interpretation of a VOI crew performance exception.
THE AERIAL VEHICLE RECOGNISED MARITIME PICTURE
27. The Aerial Vehicle Recognised Maritime Picture (ARMP) for the operations area (see Figure 2, on page 7) must be recorded for the duration of the experiment according to the following specifications:
a. Output Data. The following shall be extracted from the ARMP database with dedicated software applications:
(1) HA , IA, JA , and KA at regular time intervals (every six minutes or
B-40
less) where:
(a) HA : the number of detected contacts in the ARMP;
(b) IA : the number of identified contacts in the ARMP;
(c) JA : the number of false detections in the ARMP since the start of the patrol; and
(d) KA : the number of classified contacts in the ARMP.
(2) LA for all contacts that were detected and identified during the experiment (with a time-resolution of six minutes or less), where:
(a) LA : the contact-identification latency in the ARMP. b. Source. The ARMP; c. Collection Mechanism. Automated (computer database); d. Location. Tofino airport; e. Storage. Save three copies of the ARMP database on CD-ROM disks, one copy remains at Tofino, and two copies are shipped to CFEC for analysis; f. Human Resources. Technical personnel capable of the following:
(1) Initiate and terminate automatic recording;
(2) Produce three copies on CD-ROM; and
(3) No training required. g. Standards. The database must be in a standard format that is portable to common computer operating systems; h. Quality Control. Proxy XDE to monitor the process, and apply a test to all three copies of the ARMP database before original is destroyed; and i. Data Management. Proxy XDE to notify XMAN if the data was successfully collected or not, and where it is archived.
B-41
REFERENCE VESSELS 28. A number of Reference Vessels shall be available to provide their navigational records, or “ground truth”. For example, the Reference Vessel positional data can be compared with the vessel position in the ORMP and XRMP. In addition to the Vessel of Interest, there may be as many as three Canadian Coast Guard vessels. Reference Vessel data shall be collected as follows: a. Output Data. The following shall be extracted from the data: (1) q(k) is the observed state of the Reference Vessel k = 1, 2, 3, 4. For each observation, the time-of-observation and following components of the state vector q(k) are recorded:
(k ) (a) qLat the latitude of Reference Vessel k;
(k ) (b) qLon the longitude of Reference Vessel k;
(k ) (c) qHdg the heading of Reference Vessel k; and
(k ) (d) qSpd the speed of Reference Vessel k.
b. Source. Reference Vessel Navigation system; c. Collection Mechanism. Automated, retained by Reference Vessel until after the experiment, when it is shipped to CFEC; d. Location. Onboard each Reference Vessel that is located within the following latitude and longitude bounds: (1) South of 51° N; (2) North of 48° N; (3) West of 125° W; (4) East of 129° W; and (5) Note that PLIX-1 OPAREA is contained within the above geographic bounds. e. Storage. Electronic files. f. Human Resources. A member of the crew shall control the automated recording equipment and software applications; g. Standards.
B-42
(1) The position, heading and speed of the Reference Vessel is to be saved every six minutes; (2) All times regarding the Reference Vessel data must be recorded and the time zone explicitly stated (e.g. UT or PDT); (3) Position units must be identified (e.g. geographic latitude and longitude in degrees minutes and seconds); (4) Heading units must be identified (e.g. degrees true); (5) Speed units must be identified (e.g. knots); and (6) Electronic records must be portable to common operating systems. h. Quality Control. Reference Vessel crew inspects the recorded data; and i. Data Management. Reference Vessel notifies XMAN of data collection success or failure, and sends the data to CFEC at the earliest opportunity after completion of the experiment.
B-43
This page intentionally left blank
B-44
ANNEX C DOR(JOINT) RN 2003/06 JUNE 2003
ANNEX C. A SIMPLE DETECTION MODEL 1. Consider the following autonomous system of first-order differential equations:
dH f H, K, I dt H dK f H, K, I (C.1) dt K dI f H, K, I dt I
The derivation of the explicit forms of f H , f K , and f I considers the basic interactions between the processes of detection, classification, and identification. The basic interactions are related to the number of contacts available for each process, the capability of each process, and the stability of precursor processes. A process cannot proceed until its precursor process or processes have started and stablised. For example, detection is a precursor process relative to classification, because a target must be detected before it can be classified. Since the number of contacts is assumed constant, the loss rate for a process is due to subsequent processes discovering errors made in precursor processes. For example, a classification of contact X is found to be incorrect (i.e. contact X is not present or needs to be reclassified) when an attempt is made to identify contact X.
BASIC INTERACTIONS
2. The derivation begins with the consideration of basic interactions. In this way, it is similar to the development of the Volterra-Lotka model, which pertains to the population dynamics of a simple predator-prey system (see Reference [16] for a summary). 3. For the detection rate, the basic interactions are between the detector(s) and the undetected objects. Thus, the detection rate is proportional to the number N of objects available for detection
dH N , (C.2) dt
C-1
and proportional to the detection-capability limit
dH N H , (C.3) dt where is the fraction of the objects available for detection that the system is capable of detecting. The parameter introduces the notion of detection capability into the derivation and the product N is the maximum number of contacts that the detector(s) are capable of discovering. 4. Assuming that N is constant, then from (C.2) and (C.3) the detection rate from Basic Interactions (BI) is simply
dH aH N H , (C.4) dt BI where aH is the detection-rate constant. 5. The integration of the differential equation in (C.4) yields
aH t aH t H t BI H 0 e N1 e . (C.5)
Result (C.5) helps to illustrate the statement regarding the product N in paragraph 3, because the maximum number of contacts that the detector(s) can find is
H t BI t N .
6. Some of the basic interactions for the classification rate are analogous to the detection rate. For instance, the classification rate is proportional to the number H of objects that have been detected and are available for classification (note the normalization by the constant N )
dK H , (C.6) dt N and proportional to the classification-capability limit
dK H K , (C.7) dt
C-2
where is the fraction of detected objects that can be classified. The parameter introduces the notion of classification capability because the product H is the maximum number of contacts that the system is capable of classifying. 7. The basic interactions for the classification rate are more complicated than those governing the detection rate. This is because detection is a precursor process to classification. The process of classification requires a stable set of detected objects; therefore, the classification rate is proportional to the stability of the detection process. The instability of the detection process is proportional to (C.3), hence the instability is estimated to first order as follows
dK N H H 1 . (C.8) dt N N
8. The detection rate from basic interactions is the product of (C.6), (C.7), and (C.8)
2 dK H aK H K . (C.9) dt BI N where aK is the classification-rate constant. Suppose H reaches its equilibrium value N, then from (C.4) the detection-rate is dH / dt BI 0 , and under this condition (C.9) integrates to
aK t aK t Kt BI K0e N1 e , H t BI N . (C.10)
From (C.10), the maximum number of contacts that the system can classify is
Kt BI t N (C.11)
9. The basic interactions for the identification rate are similar to those for the classification rate. The identification rate is proportional to the number K of objects that have been classified and are available for identification, which is normalized by the constant N from (C.11)
dI K , (C.12) dt N and proportional to the identification-capability limit
C-3
dI K I , (C.13) dt where is the fraction of classified objects that can be identified. The parameter introduces the notion of identification capability because the product K is the maximum number of contacts that the system is capable of classifying. 10. Classification is a precursor process to identification. The process of identification requires a stable set of classified objects; therefore, the identification rate is proportional to the stability of the classification process. The instability of the classification process is proportional to (C.7), hence the instability is estimated to first order as follows
dI H K 1 . (C.14) dt N
11. The classification rate from basic interactions is the product of (C.12), (C.13), and (C.14)
dI K H K a 1 K I (C.15) I dt BI N N where aI is the identification-rate constant. Suppose H reaches its equilibrium value N and K reaches its equilibrium value N , then from (C.4) and (C.9) the classification-rate and identification-rate are dH / dt BI dK / dt BI 0 ; under these conditions (C.15) integrates to
aI t aI t It BI I0e N1 e , H t BI N and Kt BI N . (C.16)
From (C.16), the maximum number of contacts that the system can identify is
It BI t N (C.17)
INTRODUCTION OF ERRORS
12. The system was assumed to be of limited capability in the formulation of the basic interactions above. Next, the system is assumed to be prone to error. The false-detection function gH K, I is given by
gH K, I K N K gK I , (C.18)
C-4
where K is the false-detection discovery fraction at classification and gK I is the false- classification function given by
g K I I N I , (C.19) where I is the false-classification discovery fraction at identification.
13. The false-detection function g H K, I affects the detection operation by making it appear that there are more objects that the system is capable of detecting; this concept can be described in symbols (Note: “ ” denotes “assumes a new value of”)
N N g H K, I . (C.20)
Apply (C.20) to (C.4) to get the detection-rate for Basic Interactions with Errors (BIE)
dH aH N gH K, I H . (C.21) dt BIE
14. Similarly, the false-classification function gK I affects the classification operation by making it appear that there are more objects that the system is capable of classifying
H H gK I . (C.22)
Apply (C. 22) to (C. 9) to get the classification-rate for Basic Interactions with Errors (BIE)
2 dK H aK H gK I K (C.23) dt BIE N
CORRECTION OF ERRORS
15. The system was assumed to be of limited capability and prone to error. The last major assumption is that the system can correct errors that were introduced in a precursor process in a subsequent process. The detection-rate is reduced by its Error Correction (EC) rate, which is when errors are discovered and corrected in the classification and identification operations
dH dH dH . (C.24) dt dt BIE dt EC
C-5
Errors are discovered only if there is a net positive classification-rate or identification-rate, because if there is no effort to classify and identify contacts in subsequent operations, then no errors can be discovered. Thus, the detection error correction rate is
dH K max0, f K H, K, I I max0, f I H, K, I , (C.25) dt EC
where K is defined in (C.18), f K H, K, I is the net classification-rate defined in (C.1), I is defined in (C.19), and f I H, K, I is the net identification-rate defined in (C.1). 16. The classification-rate is reduced by its Error Correction (EC) rate, which is when errors are discovered and corrected in the classification and identification operations
dK dK dK (C.26) dt dt BIE dt EC
Classification errors are discovered only if there is a net positive identification-rate, because if there is no effort to identify contacts, then no classification errors can be discovered. Thus, the classification error correction rate is
dK I max0, f I H, K, I (C.27) dt EC
where I is defined in (C.19), and f I H, K, I is the net identification-rate defined in (C.1).
C-6
THE RATE FUNCTIONS
17. The detection-rate function f H H, K, I follows from (C.1), (C.23), (C.24), and (C.25)
f H, K, I a N g K, I H max0, f H, K, I H H H K K . (C.28) I max0, f I H, K, I
18. The classification rate function f K H, K, I follows from (C.1), (C.23), (C.26), and (C.27)
2 H f K H, K, I aK H gK I K I max0, f I H, K, I . (C.29) N
19. Since the identification-rate depends only upon basic interactions, the function f I H, K, I follows from (C.1) and (C.15)
K H K f I H, K, I aI 1 K I . (C.30) N N
AN EXAMPLE SOLUTION
20. This section presents a fictitious example. Table C-1 gives the notional values of the nine parameters in the rate functions (C.28), (C.29), and (C.30) for the dynamical system (C.1). The assumed initial conditions used in this example are given in Table C-2. The example solution for the ARMP is plotted in Figure C-1, and the example solution for the XRMP is shown in Figure C-2. If the hypothesis holds, then 1, and Figure C-3 presents the solution where the number of objects detected, classified, and identified are equal at the end of the eight-hour patrol.
C-7
Table C-1: Example Parameters
Parameter Value
N 30
0.9 0.9 0.9
-1 aH 2.0 h -1 aK 1.5 h -1 aI 1.0 h
K 0.3
I 0.03
Note: h-1 denotes “per hour”
Table C-2: Example Initial Conditions
Initial Condition Example H (0) K(0) I(0)
ARMP 0 0 0 XRMP 20 10 6
C-8
Figure C-1: ARMP Example Solution
Figure C-2: XRMP Example Solution
C-9
Figure C-3: XRMP Example Solution if the PLIX-1 Hypothesis Holds
C-10
UNCLASSIFIED SECURITY CLASSIFICATION OF FORM
DOCUMENT CONTROL DATA (Security classification of title, body of abstract and indexing annotation must be entered when the overall document is classified)
1. ORIGINATOR (the name and address of the organization preparing the 2. SECURITY CLASSIFICATION (overall security classification of the document, document. Organizations for whom the document was prepared e.g. including special warning terms if applicable) Establishment Sponsoring a contractor's report, or tasking agency, are entered in Section 8). Operational Research Division UNCLASSIFIED Department of National Defence 121&21752//'*22'6 '0&$5(9,(: *&(&-81( Ottawa, Ontario K1A 0K2
3. TITLE (the complete document title as indicated on the title page. Its classification should be indicated by the appropriate abbreviation (S, C or U) in parentheses after the title) Pacific Littoral ISR Experiment 1 Design
4. AUTHORS (last name, first name, middle initial) Van Bavel, Gregory, H
5. DATE OF PUBLICATION (month Year of Publication of document) 6a. NO OF PAGES (total containing 6b. NO OF REFS (total cited in September 2003 information. Include Annexes, document) Appendices, etc.) 92 16
7. DESCRIPTIVE NOTES (the category of document, e.g. technical report, technical note or memorandum. If appropriate, enter the type of report e.g. interim, progress, summary, annual or final. Give the inclusive dates when a specific reporting period is covered.) Research Note (summary)
8. SPONSORING ACTIVITY (the name of the department project office or laboratory sponsoring the research and development. Include the address). Canadian Forces Experimentation Centre, National Defence Headquarters
9a. PROJECT OR GRANT NO. (if appropriate, the applicable research 9b. CONTRACT NO. (if appropriate, the applicable number under which the and development project or grant number under which the document was document was written.) written. Please specify whether project or grant.) N/A N/A
10a. ORIGINATOR's document number (the official document number by 10b. OTHER DOCUMENT NOS. (Any other numbers which may be assigned which the document is identified by the originating activity. This number this document either by the originator or by the sponsor.) must be unique to this document.) N/A DOR(Joint) Research Note RN 2003/06
11. DOCUMENT AVAILABILITY (any limitations on further dissemination of the document, other than those imposed by security classification.) (X) Unlimited distribution ( ) Distribution limited to defence departments and defence contractors: further distribution only as approved ( ) Distribution limited to defence departments and Canadian defence contractors; further distribution only as approved ( ) Distribution limited to government departments and agencies; further distribution only as approved ( ) Distribution limited to defence departments; further distribution only as approved ( ) Other (please specify):
12. DOCUMENT ANNOUNCEMENT (any limitation to the bibliographic announcement of this document. This will normally correspond to the Document Availability (11). However, where further distribution (beyond the audience specified in 11) is possible, a wider announcement audience may be selected.) N/A
SECURITY CLASSIFICATION OF FORM UNCLASSIFIED UNCLASSIFIED
SECURITY CLASSIFICATION OF FORM
13. ABSTRACT (a brief and factual summary of the document. It may also appear elsewhere in the body of the document itself. It is highly desirable that the abstract of classified documents be unclassified. Each paragraph of the abstract shall begin with an indication of the security classification of the information in the paragraph (unless the document itself is unclassified) represented as (S), (C), or (U). It is not necessary to include here abstracts in both official languages unless the test is bilingual).
The Canadian Forces Experimentation Centre is tasked with Concept Development and Experimentation (CD&E), which is contributing to the transformation of the forces. Strategy 2020 indicates the goal of the transformation, and the Canadian Joint Task List specifies the capability areas that require CD&E. Information and Intelligence (I2) is the capability area of interest to the Pacific Littoral ISR Experiment 1 (PLIX-1). This experiment attempts to enhance I2 capabilities pertaining to a selected operations area in a littoral environment. An uninhabited aerial vehicle shall survey the littoral operations area with its sensors. Level-1 analysts shall use the radar and camera data to generate a tactical-level operating picture and post selected information and images to a website. Higher-level analysts shall fuse the tactical-level operating picture with the Recognised Maritime Picture (RMP) to generate an Experimental RMP (XRMP). The measure of effectiveness of this I2 experiment shall be based upon the completeness of the XRMP relative to the ordinary RMP. The completeness will be measured by a count of the number of positively identified contacts against the null hypothesis that the XRMP will not be improved. The measure of effectiveness is quantifiable and the hypothesis is falsifiable. The I2 operation shall also be assessed with a measure of force effectiveness. During PLIX-1, two command teams will generate plans for four assigned missions; one command team shall use the ordinary RMP, the other the XRMP. The indicator for the measure of force effectiveness shall be savings in CF resources and improved timeliness to execute the mission-plan under the XRMP relative to the ordinary RMP.
14. KEYWORDS, DESCRIPTORS or IDENTIFIERS (technically meaningful terms or short phrases that characterize a document and could be helpful in cataloguing the document. They should be selected so that no security classification is required. Identifiers, such as equipment model designation, trade name, military project code name, geographic location may also be included. If possible keywords should be selected from a published thesaurus, e.g. Thesaurus of Engineering and Scientific Terms (TEST) and that thesaurus-identified . If it is not possible to select indexing terms which are Unclassified, the classification of each should be indicated as with the title.)
Concept Development and Experimentation (CD&E) Military Experimentation Design Data Collection Intelligence, Surveillance, and Reconnaissance (ISR) Integrated ISR Architecture (IISRA) Unmanned Aerial Vehicles (UAV) Uninhabited Aerial Vehicles (UAV)
SECURITY CLASSIFICATION OF FORM UNCLASSIFIED