An Analytical Approach to Emergency Communications Exercises Cascadia Rising
• June 8 and 9, 2016
• Scenario: A catastrophic 9.0 earthquake and tsunami impacting the Pacific Northwest.
• 6,000 participants drawn from local, state, federal agencies and NGOs.
• Major disruptions to commercial and government telecommunications infrastructure and the power grid anticipated.
• Situation reports from within the disaster area would be critical to developing a cohesive and efficient response.
1/30/2017 Radio Relay International 2 FEMA Requirements
• High Frequency methods would be FEMA’s “weapon of choice” – Survivable – Decentralized – Not reliant on distributed infrastructure – Immune to topographical problems
• All Radio Frequency – No reliance on Internet, PSTN or other distributed infrastructure.
• Mode neutral – maximum interoperability.
• Focus: Long-haul communications between simulated disaster area and FEMA National Response Coordinating Center (NRCC) at Washington, D.C. tested.
1/30/2017 Radio Relay International 3 Selected System Characteristics
• Standardized procedures throughout North America.
• Universal message format provides superior accountability and network management tools including rules for message prioritization.
• Unified national-level management facilitates coordination across jurisdictional boundaries.
• Full interoperability across multiple modes and networks.
• Common denominator modes (universality) retained.
• Automated digital network available.
• Operator dispersal – decentralized.
1/30/2017 Radio Relay International 4 Specific Areas of Interest
• Test reliability of the communications system to maintain connectivity across a broad range or RF conditions.
• Test the system’s prototype National Emergency Communications Response Plan.
• Examine the relative performance of the individual networks within the system: – Timeliness – Accuracy – Operator performance
1/30/2017 Radio Relay International 5 System Structure
An overview of network types tested during the exercise. Western Area Central Area Eastern Area
1/30/2017 Radio Relay International 7 To Western To Eastern Area Area Central Area Network
1/30/2017 Radio Relay International 8 Digital Traffic Network
• Modified hybrid mesh network built on PACTOR methods. International in scope (connections to Europe, Asia, Oceana, etc.).
• Operational 24-hours per day.
• All RF – no Internet.
• Interoperable with voice, CW circuits, and local digital networks via the Digital Traffic Station (DTS) function.
• Backwards compatible from Pactor 3(4) to Pactor 1.
• VHF gateways can be easily established at county/municipal level
1/30/2017 Radio Relay International 9 “Area Backbone”
“Region Hubs”
“State Hubs”
“Local Access Points”
1/30/2017 Radio Relay International 10 DTN is modern and easily implemented
Photo courtesy Timothy Eldridge Photo courtesy J. DeAngelo
1/30/2017 Radio Relay International 11 Radiotelegraph & Radiotelephone
• Universal common-denominators.
• Forward-deployable as man-pack, portable, mobile, etc.
• CW well suited to low-power and renewable energy resources.
• Voice-capable transceivers are ubiquitous.
• Efficient in the hands of qualified operators. Photo courtesy Portable Zero, LLC • Basic HF modes work everywhere.
1/30/2017 Radio Relay International 12 Central Region Network Topography
1/30/2017 Radio Relay International 13 Inter-Area Traffic Network Topography
Area Networks Point-to-Point Circuits
1/30/2017 Radio Relay International 14 Prototype National Response Plan
• Special IATN routings to FEMA NRCC via watch (QSX) frequencies spaced between 3 and 30 MHz.
• QSX frequencies assigned for exercise priority and exercise emergency traffic only.
• IATN ran parallel to automated Digital Traffic Network. – Common denominator access. – Order wire service. – Overflow message traffic. – Redundancy.
1/30/2017 Radio Relay International 15 Network Topology
• Multiple injection points speed priority or emergency traffic – spread traffic load.
• Network failures easily and seamlessly bypassed.
• Full interoperability between modes and network layers.
• Interoperable features are scalable to dynamically respond to varying traffic loads and priorities.
1/30/2017 Radio Relay International 16 Communications Security • Frequency and network management plan distributed only to operational units and individual participants.
• Frequency/Mode Matrix not published on the Web.
• Considerable research was required to identify suitable allocations for the prototype plan.
• Frequency designators used to identify QSX frequencies on- air.
• Radiotelegraph and digital networks resistant to public intercept.
1/30/2017 Radio Relay International 17 Sample Portion of State/Region Net Frequency Matrix
1/30/2017 Radio Relay International 18 DTN Partial Mode/Frequency Matrix
1/30/2017 Radio Relay International 19 IATN Mode/Frequency Matrix
1/30/2017 Radio Relay International 20 HF phone / cw
HF Pactor Internet HTTPS
1/30/2017 Radio Relay International 21 Background Does the emergency plan work?
Cascadia Rising Exercise Design Philosophy Designing a Communications Exercise
Important questions must be answered:
1. Is connectivity established and maintained?
2. Is the network survivable?
3. Is the network capable of accurately conveying information?
4. Is the information conveyed in a timely manner?
5. Is the proper accountability and network management data in place?
Most emergency management exercises answer ONLY the first question, but we need to examine the entire communications “chain.”
1/30/2017 Radio Relay International 23 The Emergency Communications Pyramid
1/30/2017 Radio Relay International 24 An Analogy – Train Control Network
Total Safety Train Detection Interlocking Control Relays Wayside Signals Level = SIL4 Processor SIL1 SIL4 SIL1 SIL4
1/30/2017 Radio Relay International 25 Train Control Network + Human Factor
SIL4 Train Control Network
The highest level of technological safety can be compromised by human failure. The same is true of the emergency services communications process.
A lack of training in effective communications procedures undermines any technological solution.
1/30/2017 Radio Relay International 26 Human Factors vs. Technological Factors • Most communications specialists concentrate on technological solutions.
• Technology alone does not solve the “human interface” problem. – Data entry errors; “garbage in – garbage out.” – Misunderstandings related to technical terms. – Loss of accountability data (authority, position, etc). – Temporal misplacement or date-time group errors. – Inability to catalog and reference message traffic from a file. – Inability to keep a running radio log with understandable summaries – Lost messages.
Never seek a technological solution until managerial problems are resolved.
1/30/2017 Radio Relay International 27 What size Aspirin?
15.6 81 15.6 g 81 mg
One missing letter can make a difference!
1/30/2017 Radio Relay International 28 What are we testing? • The exercise must test the entire communications process including both the technology and the methodology of communications. The latter contains a significant human element.
• Technology = Medium whereas Communications = Entire Process.
– The exercise must be realistic. – The exercise must be measurable. – The goals must be attainable.
The timely delivery of accurate messages, whether by cellular data network or carrier pigeon is what matters!
1/30/2017 Radio Relay International 29 Medium vs. Final Product
Above: Press telegraphers at the infamous Lonergan murder trial in New York City – 1943
Left: Carrier pigeons used to carry message traffic during World War Two
1/30/2017 Radio Relay International 30 Cascadia Rising Exercise Design & Evaluation
1/30/2017 Radio Relay International 31 Collecting Data - Metrics
• Controlled inject messages were developed to determine accuracy through comparison to the delivered product.
• The time at which each message was injected was controlled and accurately logged.
• The time at which messages were relayed or transferred between networks, and to/from whom were logged at various points throughout the network.
• Network topology could then be defined by tracking message flow. This allows evaluators to identify network (or human) failure points.
A message may be transferred between several networks to achieve “last mile” connectivity or to bypass network failures.
1/30/2017 Radio Relay International 32 Cascadia Rising Message Origination • Preformatted messages and guidance documents distributed to operators in Alaska, Idaho, Northern California, Oregon and Washington State.
• Messages properly formatted in standard “radiogram” (“ICS213 on steroids”) format. – Accountability – Essential network management data – Temporal context (date-time group)
• All message text consisted of five-letter cipher groups.
• Occasional minor variations introduced into a small quantity of messages to test “assumptions.” Example: inclusion of a six letter cipher group or inclusion of a counter-intuitive date-time group or group-count (check).
• Original messages contained in sealed, date-time stamped envelopes indicating time to be presented for origination (injection into exercise).
1/30/2017 Radio Relay International 33 Typical exercise packet SASE for returning all distributed in Field message copies and logs at exercise conclusion
Message Logs and Exercise Guidelines Documents
Sealed messages to be opened & origination at time indicated
1/30/2017 Radio Relay International 34 Message Destination
• Requirement: Route was entirely RF from Pacific Northwest to final delivery to FEMA National Response Coordinating Center.
• Messages delivered only upon reaching the destination area. Grid-down scenario.
• WebEOC used to facilitate message delivery and to provide accurate, written and date-time stamped record of final product as delivered upon network exit.
• Carefully vetted operators served as liaison to NRCC to limit skill variables to assets in the field.
1/30/2017 Radio Relay International 35 Network Management Messages
• Large scale disasters require knowledge of network topology and connectivity:
– On what comm. network is a specific agency/official represented?
– What frequencies, modes/methods are available within a particular communications unit?
– What unique operational limitations does an individual operator/agency have (limited fuel supply, staffing limitations, frequency limitations, transmitter failures, etc.).
1/30/2017 Radio Relay International 36 Network Manager • Exercise participants were required to originate periodic status messages at specific times to an assigned network manager.
• Network manager maintained a database of operator capabilities and to generate a picture of network topology for reply message routing, access to served agencies and specific officials.
• Of particular interest were alternate routings (paths) and last-mile capabilities.
• Network manager was located off-site (not at NRCC).
• Because of their dynamic nature, network management messages were reviewed from a qualitative standpoint, but NOT scored (i.e. controlled injects unavailable).
1/30/2017 Radio Relay International 37 Measurements
• Text of messages as delivered were compared against control copies of the messages presented for origination and injected within the simulated disaster area.
• Messages scored according to specific, objective criteria.
• Both quantity of errors and error pattern determined fatal or nonfatal errors.
• Time from origination to time of delivery recorded for each message.
• Connectivity failures, delays, significant outliers noted.
1/30/2017 Radio Relay International 38 Grading Criteria
Cipher groups require analytical grading scheme:
1. Each letter or figure within the radiogram was treated as a unique data point.
2. Each distinct error, such as an improperly transcribed letter or figure was graded as a single error.
3. Up to three errors associated with different cipher groups were treated as individual errors. Four or more errors within an entire message were treated as a fatal error and the message was discarded.
4. Two errors within a single group (five letter cipher group or word) were counted as five errors.
5. Three errors within a single group were counted as a fatal error and the message was discarded.
1/30/2017 Radio Relay International 39 Sample Message Originated vs. Delivered
Controlled Inject Message
Message as delivered
1/30/2017 Radio Relay International 40 Typical completed log
1/30/2017 Radio Relay International 41 Analysis
How did the system perform? Some surprising results were obtained and important lessons learned. Accuracy Measurements
Alaska Intrastate PACTOR Network: 100 % Digital Traffic Network: 99.997% Combined Radiotelegraph: 99.998%
Radiotelephone: Voice methods not used beyond state level and were therefore not measured.
1/30/2017 Radio Relay International 43 Message Propagation Times Radiotelegraph Measured from time message was originated in disaster area until the time it appeared in the data stream at NRCC:
– Average message propagation time: 30.5 minutes.
– A few statistical outliers due to human error. For example; one misplaced message resulted in delay of 110 minutes.
– Remove several outliers, and average total message propagation time drops into 10 to 15 minute range.
– One message propagation time measured at 1 minute!
1/30/2017 Radio Relay International 44 Message Propagation Times Alaska Intrastate Network • Hybrid Network. Intrastate traffic transmitted using PACTOR to a radiotelegraph gateway at Fairbanks, AK.
• Radiotelegraph circuit from Alaska to Region 7 Washington State (appx. 2000 mile circuit).
• Traffic then routed via IATN to FEMA NRCC.
• Alaska message propagation times integrated into IATN (radiotelegraph) network statistics.
1/30/2017 Radio Relay International 45 Message Propagation Times Digital Traffic Network • Average total message propagation time: 82.4 min.
• Marginal radio frequency conditions during exercise phase one and three resulted in connect failures.
• Co-channel and adjacent channel interference: Heavy demand on the automated sub-bands resulted in excessively long delays due to carrier detect holds. DTN shares small slice of spectrum with WL2K.
• WL2K frequencies used at local/state level – “all eggs in one basket.”
• No manual attendants (sysops) present at DTN hubs (by exercise design).
1/30/2017 Radio Relay International 46 Environmental Influences: Poor RF conditions negatively impacted DTN performance RF Conditions – CONUS
1/30/2017 Radio Relay International 47 A Sample of Metrics
1/30/2017 Radio Relay International 48 Lessons Learned
1/30/2017 Radio Relay International 49 Lessons from exercise results…
• Radio operator skill and experience is the key to success.
• A diversity of modes is essential to an effective disaster communications strategy.
• Manual modes (radiotelegraph) proved more effective due to the ability of operators to respond dynamically (and instantaneously) to frequency occupancy and poor radio frequency conditions.
• Common-denominator methods offer versatility and universal access.
1/30/2017 Radio Relay International 50 Lessons continued….
• Digital Traffic Network (DTN) is less agile. It therefore requires additional hubs with better geographic dispersal to accommodate poor radio frequency conditions.
• During major disasters, an attendant (sysop) should be present at hubs to monitor performance and message throughput.
• Higher power capabilities and, if possible, directional antennas to target a disaster zone would be beneficial at region and area hub facilities.
• By staffing hubs, alternate frequencies outside the automated sub- bands could be employed in time of emergency. NECRP to be modified accordingly.
1/30/2017 Radio Relay International 51 Lessons continued…
• Ultimately, the human element proved most important:
– Experience played an important role. Participant backgrounds included US Army Security Agency, US Navy, Maritime Radio Officers, and radio amateurs with extensive, daily experience in message handling.
– It is easy to misplace a message when large quantities are manually transferred between networks. Good administrative skills are essential to success.
– Multi-operator facilities outside the affected area would be beneficial.
– Greater depth of operators would be required to support an actual event over a multi-day/24-hour operation….more volunteers.
1/30/2017 Radio Relay International 52 Lessons continued
• Additional network management data associated with the radiogram format afforded improved tracking and administration.
• Survivable high frequency radio networks hold significant promise for basic emergency communications response and situational awareness reporting.
• Ultimately, the most survivable networks proved most effective. In order of importance: – Survivability – Flexibility – Circuit Capacity – Security
1/30/2017 Radio Relay International 53 Contact Information
James Wades Radio Relay International PO Box 192 Buchanan, MI. 49107 269-650-0215 [email protected] [email protected]
All contents copyright 2017 Radio Relay International – All rights reserved.
1/30/2017 Radio Relay International 54