Typhoon Simulator Record and Replay Debrief Facility

UNCONTROLLED 1

Typhoon Simulator Record Replay De-brief Facility (RRDF)

Duncan Porteous Software Team Leader Air Service – Aircrew Training Simulation Systems BAE Systems – Air

Typhoon Simulator Record Replay De-brief Facility (RRDF)

Summary

• Overview of the Typhoon Simulators used to train Air Force pilots

• Debrief capability developed for the RAF Typhoon Simulators • key requirements • system design • technology used • technical challenges

Typhoon Simulators RAF Lossiemouth

• Typhoon Simulators deployed for UK (RAF), Kingdom of Saudi Arabia (RSAF) and Oman (RAFO) supporting aircrew and ground crew training.

• Provides concurrent training capability to Air Force – software updates done to simulators prior to front line aircraft upgrade.

• Provides training activities from cockpit familiarisation, emergency procedural training, RAF Coningsby engineering training, operational tactical training and mission rehearsal in networked environments.

• Wide area network support for collective training with other distributed simulators (e.g. E3, Type 45) through Air Battle-Space Training Centre (ABTC).

Typhoon Simulators

• Range of device types – varying Human Machine Interface – physical cockpit, visual projection system, instructor console. Avionics Part Task Trainer (APTT)

Enhanced Typhoon Avionics Demonstrator (ETAD+) - Groundcrew Emulated Deployable Cockpit Trainer (EDCT) Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 5

Typhoon Simulators

• All simulator types share common system architecture - combination of rehosted aircraft software and simulated software.

• Approximately 3.6 Million Software Lines of Code.

• Simulates databus interactions of each Line Replaceable Items (LRIs) on the aircraft.

• 372 individual malfunctions simulated.

• Customisable Scenario including weather effects and Computer Generated Forces that provide hostile and friendly entities with which to interact.

• Instructor led training from “Over-the-Shoulder” or remote Operator Station.

• Can be operated in “self-help” mode with entire sim controlled from pilot in cockpit.

Every sim needs a debrief…

Background

• The RAF were debriefing Typhoon pilots using hand-written notes taken during simulated sorties conducted on the Emulated Deployable Cockpit Trainers (EDCTs). These notes were expanded on whiteboards post simulated flight.

• In 2015, the RAF were preparing to include Typhoon in UK deployment to Syria as part of Operation Shader and ability to provide a level of debrief capability as part of combat ready work-up in the simulators would have been advantageous.

Interim Playback Facility

A temporary solution for the RAF • A solution using COTS hardware and software components was rapidly developed and integrated with the existing training simulators allowing delivery of a debrief capability to both RAF bases (Lossiemouth and Coningsby) within 4 months of conception. • Cockpit displays, Pilot and Instructor audio and the Tactical Map recorded to video file using Bandicam. • Open Source Media player (VLC) to play back video files – loosely synchronised. • Easy to operate, low cost. • No bookmarks for events. • No interaction with tactical map (recorded video).

RRDF System

Key Requirements

• “Theatre Style” debrief facility, with text readable up to 5m.

• Record any combination of simulator devices, from simultaneously running but independent missions, to linked (upto 4 simulator) multi-ship scenarios.

• Simultaneous Record and Playback or Live Monitor with audio, video and data synchronised between each simulator.

• Capable of playback from ¼ speed upto x32, forward and backwards.

• Tactical Scenario Viewer range of situational awareness functions.

• Timeline & Event markers (manual and automatic bookmarks).

1 display per High Contrast RRDF Theatre simulator device Tactical Map IPS Monitors • 15.6” portable (tethered) HD touch screen provides single point of control for theatre.

• Recording or device to Live Monitor selection • Video Stream Layout Selection • Individual audio stream volume control/mute • Playback navigation (timeline & event bookmarks) • Tactical Map Management • Recording Management (favourite, delete, archive)

Touchscreen (15.6” HD) Video Wall Displays - (6 x 55” UHD 3840 x 2160)

Video Streams HUD Symbology + Helmet Mounted Symbology System + Outside World Generated Image Outside World Generated Image 1920 x 1080 1920 x 1080 @ 60Hz @ 60Hz

Driven by Head pointing Background angles from switchable inertia cube on/off to leave just Symbology

Switch highlighting on change

2560 x 1440 2560 x 1440 @ 20Hz @ 20Hz Primary Displays HOTAS + Cockpit Switches Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 11

Video Streams

HUD Symbology + Outside World Generated Image (turned off)

Individual Multifunction Head Down Displays

800 x 600 @30Hz

RRDF System Architecture

Encode Stream Record Store • Utilise dedicated h/w on the GPU to capture Audio & Video HLA and compress an image to a H.264 Data Simulator Media Real Time Recorder & PostgreSQL video stream (RTP). Streamer Infrastructure Recorder Manager Live Monitor

• Total Data Rate: ~215 Mbps • 5 x COTS PCs, 10Gb Ethernet Media Real Time Recorder & Streamer Infrastructure Recorder Manager • Database PC uses NVMe Solid State Drive (SSD) for read/write • Media Player that will Data Audio & HLA Replay Video performance. render the H.264 RTP • Mechanical drives (Raid 10) for video streams, bulk storage. including the blending Tactical Media Controller of multiple video Map Player • Typical recording size for 4 streams together. Decode & Display Control simulators ~95GB / hour

Video Wall Monitors RRDF Controller 2 x UHD (3840 x 4320)

• Interactive Tactical Map is a separate application and provides scenario awareness, with graphical and textual overlays.

• Runs an instance of the Digital Map Generator (DMG) simulation from the simulator, which includes ability to display tactical symbology such as missile engagement zones.

• Desktop Windows Manager (DWM) provides a Tactical Map Map Controls hardware accelerated repeat of both the tactical map and controls display windows and embeds them in the Theatre controller GUI.

• Touchscreen inputs passed back to Tactical Map application using standard Windows API calls. PC

Challenges

• Mechanism for encoding and streaming multiple video, audio and data feeds from each simulator – negligible impact on the execution performance of the simulator.

• Recording and playback system – capture and store streams in synchronised manner, sufficient bandwidth high speed playback.

• Present and manage simulator information to the user with a coherent front end interface - integration of a number of systems and applications (different software languages).

• Make / buy trade-off (rare to find COTS products that do exactly what you want). • Combination of COTS, development to COTS and in-house developed capability.

Challenges

• Tuning of the system to cope with high bandwidth of streams associated with 4 devices, especially during high speed playback. • Limit system demand to <600 Mbps. • Mute select streams during high speed playback. • Optimisations made to UDP buffer size in Windows and applications. • Pitch application optimisations to increase performance for database access.

• Involvement of customer and end-user is essential during all stages of development. • Work shops covering requirements and GUI design. • Maturity reviews demonstrating system performance and enabling early validation of design.

Summary

• Produced a system that implements a highly responsive and capable record and playback system with just standard COTS hardware and software applications. • No specialist hardware for encoding and decoding audio/video needed. • Bandwidth of data streams scalable according to hardware capacity (No. PCs, HD read/write, network capacity). • Built on typical software applications used for interoperability. • Readily available tactical scenario viewers & media players can be used.

• Flexible and expandable architecture to support future capability.

Thank You

Restrictions on use: Permission to reproduce any part of this document should be sought from BAE Systems. Permission will usually be given providing the source is Acknowledged and the copyright notice and this notice are reproduced.