UNCONTROLLED 1 Typhoon Simulator Record Replay De-brief Facility (RRDF) Duncan Porteous Software Team Leader Air Service – Aircrew Training Simulation Systems BAE Systems – Air Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 2 Typhoon Simulator Record Replay De-brief Facility (RRDF) Summary • Overview of the Typhoon Simulators used to train Air Force pilots • Debrief capability developed for the RAF Typhoon Simulators • key requirements • system design • technology used • technical challenges Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 3 Typhoon Simulators RAF Lossiemouth • Typhoon Simulators deployed for UK (RAF), Kingdom of Saudi Arabia (RSAF) and Oman (RAFO) supporting aircrew and ground crew training. • Provides concurrent training capability to Air Force – software updates done to simulators prior to front line aircraft upgrade. • Provides training activities from cockpit familiarisation, emergency procedural training, RAF Coningsby engineering training, operational tactical training and mission rehearsal in networked environments. • Wide area network support for collective training with other distributed simulators (e.g. E3, Type 45) through Air Battle-Space Training Centre (ABTC). Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 4 Typhoon Simulators • Range of device types – varying Human Machine Interface – physical cockpit, visual projection system, instructor console. Avionics Part Task Trainer (APTT) Enhanced Typhoon Avionics Demonstrator (ETAD+) - Groundcrew Emulated Deployable Cockpit Trainer (EDCT) Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 5 Typhoon Simulators • All simulator types share common system architecture - combination of rehosted aircraft software and simulated software. • Approximately 3.6 Million Software Lines of Code. • Simulates databus interactions of each Line Replaceable Items (LRIs) on the aircraft. • 372 individual malfunctions simulated. • Customisable Scenario including weather effects and Computer Generated Forces that provide hostile and friendly entities with which to interact. • Instructor led training from “Over-the-Shoulder” or remote Operator Station. • Can be operated in “self-help” mode with entire sim controlled from pilot in cockpit. Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 6 Every sim needs a debrief… Background • The RAF were debriefing Typhoon pilots using hand-written notes taken during simulated sorties conducted on the Emulated Deployable Cockpit Trainers (EDCTs). These notes were expanded on whiteboards post simulated flight. • In 2015, the RAF were preparing to include Typhoon in UK deployment to Syria as part of Operation Shader and ability to provide a level of debrief capability as part of combat ready work-up in the simulators would have been advantageous. Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 7 Interim Playback Facility A temporary solution for the RAF • A solution using COTS hardware and software components was rapidly developed and integrated with the existing training simulators allowing delivery of a debrief capability to both RAF bases (Lossiemouth and Coningsby) within 4 months of conception. • Cockpit displays, Pilot and Instructor audio and the Tactical Map recorded to video file using Bandicam. • Open Source Media player (VLC) to play back video files – loosely synchronised. • Easy to operate, low cost. • No bookmarks for events. • No interaction with tactical map (recorded video). Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 8 RRDF System Key Requirements • “Theatre Style” debrief facility, with text readable up to 5m. • Record any combination of simulator devices, from simultaneously running but independent missions, to linked (upto 4 simulator) multi-ship scenarios. • Simultaneous Record and Playback or Live Monitor with audio, video and data synchronised between each simulator. • Capable of playback from ¼ speed upto x32, forward and backwards. • Tactical Scenario Viewer range of situational awareness functions. • Timeline & Event markers (manual and automatic bookmarks). Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 9 1 display per High Contrast RRDF Theatre simulator device Tactical Map IPS Monitors • 15.6” portable (tethered) HD touch screen provides single point of control for theatre. • Recording or device to Live Monitor selection • Video Stream Layout Selection • Individual audio stream volume control/mute • Playback navigation (timeline & event bookmarks) • Tactical Map Management • Recording Management (favourite, delete, archive) Touchscreen (15.6” HD) Video Wall Displays - (6 x 55” UHD 3840 x 2160) Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 10 Video Streams HUD Symbology + Helmet Mounted Symbology System + Outside World Generated Image Outside World Generated Image 1920 x 1080 1920 x 1080 @ 60Hz @ 60Hz Driven by Head pointing Background angles from switchable inertia cube on/off to leave just Symbology Switch highlighting on change 2560 x 1440 2560 x 1440 @ 20Hz @ 20Hz Primary Displays HOTAS + Cockpit Switches Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 11 Video Streams HUD Symbology + Outside World Generated Image (turned off) Individual Multifunction Head Down Displays 800 x 600 @30Hz Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 12 RRDF System Architecture Encode Stream Record Store • Utilise dedicated h/w on the GPU to capture Audio & Video HLA and compress an image to a H.264 Data Simulator Media Real Time Recorder & PostgreSQL video stream (RTP). Streamer Infrastructure Recorder Manager Live Monitor • Total Data Rate: ~215 Mbps • 5 x COTS PCs, 10Gb Ethernet Media Real Time Recorder & Streamer Infrastructure Recorder Manager • Database PC uses NVMe Solid State Drive (SSD) for read/write • Media Player that will Data Audio & HLA Replay Video performance. render the H.264 RTP • Mechanical drives (Raid 10) for video streams, bulk storage. including the blending Tactical Media Controller of multiple video Map Player • Typical recording size for 4 streams together. Decode & Display Control simulators ~95GB / hour Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 13 Video Wall Monitors RRDF Controller 2 x UHD (3840 x 4320) • Interactive Tactical Map is a separate application and provides scenario awareness, with graphical and textual overlays. • Runs an instance of the Digital Map Generator (DMG) simulation from the simulator, which includes ability to display tactical symbology such as missile engagement zones. • Desktop Windows Manager (DWM) provides a Tactical Map Map Controls hardware accelerated repeat of both the tactical map and controls display windows and embeds them in the Theatre controller GUI. • Touchscreen inputs passed back to Tactical Map application using standard Windows API calls. PC HD Touchscreen Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 14 Challenges • Mechanism for encoding and streaming multiple video, audio and data feeds from each simulator – negligible impact on the execution performance of the simulator. • Recording and playback system – capture and store streams in synchronised manner, sufficient bandwidth high speed playback. • Present and manage simulator information to the user with a coherent front end interface - integration of a number of systems and applications (different software languages). • Make / buy trade-off (rare to find COTS products that do exactly what you want). • Combination of COTS, development to COTS and in-house developed capability. Copyright © 2018 BAE SYSTEMS. All rights reserved. BAE SYSTEMS is a registered trademark of BAE Systems plc. (see final slide for restrictions on use) UNCONTROLLED 15 Challenges • Tuning of the system to cope with high bandwidth of streams associated with 4 devices, especially during high speed playback. • Limit system demand to <600 Mbps. • Mute select streams during high speed playback. • Optimisations made to UDP buffer size in Windows and applications. • Pitch application optimisations to increase performance for database access. • Involvement of customer and end-user is essential during all stages of development. • Work shops