Human Interface Technologies Team: West Country - Based Projects [email protected]

Human Interface Technologies Team: West Country - Based Projects Profbobstone@Gmail.Com

Page | 1

Contents

Background 1 Virtual Scylla 2 Visualising Plymouth Sound 2 Submarine Training – SubSafe 4 HMS Courageous 4 The Maria 5 The A7 Submarine 5 The Amethyst 6 The GLAUCUS 6 Virtual Wembury 7 Wembury … The Dock That Never Was 7 Virtual Burrator 8 Burrator & Sheepstor Halt 8 Burrator: What Lies Beneath 9 RAF Harrowbeer, Yelverton 11 Drones Over Plymouth & Dartmoor 12 Longstone Manor 12 Foggintor & Haytor Quarries 13 Hooe Lake Wrecks 14 3D Scanning Projects 16 Medical Emergency Response Team Trainer 18 Pengelly Cave Studies Trust 18 The Virtual Mayflower 19 Mayflower Autonomous Ship 21 Mixed Reality Science Station 21

Background

The Human Interface Technologies (HIT) Team has been undertaking high-tech research and development projects in Plymouth and surrounding coastal and moorland areas since 2003, when the Team was formed by its current director, Plymothian born-and-bred Professor Bob Stone. The choice of Devon for the Team’s R&D projects is not difficult to understand when reading the summaries provided below. The County has provided the HIT Team researchers with an abundance of opportunities to survey and recreate areas of natural beauty using a combination of Virtual Reality, Augmented Reality (VR/AR) and drone-based technologies. Locations such as Wembury Bay and Burrator have become focal areas for the purposes both of delivering simulated virtual scenes of nature for hospital recovery and rehabilitation applications, and for projects in the domain of Virtual Heritage – using VR and AR techniques to turn back time or to make the historically invisible visible once again. Indeed, the maritime heritage evident throughout the County of Devon was responsible for “kick-starting” what, today, amounts to nearly 15 years of project work in the region, building on a foundation provided by two key, early projects, the virtual recreation of the ex-Leander Class Frigate, the Scylla, and the successful trials, with Her Majesty’s Naval Base at Devonport, of the first VR training system for nuclear submarines, SubSafe.

Page | 2

Virtual Scylla

Virtual Scylla was a project undertaken with the aim of showing how state-of-the-art developments in VR could deliver an environmental research and awareness tool accessible to a wide range of possible users, from marine biologists and engineers to schoolchildren and members of the general public. A 3D model of the Scylla was developed that could be explored in real time by piloting a virtual remotely operated (submersible) vehicle (ROV) using a Microsoft Xbox gamepad or other interactive controller. To add context to the experience, a geographically accurate model of the Whitsand Bay coastline, from Rame Head to Portwrinkle was also constructed using digital terrain map data and aerial photography. To endow the subsea wreck model with an educationally-relevant experience, members of the HIT Team also developed software techniques in artificial life (or alife) – the scientific study and simulation of the behaviour of biological organisms and systems – to investigate and predict how natural environments survive, reproduce, colonise and evolve and may be affected by environmental changes brought about by climate change, extreme weather events or pollution.

Throughout the project, the Team collaborated with the National Marine Aquarium in Plymouth (NMA) and the Marine Biological Association, and further technical support was provided by the Royal Navy’s Hydrographic, Meteorological and Oceanographic Training Group in Devonport. The Team visited the site of the actual Scylla Reef on numerous occasions using a VideoRay remotely operated submersible. In 2009, the Virtual Scylla alife and wreck “fly-through” demonstrators were presented to schoolchildren and specialist adult audiences at the NMA.

Visualising Plymouth Sound

This subsea visualisation project, although originally defence-oriented in nature, evolved from a civilian sector R&D study, in collaboration with the National Marine Aquarium (NMA) and the Marine Biological Association in Plymouth. This study, described earlier, addressed how artificial life techniques were able to simulate the marine colonisation of the virtual recreation of HMS Scylla. Whilst presenting the Virtual Scylla project at an NMA public event, a representation of FOST HM (Flag Officer Sea Training – Hydrography and Meteorology) expressed interest in developing the visualisation effort further, to address the visualisation of seabed topography and artefacts, using bathymetric data collected by the Royal Navy’s Hydrographic Fleet. This requirement evolved further into the development of a visualisation tool that could be used by future Mine Countermeasures Vessel personnel to help foster a strong spatial awareness of the “arrangement” of the seabed and various artefacts – targets, debris, wrecks, mines and the like – and to plan the deployment of appropriate countermeasures processes and assets, such as remotely operated or autonomous submersibles.

A process was developed to support the rapid conversion of FOST’s bathymetric data into a format suitable for real-time exploration using an appropriate games or VR engine. To do this, high-resolution greyscale height map images derived from Fledermaus bathymetric survey datasets (in this case data from surveys in Plymouth Sound) were acquired, and the brightness levels maps were used to generate a topographical map in 3D, using an industry-standard modelling package. This map was then converted into a polygonal representation, suitable for import into the games engine. This enabled the development of an early concept human-system interface, comprising a multi-window interface format depicting close-proximity seabed topography and simulated views from port-, front-, starboard- and downward-pointing virtual cameras onboard a deployed probe, or ROV, (the views being controlled using an Xbox hand controller). The © HIT Team 2020

Page | 3 demonstrator was further developed to show how UK Hydrographic Office Assets (Maritime Foundation Data), in this case a navigational chart of Plymouth Sound, could be digitised and overlaid onto the converted bathymetric scan data. It was also possible to vary the transparency of this digital chart, thus allowing the end user to visualise the seabed and any virtual artefacts that had been modelled in 3D – underwater sensor networks, wrecks, sealife colonies, and so on.

Page | 4

Submarine Training - SubSafe

SubSafe was an experimental interactive VR spatial awareness training tool, designed to investigate the replacement of legacy training media in use by Royal Navy submarine qualification (SMQ) instructors. Navigating the decks and compartments in a “first-person” mouse-and-keyboard game style, SMQ trainees had access to all decks forward of the control room, comprising over 30 compartments and 500 different objects – including fire extinguishers, hose reels, high-pressure air valves, and emergency breathing system masks. A statistical analysis of knowledge transfer data, collated following over a year of experimental trials with the RN’s Submarine School at HM Naval Base Devonport, revealed that use of SubSafe during classroom training significantly improved the final “walkthrough” examination scores of trainees onboard an actual submarine.

Interest in SubSafe has stimulated similar developments for the international submarine community, including virtual walkthroughs for the UK’s Astute Class, the Canadian Victoria Class and the Australian Collins Class. Digital SubSafe assets were also re-used to develop a 3D animation of the March 2007 HMS Tireless Self-Contained Oxygen Generator (SCOG) explosion incident for the Coroner’s Court of Inquiry.

HMS Courageous

Following on from research conducted in support of the SubSafe project described above, the HIT Team was given access to a restored Cold War nuclear submarine based at the Royal Navy’s Heritage Centre in Devonport, as part of a project to evaluate the use of digital media to provide visitor accessibility to an important naval heritage project. The boat in question, HMS Courageous, was a Churchill Class nuclear fleet submarine, launched in 1970 and commissioned with the Royal Navy from 1971. During her 21 years of service she was part of the Third Submarine Squadron, based at the Faslane Submarine Base in Scotland. As well as her role as a major North Atlantic surveillance asset during the Cold War, in 1982, to help retake the Falkland Islands from its Argentinean invaders, Courageous was despatched to support the British Task Force, including her sister ship, HMS Conqueror.

In her current location, within Her Majesty’s Naval Base at Devonport, visitors are still required to descend vertical ladders and negotiate narrow bulkhead hatches to reach the internal compartments. This means that the submarine is not accessible to the elderly (including the submarine’s surviving veteran crewmembers) and disabled. The HIT Team has been working with the Museum to develop an interactive exhibit, merging both Virtual Reality representations of the submarine and more detailed 360o panoramic images.

Whilst conducting a study for Flag Officer Submarines in 1999, it was concluded that a fully interactive 3D model of a submarine would require enormous manpower and computational resources, especially when one considers the massive amounts of detail evident from compartment to compartment and deck to deck. Instead, a solution was proposed that would present trainees with a simplified 3D layout of the vessel – decks, compartments and major items of equipment – but would combine this simplicity with detailed panoramic images taken inside each compartment onboard. Although this approach was not implemented in the final SubSafe system described earlier, it was resurrected as an appropriate and affordable alternative for the HMS Courageous “walkthrough”. In this way, it is possible for “virtual tourists” to understand exactly where they are on the submarine (in terms of compartments and decks) whilst being able to look around detailed 360o digital photographic images of their current location onboard. The 360o panoramic images were taken onboard in a wide range of locations, by attaching a Ricoh Theta camera at specific points using a combination of tripods, suction cup and magnetic mountings.

Page | 5

The Maria

In 1774, a 31-foot, 50-ton converted wooden sloop called Maria, purchased for £340, entered the history books as being host to the first of 65,000 recorded submariner fatalities. The Maria was equipped with a wooden-beamed chamber measuring some 12ft by 9ft by 8ft, containing around “75 hogsheads” of air (24.5m3, providing 25 tons of buoyancy and allegedly sufficient for 24 hours life support). A topside hatch allowed the vessel’s only occupant, a shipwright by the name of John Day, access to the vessel. For the Maria to surface after the experiment, netted groups of 21-ton rocks were to be released by undoing (and quickly plugging!) bolts within the chamber. The Maria was towed from Plymouth’s Sutton Harbour to Firestone Bay on June 20th 1774. Day boarded the Maria carrying a hammock, a watch, a candle, a bottle of water and some biscuits. The Maria “submerged” stern first and her disappearance was followed 15 minutes later by bubbles on the surface, indicative of the escape of a considerable amount of air from an underwater source. After 3 days of effort by 200 dockyard workers, the salvage operation was abandoned.

The A7 Submarine

The Royal Navy submarine HMS A7 sank off the Cornish coast in January 1914 with the loss of all eleven crewmembers onboard. Despite numerous salvage attempts, the A7 refused to move from her unusual situation, with a bow-up angle of around 30° and her stern embedded in over 20ft of soft clay, so it was decided the leave her where she lay. Listed in © HIT Team 2020

Page | 6

2001 as a “Controlled Site” under the terms of the UK Government’s Protection of Military Remains Act 1986, special permission was required from the Ministry of Defence to dive on the wrecksite. This was granted in 2014 and, in the summer months, a number of dives were made on the submarine by the Plymouth-based Promare SHIPS organisation, using an early VR model to plan activities and allocate areas of the vessel to individual divers, thus maximising their short time on the seabed, 135ft down.

The schematics, images and videos of the wreck collected during the dives supported a much more accurate model of the vessel, showing how much decimation has been caused over the years – not only by the harsh subsea conditions, but also as a result of human practices, such as random anchor dropping by fishing vessels and unauthorised dive boats. With the new VR model, features such as the bent periscope, the complete destruction of the “flying bridge” aft of the conning tower, damage to the torpedo tube doors, multiple holes across the hull (some containing very large specimens of conger eel!) and the remains of the hawsers used by the Royal Navy in their attempts to raise the submarine can be seen and explored.

The Amethyst

Although not a wreck, an important part of Plymouth Maritime History was sadly destroyed forever by the breaker’s yard in Sutton Harbour in 1957. HMS Amethyst, launched in 1943, was a “modified” Black Swan-class “sloop”. Whilst transiting the River Yangtse in April 1949, Chinese Communists mistook the ship as one belonging to the enemy Nationalists and opened fire, inflicting significant damage, grounding the ship and resulting in 19 fatalities (including the ship’s Commanding Officer, Lt Cdr Skinner). With rapidly dwindling food and fuel supplies, Commander John Kerans, the British Naval Attaché in China (played by Richard Todd in the film Yangtse Incident), arrived from Nanking and took command of the ship. On 30 July 1949, Kerans decided to make a night-time bid for escape. Once again the ship took heavy fire, but on 31 July, at 05:00, the Amethyst rendezvoused safely with the destroyer HMS Concord. On 19 January 1957, she was towed into Plymouth’s Sutton Harbour, coming to a final stop on Marrowbone Slip, where she was broken up. This Augmented Reality project was designed to visualise the Amethyst’s final days, transiting the River Tamar towards Devonport and returning her to where she was finally scrapped, next to the China House Pub and Restaurant (itself an historic building, dating back to the 1600s), and to draw attention to what must have been a spectacular sight – a 1350-ton, 283-foot long Frigate laying silent in a small harbour which was, at the time, more used to welcoming fishing trawlers and sailing ships.

The GLAUCUS

The UK’s first underwater habitat, GLAUCUS, took the form of a 2-ton, 3.7m long and 2.1m diameter cylinder mounted onto a four-legged platform and was, in September 1965, anchored 100m off the Breakwater in Plymouth Sound at a depth of 10.7m, (not far from where its rusting remains still lie today). One of the original aims of the GLAUCUS project was to demonstrate that living underwater could be achieved on a much smaller budget (£1000!) than that being © HIT Team 2020

Page | 7 expended at the time by Jacques Cousteau’s Conshelf project team or the US SEALAB I/II programme. Air was provided by a closed circuit air tank system (GLAUCUS was the first subsea habitat in the world to demonstrate this), with carbon dioxide scrubbing (soda lime) trays mounted within the structure. The “Virtual GLAUCUS” began life as an Undergraduate project, with important contributions from Colin Irwin, one of the two divers who made history by spending a week living within the tiny structure.

Virtual Wembury

Healthcare research undertaken by the HIT Team is addressing how to exploit VR recreations of areas of natural beauty to help patients recover from traumatic incidents (including operations) and improve the well-being of others, from settings as diverse as Intensive Care Units to elderly care homes. Research conducted since the late 1970s suggests that exposing individuals to natural environments, such as rural and coastal settings and smaller-scale urban areas with natural features (gardens, parks, etc.) can promote stress reduction, enhance mental recovery following tasks requiring high levels of attention, even reduce post-operative recovery times and the need for pharmaceutical pain relief.

These “restorative environments” are now recognised as powerful tools in the treatment of a range of psychological conditions and a number of hospital-based projects are being conducted to encourage engagement with the natural environment to promote both psychological well-being and physical recovery. In collaboration with the Royal Centre for Defence Medicine (RCDM) and the Queen Elizabeth Hospital in Birmingham, the Virtual Restorative Environment Therapy (VRET) project aims to exploit simulated restorative environments to deliver similar benefits to those individuals who are unable to access and experience real natural environments. This research, which includes developing dynamic participative activities to support post-operative recovery of lung function and lower limb rehabilitation, is based on the virtual recreation of a popular West Country location – Wembury Bay. Virtual Wembury has also been used at special military and space research locations in the Arctic (CFS Alert) and Hawaii (HiSEAS) as part of experiments to assess the role of virtual environments in helping to sustain mental well-being for individuals located in remote, inhospitable environments.

Wembury … The Dock That Never Was

Another application for Virtual Wembury turns the clock back to 1909, when, had a proposal put forward to the House of Lords been successful, a huge passenger and commercial port would have been built within the Bay, rivalling the docks at Liverpool and Southampton. The port was to comprise extensive breakwaters, four jetties, dry docks and even a railway terminus. Fortunately, the House of Lords rejected the scheme, but VR and AR techniques have been used to bring the original proposal “to life”, helping local villagers and schoolchildren to understand the impact the dock would © HIT Team 2020

Page | 8 have had on the area, had the proposal been accepted. Given the paucity of information relating to the Dock, historical research had to be conducted using references to other UK docks, including Liverpool/Birkenhead, Southampton and the Port of London, together with Hull, Cardiff, Falmouth and Bristol, where the current Heritage Dock exhibits proved to be particularly useful. This research enabled the Team to build up a concept design featuring the main elements of the Wembury Dock and to select appropriate pre-built 3D models from established Internet sources. Given the enormity of the project, it was only possible to include a small number of features at this stage and it was important to focus efforts on the technical challenges posed by the AR software. Consequently, a large number of 3D historical models were purchased, with the aim of reflecting a range of historical buildings and vessels, ready for the real-time AR visualisation trials on the coast at Wembury.

Virtual Burrator

Virtual Burrator, like its Virtual Wembury counterpart, has its roots in a hospital rehabilitation research project, focusing on the development of virtual restorative environments and their potential effect on patients’ sense of well-being and post-operative rehabilitation. During the early investigations of the Wembury and Burrator areas, a number of interesting historical facts and sites came to the attention of the HIT Team researchers, not to mention recollections and narrative from local community members. In the case of the early Burrator surveys, it was decided to change the emphasis of the research away from healthcare and focus more on examples of Virtual Heritage instead.

Between the 16th and 19th Centuries, supplying fresh water to the city of Plymouth and the naval base at Devonport, was a continual challenge. Despite the construction of two granite-lined water channels called leats – the Plymouth (or Drake’s) Leat in 1591 and Devonport Leat in 1801 – the heavy demand for water and problems caused by pollution and dense snowfalls on Dartmoor meant that a more reliable and permanent solution had to be devised. In 1893, Plymouth Corporation authorised the construction of a reservoir at Burrator Gorge. Costing £178,000 and comprising two dams (Burrator, the largest, and Sheepstor, just to the east), the building works for Burrator Reservoir were completed in 5 years and the site was officially opened on 21st September, 1898.

Burrator & Sheepstor Halt. As part of the research project to recreate the industrial heritage evident around the Burrator and Sheepstor area, one area of particular interest is the Yelverton-to-Princetown Railway at Burrator & Sheepstor Halt, an isolated station which opened in 1924, and was used to disembark construction teams for the Burrator Reservoir dam raising project. The Halt also witnessed the transfer of French and American prisoners to Princetown Prison and the transport of Dartmoor granite for use in the building of such national icons as London Bridge and Nelson’s Column. The line closed in March 1956. Today the remnants of the Halt can be explored, courtesy of the Tyrwhitt Trail cycle path, but all that remains are the two “kissing” gates and associated steps giving access to the Reservoir and local moorland, isolated fence posts and the concrete/brick shelter foundation. This small historical area has been recreated in VR, complete with a mixed passenger-freight train and a petrol-driven “Wickham Trolley”.

Page | 9

Burrator: What Lies Beneath. In 2016, the HIT Team joined forces with Truro-based Swathe Services and Plymouth- based MSUBS to undertake the first ever high-tech underwater survey of the Reservoir. A Multi-purpose Unmanned Surface Vessel (MUSV), in effect a robot catamaran equipped with a high-definition multibeam sonar system, was used to conduct an autonomous survey the bed of the man-made lake in order to discover and map out a range of natural and historical features. As well as generating a detailed virtual topography of the underwater landscape, the team was able to trace man-made features that existed long before the valley was flooded in the 1890s, including the path of Drake’s Leat, the old road to Sheepstor Village over Sheepstor Bridge, Essworthy Farm, the deep scarring caused by the quarrying for stone to build the Reservoir Dam and evidence of walled areas associated with the 15th Century estate of Longstone Manor, the above-water remains of which have already been surveyed and modelled in 3D by the University Team using drone video-to-3D conversion technologies. The results of the high-tech underwater survey were also used early in 2017 to help the Devon & Cornwall Police Diving Team in their search for a missing 30-year-old man. Screen grabs of the virtual search areas were provided to the Police in order to provide the divers with an appreciation of the underwater environment and any obstacles or deep gullies they may encounter during the dives.

Page | 10

More recently, closer inspection of the 3D reconstruction of the sonar data from the 2016 survey indicated the existence a number of smooth, regularly-spaced underwater objects in the vicinity of the surface remains of a suspension bridge, constructed in 1923 during work to raise the main dam to provide extra water for the growing city of Plymouth. Further investigation, using photographs taken by local villagers during the drought of 1959, indicated the presence of large spherical objects on the shoreline, resting on what appeared to be chain-like meshes, with one of the spheres still afloat. These were later confirmed to be remains of a WWII “T-class” anti-torpedo net, comprising large flotation buoys and a net made up of interconnecting metal-wound rings, or “grommets”.

It is known that the German Luftwaffe conducted aerial photographic surveys of the Reservoir in 1941, presumably with the aim of torpedoing the dam at some future point, but the imagery was not of a resolution to see anything resting on the surface of the water.

Armed with this information, permission was sought and granted from South West Water and South West Lakes to dive two small remotely operated ROVs, a BlueROV2 and a Trident “underwater drone” (a smaller but highly agile submersible), into the Reservoir (taking advantage of it being at 50% capacity due to the 2018 summer drought), in order to discover if the remains of the nets still existed. On both dives and, despite the very murky conditions of the water, exacerbated by the ROV thrusters when approaching the bottom, some impressive images and video footage of the spheres and the grommets were acquired, indicating that, despite the decades of immersion, many of the structures were in remarkably good condition. © HIT Team 2020

Page | 11

More recently, in 2020, a more capable ROV was deployed into the Reservoir to continue the challenge to find out how extensive the remains of the spheres and nets actually are. Do they, for instance, still extend across the width of the lake at that point, as suggested in the VR conversion of the sonar data? Unfortunately, the underwater visibility was very poor on the day of the dive, but, thanks to the ROV’s onboard sonar system, three further spheres were located at a greater distance and at a much greater depth than those described above, with two of the three lying one above the other.

RAF Harrowbeer, Yelverton. Close to Burrator is the abandoned RAF airfield at Harrowbeer, part of the village of Yelverton (the western terminus for the old GWR Princetown Railway), about 2.5 miles west of Burrator Reservoir. The airfield (called Harrowbeer to avoid potential confusion between ‘Yelverton’ and the Royal Navy Air Station at ‘Yeovilton’ in Somerset) officially opened on 15th August 1941 and remained operational throughout World War II, under the control of 10 Group Fighter Command. It was also Headquarters to the first Air Sea Rescue Squadron. Constructed using rubble from the devastating air raids on Plymouth of March and April 1941, RAF Harrowbeer was used solely as a fighter base, although it did witness a large variety of aircraft during its existence, including Lysanders, Typhoons, Whirlwinds and Spitfires (Spitfires were used to provide enhanced protection for Channel convoys). In September 1944, the base was also visited by Glenn Miller and his orchestra, en route to concerts in Plymouth.

Page | 12

In 1945, US President Harry Truman’s flight – a Douglas VC-54C named the Sacred Cow (the forerunner of Air Force One) – was diverted from RAF St Mawgan into Harrowbeer due to fog. The aircraft was returning the President back to the US after attending the Potsdam Conference (where the victorious Allies attempted to tackle issues arising from the future split of democracy and communism in Europe).

All that now remains of RAF Harrowbeer are the foundation stones relating to accommodation, aircraft hangar and support buildings, some remains of the three (‘A’ layout) runways and the original Watch Office – now the Knightstone Tea Rooms. However, the distinct dispersal pens can still be seen around the periphery of what was the airfield. One of these pens provided the backdrop for a 2013 Augmented Reality Spitfire and Dakota DC3, with an additional AR sequence depicting the virtual Lancaster flying over Burrator Reservoir and Dam. More recently, individual buildings on the old airfield have been modelled using SketchUp, including the WWII Watch Tower. A further demonstration of Augmented Reality took place at an RAF Harrowbeer “1940s Weekend” in 2015, by bringing President Truman and his famous Sacred Cow back to Devon one last time.

Drones Over Plymouth & Dartmoor

The HIT Team is constantly experimenting with new technologies to support field surveys of complex heritage sites, especially where they are difficult to access due to plant growth, hazardous terrain or protective measures. The use of small unmanned air systems (sUAVs) or “drones” and other remotely operated devices is delivering an invaluable capability in not only the collection of data relevant to the future construction of Virtual Heritage sites, but in their subsequent visualisation, using remote AR techniques.

Longstone Manor. Longstone Manor was a 16th Century Manor House that was finally abandoned in the late 1800s, when Burrator Reservoir was completed. Today, all that remains are some of the main walls of the main manor house and those of adjacent buildings. Using drone technologies to gain overhead views of this protected site has been highly useful on helping the ongoing construction of a VR representation of what the Longstone Estate may once have looked like. The use of Pix4D has also been instrumental in saving time during the Virtual Longstone construction process (see Hooe Lake Wrecks, below).

Page | 13

Foggintor & Haytor Quarries. At the time, the aerial survey and subsequent 3D recreation of the historic Foggintor Quarry on Dartmoor, between Tavistock and Princetown was considered to be the biggest challenge for the HIT Team’s DJI Inspire 1 drone. Foggintor was one of the three great granite quarries of Dartmoor (the other being Haytor – see below – and Merrivale), its material output being transported using the local GWR railway line connecting Princetown and Yelverton with Plymouth and the rest of the UK. During the 1840s some 300 workers were employed at the quarry, many living in specially built cottages nearby, and these included blacksmiths, carpenters, stone masons, horsemen and others. Numerous buildings also existed (the remains of which can still be seen today), including a foundry, a smithy, stables, office and tool stores, a manger’s house and, later, a chapel. Although the quarry ceased operations in 1906, the granite produced was used in many famous buildings, including Nelson’s Column in London and the old London Bridge. Today, Foggintor is a popular hiking and climbing destination, the quarry remains comprising cliffs and small chasms, some over 100ft, and a deep central pool.

The Pix4D reconstruction of this large site was achieved, remarkably, following a single (one-battery) drone flight lasting approximately 15 minutes!

Page | 14

More recently, and with the support and permission of the Dartmoor National Park Authority, a similar exercise has been undertaken over the long-abandoned granite quarries of Haytor, an area described by many as Dartmoor’s most famous landmark. This project, aims to recreate the quarry scenes of the 19th Century, including the famous tramway, which opened in 1820 to transport granite for the construction of London Bridge and has been described as the first “railway” in Devon (although its “rails” were built using flanged granite slabs, due to the lack of local sources of iron).

During later flights over the area to collect data missed in earlier sessions, the opportunity was taken to carry out a special survey in support of one of the University’s MSc project students. This survey was designed to record differences in gamma radiation emitted from radon around the “weather-exposed” rocks of the tor in contrast to the “granite-incised” quarry areas. In addition to using the Pix4D-reconstructed data, integrated with a lower-resolution topographical model of the region (visualised using Unity), a separate series of flights was undertaken over the tor and quarry regions, using a DJI Matrice 100 drone. The Matrice was equipped with a Safecast bGeigie nano, a mobile, GPS-enabled radiation sensor, originally developed as a result of the lack of radiation data available immediately after the earthquake and tsunami events of 2011 that led to the Fukushima Daiichi power plant meltdown. The results showed that, as hypothesised, there was a significant difference between the gamma radiation levels recorded over the main tor and those captured over the quarry, with the quarry levels being higher than the tor (possibly due to “pooling” of the gas in the locations more protected from the weather).

Hooe Lake Wrecks. Two wrecks in particular were the focus of this particular Virtual Heritage exercise. The first involved the skeletal remains of the Arthur, a wooden “lighter” (in effect, a flat-bottomed, non-powered barge). This wreck exists, surrounded by deep mud, approximately 100m from the eastern shore of the Lake. The Arthur was

Page | 15 owned by local timber importers and was operated under tow by a small steam tug. The second wreck, the remains of the Roger (or Rodger), exists on the southern shoreline, in a state of considerable decay. According to historians, the Roger was originally a 60-foot, 27-ton trawler built in Nieuport, Belgium in 1940s. The ship was sold to a Brixham-based fisherman in 1966, a Plymouth-based fisherman in 1974, and then to a yachtsman who defaulted on payment to a local boatyard for modifications undertaken. She was finally sold on for conversion into a houseboat. The lower part of the hull can still be seen with ribs intact, interspersed with concrete ballast. Remains of the rudder lie nearby.

The 3D reconstruction of two of the wrecks was undertaken in collaboration with the Promare SHIPS organisation, partly as an exercise to evaluate a commercially available drone video data analysis package, Pix4D, and its capability to support maritime archaeological studies, especially in cases where wrecksites may be inaccessible due to treacherous mud flats or dangerous tidal conditions. Pix4D is a professional mapping toolkit, capable of producing impressive geo- referenced maps, mosaics, and 3D models purely from images captured during manually-controlled or pre-programmed drone flights. In the case of the Hooe Lake exercise, a DJI Inspire 1 drone was flown over the two wrecks and the video collected was analysed off-line a day or two after the survey, producing two very impressive 3D reconstructions.

Other drone-based projects undertaken in the Plymouth and Dartmoor areas and even further west are too many to mention on a case by case basis, but just some of the highlights from the past few years are illustrated below and include filming a US special forces minisub undergoing its final trials in Cattedown; the search around Dartmoor for two Westland Whirlwind fighter aircraft lost on 29 December 1940; University students’ submersible technology experiments around Plymouth Sound’s Breakwater and Fort; Pix4D and thermal imagery surveys around the Goonhilly Earth Station and the Eden Project; short heritage flights around Wheal Betsy, the Wheal Martyn China Clay Museum, and Brent Tor with St Michael’s Church; the lost village of Hallsands on the South Devon coast; and research-supporting flights around Burrator and Wembury Bay.

Page | 16

3D Scanning Projects

In order to complement the work undertaken using drone and submersible systems, the HIT Team has also been making use to the latest generation of 3D scanning systems, based on Matterport Pro 2 Scanner is a tripod-mounted “structured light” (infrared) 3D sensor that scans an area in 20 second-sweeps, transferring the scan data to an iOS Smartphone or iPad. The data comprise both time of flight range data, plus 4K imagery, enabling users to very quickly develop a virtual “walkthrough” of a site or object by positioning the camera at different “nodes” throughout that site. Another recent addition to the team’s “toolkit” is the latest version for the Ricoh Theta 360 camera series, the Z1, which, as well as 360 spherical images and videos, performs a similar function to the Matterport, but is substantially smaller and cheaper. Some of the examples of projects using these systems are:

 Drake’s Island Tunnels – https://my.matterport.com/show/?m=SH5rxve8Mh2  Plymouth Barbican Area – Mayflower Steps: https://my.matterport.com/show/?m=7uXhTqAaSrB

Jacka Bakery and Southside St: https://my.matterport.com/show/?m=zB5MvGvZorM The Parade: https://my.matterport.com/show/?m=6F4WyoCpBRD © HIT Team 2020

Page | 17

 Plymouth Castle Remains – https://my.matterport.com/show/?m=uuEqWr1fgQB  Cattewater Harbour Commissioners’ Office Board Room – https://my.matterport.com/show/?m=1ps8WVP7uya  Burrator Dam Water Testing Station – https://my.matterport.com/show/?m=oBUkpGFYS5e

Page | 18

Medical Emergency Response Team Trainer

A typical military Medical Emergency Response Team (MERT) consists of a number of highly trained medical personnel, including emergency medicine or anaesthesia consultants, emergency nurses and paramedics.

The role of the MERT is to deploy to incidents on the battlefield using air, land and sea assets, and to provide pre-hospital, life-saving trauma care to the seriously injured whilst in transit to a more comprehensively equipped medical facility. The HIT Team has, under sponsorship from the UK’s Joint Medical Command and the Royal Centre for Defence Medicine, been conducting research into the potential for interactive technologies generally, and Mixed Reality (MR) in particular, to provide a more representative form of training for MERT personnel, prior to their deployment on actual casualty evacuation missions.

In brief, the HIT Team’s MR Trainer consists of an inflatable enclosure, within which it is possible to house a variety of physical objects, including a physical mannequin representative of a military casualty, thus recreating the cramped and cluttered environment typically associated with a MERT operation as might be found onboard, for example, a Chinook Helicopter. Blue-screen chroma key techniques enable the trainees to see – and touch with their own hands (avoiding the use for immature and costly “haptic” feedback devices) those physical objects that have been identified as important to the training process. To provide visual in-flight and waterborne motion cues for the aircraft and amphibious vessels selected for inclusion in the MERT VR simulation, it was decided to use video effects, as opposed to the more expensive and time- consuming process that would be required to model virtual environments external to the main cabin areas.

This is once again where Plymouth and its surrounding rural and maritime scenes played an important role. For example, in the case of the view from the rear of the simulated Chinook cabin, this was obtained by flying one of the HIT Team’s drones (a DJI Inspire 1) backwards across the barren region of Foxtor Mire, near Princetown on Dartmoor. The video was then textured onto a 2D plane, or “billboard”, located a few metres in virtual space beyond the region of the virtual Chinook’s ramp. The addition of particle effects to simulate engine exhaust, or the aircraft passing through light cloud, adds to the illusion.

In addition to the in-flight effects for the Chinook, a similar approach was adopted for the Royal Marines’ Landing Craft and Griffon Hovercraft. With the invaluable support of 10 Training Squadron 1 Assault Group Royal Marines, based at Her Majesty’s Naval Base in Devonport, the HIT Team researchers deployed a combination of waterproofed GoPro cameras and one or two 360 (panoramic) cameras on the craft themselves, recording high-definition video footage of transits from the northern area of the Naval Base, along the River Tamar and its estuary, out across Plymouth Sound and past the City’s famous Breakwater, finally beaching at the small beaches at Barn Pool and Cawsand.

Pengelly Cave Studies Trust

In September 2015, fossilised bones belonging to 120,000-year-old elephants, hippopotami, lions, bears, bison and deer were stolen from the Pengelly Trust’s Joint Mitnor Cave, located near the small Devon town of Buckfastleigh. It was thought that the thefts were part of a targeted operation to satisfy the demands of black market collectors. As well as the stolen fossils, other bones were dislodged from the walls and floor of the cave, causing substantial damage to the site’s main demonstration area.

Page | 19

Converting the scan data provided by the Department of Earth Science at the Natural History Museum, not only has it been possible to develop 3D datasets of the eleven stolen fossils suitable for 3D printing technologies, it has also enabled the development of a unique Virtual Reality “catalogue”, enabling users to select any of the eleven fossil records provided, to manipulate them in real time, and to change the projected lighting effects to help with interpretation. These 3D fossil recreations have also been located in a virtual cave scenario. The virtual cave, compete with flying bats and the sounds of distant wolves and elephants, enables schoolchildren to take part in a “virtual fossil hunt”, exploring the scene, searching for and identifying the types of bones prior to seeing their detailed virtual representations in the VR catalogues and the physical facsimiles in their accurate settings.

The Virtual Mayflower (www.1620Mayflower.co.uk)

The HIT Team’s original vision for the Virtual Mayflower project centred on the development of a series of interactive maritime and colonisation heritage scenarios, based on VR and AR technologies, suitable for presentation to a wide range of end users at a similarly wide range of locations – museums/libraries, schools, restaurants/public houses, quaysides, even in home settings. These scenarios were to be designed to transport the end users back in time to experience the construction, layout, accommodation, “facilities” and resources onboard the Mayflower, and to support meaningful, educational interactions with avatar representations of the crew and passengers. In the event, no funding was ever forthcoming for the project, but the team managed to continue unperturbed, to deliver a stunning recreation of both the Sutton Pool area of Plymouth in the 1600s (today, the Barbican) and a short transit from the Harbour, via a shallop, out to the Mayflower herself, complete with basic avatar representations of passengers and crew.

Many important achievements and successes were evident throughout the six years that it took for the final result to be launched in 2020 – on the actual date the Pilgrims set sail from Plymouth 400 years ago – 16th September 1620. These ranged from to spending time onboard the Mayflower II in Plymouth Massachusetts and at the nearby recreation of Plimoth Plantation, to capturing the movements of a crewmember scaling the heights of the Jubilee Sailing Trust’s tall ship, the Lord Nelson, moored at London’s Canary Wharf whilst wearing a special motion capture (MOCAP) suit – the Perception Neuron.

Early trials using Augmented Reality were also undertaken, using the Vuforia software development toolkit plugin for Unity. Three locations were chosen for the AR demos – two on Plymouth Hoe (underneath the Citadel, looking towards the Tinside Lido (an outdoor swimming pool); and on West Hoe Pier), and one at Devil’s Point looking across Firestone Bay towards Drake’s Island.

Page | 20

The HIT Team worked with a range of partner organisations on the project, including Royal Leamington Spa College, responsible for developing the virtual Pilgrims – Barbican inhabitants, plus the passengers and crew, and Plymouth- based Bluestone 360, who kindly developed the Virtual Mayflower website, charting the project story (www.1620Mayflower.co.uk). Digital artists and VR specialists Robert Guest and Chris Harvey coordinated the development of the 3D assets.

Page | 21

Mayflower Autonomous Ship

In October 2016, Plymouth-based MSUBS Ltd (and the not-for-profit corporation Promare) agreed to team up with the HIT Team, to investigate how VR and AR could be exploited in the development of the Mayflower Autonomous Ship (MAS), an unmanned surface craft that was to be designed to cross the Atlantic in 2020 (now scheduled for 2021), commemorating the Pilgrims’ pioneering journey 400 years ago.

In the four years since that agreement, the MAS design has changed significantly, but, thanks to the existence of comprehensive computer-aided design (CAD) models of the evolving vessel, it has been possible to support MSUBS in illustrating its main functions and onboard systems, both for public engagement and technical conferences. In addition, the VR recreations made use of the original Mayflower and simple harbour scenario, allowing users to compare the “then and now” and giving them the capability to venture onboard both ships. Some of the Team’s VR efforts were even shown on one of the large-screen displays in Times Square in New York during October 2019.

Mixed Reality Science Station

The collaboration with MSUBS/Promare also inspired HIT Team researchers to investigate the role that VR/AR might play in the future of autonomous maritime systems, especially with regard to human supervisory control and situational awareness. To achieve this, the HIT Team researchers exploited the results of a number of succesful “Mixed Reality” (MR) projects developed during 2015 and 2016 for defence command and control applications. MR is a term that is increasingly being used to describe a form of AR; however, rather than simply superimposing computer-generated material onto real- time images of the world, MR attempts to exploit the existence of real-world objects in order to enhance the believability, and indeed usability, of those computer-generated elements. Such objects can be as basic as tables, boxes, wall- or ceiling-mounted frames, even complete rooms or temporarily-erected enclosures. The successful defence demonstrator projects have enabled the HIT Team to propose a similar concept for a future

Page | 22 maritime autonomous system command/science station. The concept is based on a central navigational display, depicting the current location of the vessel at sea, surrounded by user-selectable information “windows”. The simulated graphical views of data being sensed and collated by the vessel’s onboard sensors (e.g. vessel speed and direction, wind speed, wave motion, sea and air temperature, onboard power resources, etc.), together with a “panoramic” video would be presented in as close to real-time as the satellite communications links will permit.

In August of 2017, a successful demonstration trial illustrating the main features of the above concept was undertaken, involving a small radio-controlled catamaran deployed in Plymouth Sound, relaying video and sensor data back to the HIT Team’s MR “Command Table” located in Birmingham. The catamaran – in essence a modified “bait boat” – was equipped with multiple cameras, including a gimbal-stabilised GoPro module and an onboard microcomputer collecting and transmitting data from GPS, temperature, pressure and humidity sensors. These data were transmitted locally to a receding unit and laptop onboard a nearby support boat chartered for the purposes of the trial and then uploaded via WiFi to YouTube, where the live video stream was then processed using a software tool called ManyCam. This tool converted the streaming video into a webcam stream and then delivered that stream to the HIT Team’s real-time VR rendering, or “game” engine, Unity. This was then mapped to a “render texture” which enabled the stream to be displayed onto one or more 2D planes, appearing within the field of view of the end user, in this case equipped with an HTC Vive head-mounted display.