for Integration of small Unmanned Aircraft Systems (s-UAS) in National Airspace System Matthew Dechering Manish Kumar Contents

1. Introduction 2. Survey 3. Existing Solutions 4. Operational Requirements for Urban Air Mobility 5. Solutions for Urban Air Mobility 6. Ongoing/Future Work 7. Conclusions Introduction: Small Unmanned Aircraft Systems (s-UAS) • s-UAS have generated a lot of interest in civilian domains: – Emergency Management, Law Enforcement, Infrastructure Inspection, Package Delivery, Imaging/surveillance • The FAA expects between 162% and 432% growth in number of unmanned flights by 2021 • Low end estimate of 2.75 Million units in the air by 2021, up from 1.10 million units Concept – The UTM Problem • A futuristic notional scenario of UAS usage in National Airspace System consists of a large number of UAS operating in crowded airspace – Safety and reliability issues in autonomous operations – Beyond Visual Line of Sight (BVLOS) operations risky • Objectives of UTM: maintain safe separation with other manned/ unmanned aircraft to avoid collisions while fulfilling UAS mission • Challenges: involves integration and development in several technological areas: – Computation – algorithms for collision-free path planning, tracking of UAS – Sensing - onboard and off-board to obtain situational picture of environment Source: FAA drone vision- http://www.airtrafficmanagement.net/ – Communication – enable information sharing

4 Concept – UTM Requirements

• Mission planning – Allows users to request, specify and modify missions of multiple UAS

– Keeps track of all UAS in the airspace / front end for mission – Obtain collision-free and optimized flight planning path planning considering all UAS in the airspace • Detect-and-Avoid (DAA) – Have the ability to detect and track other UAS and structure in the airspace Path planning in an urban airspace – Onboard/ground-based sensors or ADS-B – Ability to modify the flight of UAS to avoid the collision in a local manner

Detect and Avoid (DAA) - https://vigilantaerospace.com

5 Study Goals

• Study advances, identify potential issues, and propose new solutions • Focus on Sense and Avoid (SAA) and Unmanned Aircraft Systems Traffic Management (UTM) aspects of s-UAS integration in National Airspace System Specific Aims of the Project

• Specific Aim # 1: Survey of Existing and Emerging Technologies to determine near-term capabilities • Specific Aim # 2: Comparative study of existing solutions proposed by industry, academia, and government • Specific Aim # 3: Development of operational requirements for UTM • Specific Aim # 4: Solutions for these operational requirements Specific Aim # 1: Technology Survey

• Sensing Devices • Onboard Computing • ADS-B and GPS • Communication Specific Aim # 1: Sensing Devices-LIDAR

Product Price Type Weight (g) Range (m) Power (W) Hokuyo 3D-LIDAR YVT-X002 $4,825 3D Scan 750 50 8.4 Hokuyo UST-10LX $1,700 Planar Scan 130 30 3.6 • LIDAR is commonly used for its Hokuyo UTM-30LX $4,800 Planar Scan 210 30 8.4 Hokuyo UTM-30LX-EW $5,290 Planar Scan 210 30 8.4 low cost and low SWAP (Size, Hokuyo UTM-30LX-F $5,000 Planar Scan 210 30 8.4 Hokuyo UXM-30LX-EW $5,165 Planar Scan 800 30 7.2 Hokuyo UXM-30LXH-EWA $5,875 Planar Scan 1200 80 7.2 Weight, and Power) compared to Ibeo LUX $20,000 Planar Scan 900 200 10 Ibeo LUX 8L $28,899 Planar Scan 1000 200 10 other sensing technologies. Ibeo LUX HD $21,599 Planar Scan 1000 120 10 Ibeo miniLUX $20,000 Planar Scan 450 40 7 lightware SF40/C $999 Planar Scan 229 100 4.5 • More expensive 3D scanning Quanergy M8 $1,000 3D Scan 800 150 15 Quanergy M8-1 $6,100 3D Scan 900 200 18 Quanergy S3 $250 3D Scan Unknown 150 Unknown LIDAR has shown success in Quanergy S3-Qi $1,200 3D Scan Unknown 150 Unknown RIEGL VQ-480-U Unknown Planar Scan 7500 950 55 autonomous vehicles on the ground RIEGL VUX-1UAV Unknown Planar Scan 3750 920 60 Scanse Sweep $349 Planar Scan 120 40 3.25 and in the air. Spectrolab SpectroScan 3D Unknown 3D Scan 2018 20 30 Velodyne HDL-32E $25,000 3D Scan 2000 100 12 Velodyne Puck Hi-res $8,000 3D Scan 830 100 8 Velodyne Puck LITE $8,000 3D Scan 590 100 8 Velodyne PUCK VLP-16 $8,000 3D Scan 830 100 8 Specific Aim # 1: Sensing Devices-RADAR

• Typically more expensive Update Name Supplier Range Accuracy Weight Dimensions Power rate FOV 11.4 x 8.7 x DK-sR-1200e lmst 307 m 0.6 m 280 g 4.25 cm 4.5 W 10-200Hz 65 x 24 ° and higher SWAP than its 7.6 x 5.4 x 1.3 muSharp Aerotenna 120 m 0.22 m 43 g cm 1.25 W 90 Hz 50 x 30 ° 18.7 x 12.1 x 4 120 x 80 LIDAR counterparts MESA-DAA Echodyne 750 m 3.25 m 817 g cm 35 W 1 Hz ° 20.3 x 16.3 x 4 120 x 80 MESA SSR Echodyne 750 m 3.25 m 1250 g cm 45 W 2 Hz ° integrated 13 x 10 x 17.5 • Still a new area the IRIS Sensor 66 m 1.24 m 360 g cm 4.5 W 3.4 Hz 90 ° 12.7 x 20.32 x 120 ° x DAA-R20 Fortem 1500 m 0.0508 m 464 g 2 cm <60 W 8 Hz 40 ° companies are exploring 11.5 x 11.5 x muSharp 360 Aerotenna 40 m 0.22 m 243 g 6.7 cm 2.5 W 80 Hz 360 ° Specific Aim # 1: Sensing Devices-Cameras

Name Supplier resolution and frame rate power SENSOR FLIR DUO Dual Sensor Uncooled VOx FLIR Systems 1920 X1080 2.2 W 5-26 VDC Thermal Camera Microbolometer, • The most Diverse group of DJI Zenmuse X5S DJI 20.8 MP CMOS 4/3" Camera 3280 x 2464 3.7V DC Sony CCD Edmund 56-578 edmund 768 x 492 12V DC @ 130 mA Interlaced CCD sensors by far HackHd 1080p -- 4000 x 2250 3.7V DC @ 500 mA Interlaced CCD PointGrey BlackFly model Point Grey 1280 x 1024 @ 60 FPS 5V / 380mA BFLY-U3-20S4C-CS PointGrey Flea3 model FL3- Point Grey 1624 x 1224 @ 15 FPS 5V / 380mA • Conventional Cameras need U3-13E4C-C e-con Systems' 4224 x 3156 @ 18 FPS, 1280 x 1080 @ E-con 5V / 380mA CMOS Image See3CAM_CU130 45 FPS binocular vision to sense e-con Systems' E-con 1920 x 1080 @ 42 FPS 5V / 380mA CMOS Image See3CAM_CU30 IDS uEye cameras UI- IDS (1280x1024, 60fps, 8bit mono) 5V / 380mA CMOS Image distance. 3241LE PointGrey Blackfly GigE PoE Power over Ethernet color camera with CS-mount Point Grey (1280x1024, 60fps, 8bit mono) (PoE); or 12 V CMOS Image • Stronger in bright conditions, lens and Global Shutter nominal (5 - 16 V) LI-USB30-IMX185 2.42M 2.42M pixels CMOS Resolution: 1952H x 1241V USB 3.0 Camera Sensor where radar and lidar can 1080p/29.97 mode to 720p/59.94, FCBEH3300 Sony 1,450,000 pixel 20x Zoom HD Color CMOS Image Block Camera, image stabilization struggle Color Camera Module Sony 6 to 12 V DC/ 3.0 W CCD FCB-EX1020/EX1020P LI-M034USB3-AF 720p WDR USB 3.0 Camera with 18x Aptina MT9M034 1.2M pixels Sensor USB 3.0 +5VDC Zoom Lens Hero4 Session goPro 1440p30 1080p60 720p100 480p120 Hero4 Silver goPro 4k15 2.7k30 1080p60 720p120 480p240 Specific Aim #1: Computing Devices

Open Waypoint ADS-B Board Company Source Navigation Compatibility A2 DJI No Yes Yes AeroQuad v2.2 Kit AeroQuad Yes Optional AfroFlight Naze32 Rev6 Acro HobbyKing APM 2.8 3DR Yes Yes Yes • There are a large amount of AutoQuad v6.6 Viacopter/FlyDuino Yes Yes Crius All In One PRO Crius/Hobbyking Yes Optional Crius MultiWii Lite Crius/Hobbyking Yes Crius MultiWii SE Crius/Hobbyking Yes Optional autopilots that can boast high DJI Naza-M V2 DJI No Yes Yes DJI Wookong DJI No Yes Yes Erle Brain 3 erlerobotics Yes Yes Yes Erle-Brain PRO erlerobotics Yes Yes Yes FF Auto Balance Controller Free Flight No accuracy with a proper GPS FY-40A Feiyu Tech No FY-41AP Feiyu Tech Optional FY-41AP Lite Feiyu Tech FY-DOS Feiyu Tech connection HobbyKing KK2.1HC HobbyKing Yes HoverflyPRO Hoverfly Optional iNav Sirius™ AIR3 F3 SPI MultiWiiCopter Yes Add-on Intel Aero Intel Yes Yes Yes • Some are capable of intelligently LibrePilot CopterControl/Atom LibrePilot Yes Platform LibrePilot Revolution LibrePilot Yes MikroKopter Flight-Ctrl ME 2.1 MiKroKopter Yes avoiding obstacles at low altitude Complete Navio2 emlid Yes Yes Yes Naza-M Lite DJI No Optional Yes Panda2 Feiyu Tech Yes • ADS-B connectivity is common Pixfalcon holybro Yes Yes Yes Pixhawk 2.1 Cube proficnc Yes Yes Yes Pixhawk Mini 3DR Yes Yes Yes Pixracer R14 mRobotics Yes Yes Yes but not ubiquitous RVOSD 6 Range Video Yes SmartAP 3.0 Pro SmartAP Yes SmartAP 4 Set SmartAP Yes Yes Snapdragon Flight Autopilot Intrinsyc Yes Yes Yes UAVX-ARM32 Full Sensors QuadroUFO Yes Specific Aim #1: ADS-B and GPS

Input ADS-B ADS-B Internal • Low SWAP ADS-B and GPS have a few Product Refernce Power (W) Weight (g) in out Size (mm) GPS products on the market ping2020 uAvionix 0.5 20 yes yes 25 x 39 x 12 on ping2020i ping1090 uAvionix 0.5 20 yes yes 25 x 39 x 12 on ping1090i • GPS accuracy is limited to 7.8 m unless XPS-TR Sagetech 100 no yes 89 x 46 x 18 no additional technology is used to improve XPG-TR Sagetech 100 no yes 89 x 46 x 18 yes the results MXS Sagetech 15 150 yes yes 84 x 64 x 19 available

• GPS denial and multipath errors limit the usage of GPS in urban “canyons” created by buildings • ADS-B is not a large range of broadcast frequencies. It could become crowded with the predicted amount of s-UAS • An ADS-B – like system is preferred Specific Aim #1: Communication - LTE

• Trials with LTE enabled s-UAS have shown consistent communication in Product urban environments. Snapdragon E3272 • Several different LTE modems are AES-ATT-M14A2A-IOT-ADD-G available for s-UAS Quectel Raspberry pi kit • LTE is better for urban environments: it may be absent in very remote locations Specific Aim #1: Communication - WAVE

• Wireless Access in Vehicular Environments (WAVE) • Mostly marketed for automotive Product WaveCombo applications, but could have some LocoMate application on s-UAS that can deal SnapDragon with the products’ typically high SWAP Specific Aim # 2

• NASA and FAA work on UTM • Work by Amazon • Work by Google Specific Aim # 2 – NASA and FAA Work on UTM NASA and FAA UTM Roadmap • The progression of UTM is divided into four technical capability levels (TCL). Each TCL is differentiated by the level of risk associated with the assumed environment and the types of operations envisioned.

Drone America conducting flight approach with Savant UAS during NIAS NASA UTM Test 17 Specific Aim # 2 – NASA and FAA Work on UTM • The FAA is authorizing several third parties to act as UAS Service Suppliers (USS)

• So far Skyward, Project Wing, Airmap and Rockwell Collins are LAANC (Low Altitude Authorization and Notification Capability) Service Suppliers

NASA UTM Architecture Specific Aim # 2 – Work by Amazon • Amazon: Best-equipped, best-served model – access to airspace is granted based on whether or not the required level of safety – as determined by the relevant performance standards and rules – is achieved. • Four classes of safe operation

No fly zone – emergency use only! Well equipped vehicles as determined by relevant performance standards and rules For terminal non-transit operations (surveying, videography and inspection) and lesser-equipped vehicles, i.e. ones without sophisticated SAA – no access over heavily populated areas.

19 Specific Aim # 2 – Work by Google • Google: – Airspace Service Providers (ASPs) to license aircraft for operation and provide separation and planning to UAS via cellular networks. – UAS will give way to manned traffic via ADS-B like transceiver. – Collision avoidance using ADS-B, LTE, or 802.11p. – ASPs are interface between UAS operator and ATC. – ASP will also maintain database of no-fly zones, weather, obstacles and terrain, traffic, and flight plans through NOAA, FAA, ATCs, and weather data sources.

20 Specific Aim #3 – Development of Operational Requirements

• Urban Air Mobility and Airspace Restrictions • Traffic Restrictions and Optimization Specific Aim #3-Urban Air Mobility

• Urban Air Mobility (UAM) is the ability for s-UAS to operate safely and effectively within an urban environment • Constraints include: – Stationary obstacles: buildings, terrain – Flight ceiling: 400 ft. AGL, less if restricted by nearby airport or airspace – Keep-out geofences around important areas that can be permanent or temporary Specific Aim #3-Urban Air Mobility-Goals

• Avoid collisions with obstacles and other aircraft • Other aircraft include: – Other s-UAS – Helicopters, Blimps, other low-flying manned aircraft • Handle traffic efficiently: – Minimize the time added to a trip with traffic vs. a trip without Specific Aim #4 – Urban Air Mobility Solutions

• Environment • 2D MILP Solutions • 3D A* avoidance with rerouting Specific Aim #4 – Urban Air Mobility Solutions: Environment • For these algorithms, downtown Cincinnati was used as the location. • The buildings and ground were modeled using USGS LIDAR data • For 2D scenarios, an area was considered to be an obstacle if any point returns within it were greater than or equal to flight height minus vertical separation from ground • For 3D scenarios, all areas had vertical separation from ground added to the highest point within them Specific Aim #4 – Urban Air Mobility Solutions: 2D MILP Solutions • MILP – Mixed integer Linear Programming

𝑥, 𝑦 : position of UAS at time step 𝑣,v : velocity of UAS at time step n 𝑥, 𝑦, 𝑣,, 𝑣, : initial position and velocity of UAS provided by UAS or previous iteration of MILP Initial Constraint Finite Horizon Constraint 𝑥 𝑥 𝑥 𝐿 𝑥 𝑥 𝐿 ∀ 𝑛∈𝑁 𝑦 𝑦 𝑦 𝐿 𝑦 𝑦 𝐿 ∀ 𝑛∈𝑁 𝑣 𝑣, 𝑇 𝑁 𝑛∈𝑍 𝑛𝑐𝑒𝑖𝑙 𝑣 𝑣, 𝑇 𝑇∶time for which MILP is calculated 𝑇 : length of time step 𝐿: Length of the Finite Horizon Specific Aim #4 – Urban Air Mobility Solutions: 2D MILP Solutions

𝑓, 𝑓 : forces on the UAS in the x and y directions at time step n 𝑥 Constraints for the dynamic model: 𝑦 𝑇 𝑆 𝑆 𝐴 𝑆 𝐵 𝐹 ∀ 𝑛 ∈ 𝑁|𝑛𝑐𝑒𝑖𝑙 𝑣 𝑇 𝑣 𝑇 0 𝑚 𝑇 10𝑇 0 0 𝐵 𝑚 01 0 𝑇 𝑓 𝐴 𝑇 00 1 0 𝐹 0 𝑓 00 0 1 𝑚 𝑇 0 𝑚 Specific Aim #4 – Urban Air Mobility Solutions: 2D MILP Solutions

𝑓 : maximum force the vehicle can generate 𝐻 : integer chosen to discretize the circle 𝑣, 𝑣 : minimum and maximum velocity of the vehicle 𝑏,, : binary variable for velocity at time step n in direction h 𝜂 : very small value Acceleration and Velocity Constraints 2𝜋ℎ 2𝜋ℎ 𝑓 sin 𝑓 cos 𝑓 ∀𝑛 ∈ 𝑁 , ℎ∈𝑍, ℎ𝐻 𝐻 𝐻 2𝜋ℎ 2𝜋ℎ 𝑣 sin 𝑣 cos 𝑣 𝑣 𝑣 1 𝑏 ∀𝑛 ∈ 𝑁 , ℎ∈𝑍, ℎ𝐻 𝐻 𝐻 ,, 2𝜋ℎ 2𝜋ℎ 𝑣 sin 𝑣 cos 𝑣 𝜂 𝑣 𝑣 𝜂 𝑏 ∀ 𝑛∈𝑁 , ℎ∈𝑍 ℎ𝐻 𝐻 𝐻 ,,

𝑏,, 𝐻1 ∀𝑛 ∈ 𝑁 Specific Aim #4 – Urban Air Mobility Solutions: 2D MILP Solutions • The remaining constraints cover:

• For all n, either , or , where SF is a safety factor and , and , are the x constraints of obstacle o. The same is true for all obstacles, and intruding aircraft collisions are prevented by a similar constraint Specific Aim #4 – Urban Air Mobility Solutions: 2D MILP Solutions: results Specific Aim #4 – Urban Air Mobility Solutions: 2D MILP Solutions: results Specific Aim #4 – Urban Air Mobility Solutions: 3D A* Solutions-pseudocode

Initialize the Start_Node with starting coordinates, no parent, and g and f scores of 0 Initialize all other Nodes with no parent and g and f scores of infinity Open_set = new priorityQueue Closed_set = new set Open_set.add(Start_Node) Closed_set.add(nodes from grid that are not valid) While not_empty(Open_set) { Best_node = Open_set.pop() Closed_set.add(Best_node) If Best_node.coordinates == goal coordinates { Path = new set Current_node = Best_node While Current_node has a parent { Path.prepend(Current_node) Current_node = parent(Current_node) } Return Path } For each neighbor of Best_node { If neighbor not in Closed_set { tentativeGScore = Best_node.gScore + distance(Best_node, neighbor) if tentativeGScore < neighbor.gScore { neighbor.gScore = tentativeGScore neighbor.fScore = neighbor.gScore + distance(neighbor, goal coordinates) parent(neighbor) = Best_node Open_set.update(neighbor) } } } } Return path unavailable message Specific Aim #4 – Urban Air Mobility Solutions: 3D A* Solutions-results Specific Aim #4 – Urban Air Mobility Solutions: 3D A* Solutions-A* with re-routing

emptyTerrain = setupTerrainModel() For each s-UAS { path(s-UAS) = A*(start(s-UAS),end(s-UAS),emptyTerrain) current_Position(s-UAS) = start(s-UAS) Current_Goal(s-UAS) = path(s-UAS,goal =2) } Time_step = 0 While all paths are not complete { reservedTerrain = emptyTerrain For each s-UAS { If path(s-UAS) not complete { Update the current_position(s-UAS) according to the prevous velocity If current_position(s-UAS)==end(s-UAS) { Path(s-UAS) is complete } elseIf current_position(s-UAS) == current_goal(s-UAS) { Update current_goal(s-UAS) to the next goal in path(s-UAS) } Update the s-UAS's remaining flight time Determine the s-UAS's priority based on its nearness to its final goal and its remaining flight time Update the position in reservedTerrain occupied by the s-UAS to inaccessible } Else { Remove the s-UAS } } For each s-UAS, in order of priority { Calculate desired velocity based on current_position(s-UAS) and goal(s-UAS) temporaryTerrain = emptyTerrain for all spaces not neighbors of current_position(s-UAS) temporaryTerrain = reservedTerrain for all spaces that are neighbors of current_position(s-UAS) Predict future position if desired velocity is maintained If future position is inaccessible in the temporaryTerrain{ ReRoute = A*(current_position(s-UAS), end(s-UAS), temporaryTerrain) Modify path(s-UAS) to indicate the re-Route Update current_goal(s-UAS) Update desired velocity of s-UAS Predict future position of s-UAS } Update the future position of s-UAS in reservedTerrain to inaccessible } } Specific Aim #4 – Urban Air Mobility Solutions: 3D A* Solutions-A* with re-routing results Ongoing/Future Work

• Integrating the 3D A* algorithms with UC’s Flymaster software • Simulating multiple s-UAS with Flymaster and UTM algorithms in Gazebo with the PX4 Firmware • Using the 3D pathfinding algorithms to build multi- s-UAS handling strategies Ongoing work- simulating A* path planning using the PX4 Firmware and Gazebo Ongoing work - Using Flymaster and A* to handle flight plans Conclusions-Survey and Existing Solutions

• The Technology Survey showed sensing and computing capabilities for s-UAS are expanding rapidly • Excellent for the fate of situational awareness, but makes equipage difficult because of high product turnover rate • The solutions by industry and government are also progressing well. NASA has demonstrated TCL 3 capabilities and service suppliers are registering with the FAA Conclusions-MILP

• The MILP solutions provided detailed 2D solutions that led to optimal paths. • Capable of delaying s-UAS flights until takeoff was clear. • Planning optimal flights in this manner became time consuming when moved to large-scale 3D situations. • They are still well suited for handling an intersection Conclusions –A*

• The A* algorithm guarantees an optimal solution • In the MATLAB simulation, A* with rerouting has been shown to be scalable to 30+ paths • In the preliminary implementations with the PX4 firmware and gazebo, simply planning paths proved unreliable in urban environments because of the large risk of GPS loss. • The basic pathfinding algorithm could be improved by a better top-level traffic management procedure than a priority based solution, one that guarantees optimal usage of airspace for all vehicles Final Report will be posted on the Research Webpage http://www.dot.state.oh.us/Divisions/Planning/SPR/Research/Pag es/default.aspx