In-Space Condition-based Autonomous Inspection
for 12th International Symposium for NDT in Aerospace October 6, 2020
George Studor NASA Engineering and Safety Center NDE Technical Discipline Team
Sep 22, 2020 version
1 Bio • George Studor • NASA Engineering and Safety Center(NESC) • 35 Years with NASA and 20 years with USAF • NESC Technical Discipline Teams(TDTs) - Consultant with Analytical Mechanics Associates(AMA): • NDE TDT: In Space Inspection • Avionics TDT: Wireless Sensors/Avionics • Robotic Spacecraft TDT: Natural Systems in System Engineering • Industry Involvement: Workshops with Oil & Gas Industry/DOE • In Space Inspection Workshops (2012, 2014, 2017…. 2021) • Passive Wireless Sensor Technology Workshops (9)
2 Motivation Apollo 13 – No Views of the damage
“Houston, we’ve got a problem” then, after the SM was released… "There's one whole side of that spacecraft missing!“
MIR Spectre impact/leak – No Time Only 24 minutes to find/plug leak before having to shut the hatch.
3 AERCam – Manual deploy and operation – Dec 1997 - A Bird’s Eye View for EVA Astronauts - Inspections of Shuttle and Space Station Exterior “someday”
Mini-AERCam
“Mini AERCam Inspection Robot for Human Space Missions” AIAA Space 2004 Conference Steven E. Fredrickson*a, Steve Duran*b, Jennifer D. Mitchell*c NASA Johnson Space Center, Houston, Texas 77058 4 5 6 More Motivation • Columbia – suspected impact to Wing Leading Edge, but NO on-board impact sensors, inspection sensors or repair options.
AF Maui image 1/28/03 Columbia Breakup over TX - 2/1/03
7 Space Shuttle Thermal Protection Systems Inspections: Challenges: Needed Damage Survey & Focused Inspections - Planning/Operations for Robotic Arms - Sensor Engineering Operations and Image Analysis - Crew Time and RMS Time detracts from Mission 1:47
8 WingBiggest Leading Threat Edge comes Impact from Detection System Micro-Meteoroids and Orbital132 Debris(MMOD) Channels Impacts that can cause Critical Damage to the Re-entry Thermal Protection Systems
Challenges: • Ground Engineering & Operations Physical Optics Corp/Volodymir Romanov & Victor• Wiring Plessky to the Sensors Compton Imaging Tomography (3D Backscatter) 9 • Battery Cert and Replacement Inspection-Only vs Condition-based Inspection Late Focused Un-Dock Dock WLE Impact Inspection 2 wks 6 months later Condition Monitoring-based Inspection Use as is Analysis Repair Monitoring Analysis + Focused Inspection Discard Inspection Discard Discard Replan Replan Replan
Less workload, more risk reduction
Inspect Early High workload, Less risk reduction (Initial Condition) Establish Baseline Survey Inspection to Detect Damage (Compare with Baseline) Inspection Discard Only Replan 10 International Space Station – Low Earth Orbit Inspections ISS Inspection Examples: - Periodic Whole Vehicle Surveys - EVA Translation Paths - Windows - Solar Arrays/Masts - Radiator Cooling Systems Visiting Vehicles: - Soyuz - Cargo Vehicles Soyuz - Commercial Crew Vehicles - SpaceX - Boeing No Impact Event Monitoring SpaceX Dragon 2 Boeing CST100 Starliner
11 International Space Station Inspections Detecting the damage vs finding the damage Periodic ISS Solar Array 120’ Masts (8) General Surveys MMOD Impacts – Periodic Inspection
Ammonia Leak - Ongoing
Module Leaks to Vacuum – ongoing
2018 Soyuz EVA Configuration and Hole = Translation Paths 12 Mfg Defect International Space Station ISS life Extension From: 2024 To: 2030
356’ x 240’ Orion Spacecraft(LM) for beyond ISS
Lunar Gateway Space Station Restore-L Mission Satellite Servicing in Low Earth Orbit
Will there be an Inflatable Space Station? 13 Detecting Damage vs Characterizing Damage Entry Holes are often Narrow, sometimes Deep and rarely Perpendicular to the Surface
X-Ray CT
26 Lunar Mission Risks Greater than Low Earth Orbit Low Earth Orbit Beyond Low Earth Orbit Risks: Orbital Debris Risks: Critical Thruster Firings, Micro-Meteoroid and Radiation Van Allen Protection to 36K miles 240,000 miles! Communication will take about 2.5 seconds each way
15 Artemis III: Landing on the Moon
16 “Gateway” Lunar Space Station - High Cost means they will be Infrequent Missions - Radiation means crews have Limited Occupation - Remote & Autonomous Ops & Maintenance needed - Lead Time/Cost Rescue Missions – Failures are Costly - Hotter/Faster Re-Entry: TPS Less Tolerant of Damage
17 In Space Inspection
• Scheduled When Prudent: • Periodic Surveys • Establish Initial Configuration after Mission Configuration Event. • Assessments of Unacceptably High Risk if not detected. • Validate Models of the system, structure or environments. • Support for planned external operations: EVAs, Docking, Robotics And • Triggered when Conditions Warrant (Conditions-based): • Camera surveys or Astronaut observations detect a “Region of Interest” • Structural Sensors indicate an impact or pressure decay • System Sensors indicate performance deterioration or anomaly. • Natural or Induced Environment Changes • Events produce Uncertainty in the Spacecraft External Condition. 18 New Tools Needed:
• Efficient Manipulators and Human-like Robotics • Inspection in Difficult to Access Areas • Free-flying Inspection platforms • Sensors/Wireless Sensors to Trigger Inspections
19 Compact Robotic Manipulator Inspection with Roll-a-Tube Boom Wrist - Lightweight & Bulky Rigid Boom Sensor - Compact for Transport Shuttle Arm Inspection - Multiple Configurations
Elbow Joint
http://www.rolatube.com/satellite- deployment-booms/ Deployable Compact Boom Stowage
Shoulder http://www.rolatube.com/nuclear-inspection-booms/ 20Joint Robonaut 2B heading back to ISS
https://spectrum.ieee.org/automaton/robotics/space-robots/nasas-robonaut-to-return- to-iss-with-legs-attached 21 NASA Astrobee IVA (Internal) Testbed on ISS
NASA/Ames/Astrobee Working Group: Jose.V.Benavides@nasa.gov
AMES “Micro Gravity Test Facility” Astrobee in Japanese Kibo Module for Astrobee 22 What do we need Snake Inspector to do? Internal Functionality Snake Inspector 1. Stereo vision and articulating head with lighting, designed in stiffness at various sections along the length Entry hole Equipment Rack 2. SLAM - Simultaneous Location and Modeling Software - builds the 3d model as it goes in and subsequently determines position and orientation of all parts very accurately. Housing/control 3. Articulating snake robot - mechanical or piezo-control of the entire snake length- not just the tip as in a typical endoscope. Sensors that sense contact and feed controls to alleviate it. Area to be inspected 4. Snake real-time location - know/display the location of the entire snake - plot against accurate 3D model built in #1. 5. Self-following Software - each segment of the snake knows where and how to turn as it follows the head or backs out of a complex physical path. 6. Very small size, weight and power and works well Zero-g (assumed) 7. Length of at least 2 meters and diameter 3/8" or less – go through existing penetrations such as the holes for sending fire extinguishing 8. Minimum crew involvement/training - easy to use, hands-free, possibly remotely operated from ground. 9. Ability to change imagers at the end, and/or add sensors/wireless sensors at the tip. 10. Ability to grab items at the tip. 23 Autonomous Inspection with a Free-flying Small Spacecraft Inspector
Why a Free-flyer vs a Robotic Manipulator? • Avoid Robotic Manipulator infrastructure/operations • Minimal or No Crew Operations • Autonomous Ops means Minimal Earth-based Support • Continuous Monitoring • More View Angles and Distances • Faster response and more stable/no oscillations Condition Monitoring – Wireless/Optical Comm from sensors/systems • Events and Trends to Address Risks Sooner • Avoid Inspections if not required • Data from Hazardous Zones • Data from Difficult to access areas • Need Damage Characterization Capability if results warrant: • External: Free-flyer, Robotic Manipulator, Crawler, Tethered Sensor • Internal: Free-Flyer, Micro-Crawler or Humanoid Robot with tools
24 Example: Continuously Monitoring Gateway at “safe” distance (NASA’s Version of a Smart Home Security Camera System)
“Events” may Trigger additional Characterization: Multi-spectral & 3D Imaging RF/Light for ID, Location, Sensing RF: RF: Solar Array Orion Impact and Health Dynamics
– When to do it?
RF: Manipulator RF: Loads and Module Leak Dynamics
25 Free-flying Autonomous Spacecraft Inspector for Orion
• Solar Array Tip Cameras likely can see if there is damage • MMOD damage depth not possible with fixed cameras • Complex flight path to survey the “backshell” surface - then damage characterization as necessary
CubeSat Imaging: MMOD Entry Hole - 3D Maps of Damage Cubesat - Image Processing TPS Backshell Inspector - GN&C Input
Cubesat RF: Position RF Sensors
Cubesat Laser Comm: Solar Array - Command Sequence Updates Tip Cameras - “Answer” Downlink
26 Condition-based Autonomous Inspection is Reasonable and Practical
• Piggy-back off other Autonomous Systems and Sensing • Small Satellite and Cubesat Missions • Free-flyer testbed on ISS - Astrobee • Aircraft • Automobiles/Military Ground • Underwater Inspection ROVs • Security Camera Systems • Advanced On-board Image Processing • Inspection Drones
27 Aerospace Corp Aerocube 10 (A&B): Prox Ops & Pointing and On-board Image Processing
Photo at 26 meters! https://aerospace.org/article/cubesats-get-close-proximity-operation-interesting-implications Aerocube 7 and 11: Aerocube 8D:
28 Laser Communications Ion-Electrospray Propulsion Leverage Industry – Inspection Drones
• Inspection with Image Downlink • Command as well as Inspection Cameras • On-board Image Processing • Fully Autonomous Navigation & Control Modes • Human/Robotic Assistance – “Bird’s Eye View” - Audio • Realistic Model Building – SLAM • Dynamic obstacle avoidance and damage “cages” • Navigation within self-constructed models • Cell Phone Integration and Applications • Hazardous environments • Inspection booms with sensors • Fully Autonomous Missions and Turnaround Ops • And More…..
29 Condition-based Autonomous Inspection
What else do we need?
• End-to-End Autonomous Inspection - Provides Answers, also data as requested. - Confidence Factors for assessments • Scheduled or Triggered Operations • Cooperative Inspections (multi-camera/sensor ops) • Longer Duration Missions • Condition-based or Triggered Inspections • Results of Inspection Survey • Data from target vehicle on-board sensors
30 Oil, Gas and Aerospace Inspection Workshop Center for Petro chemical, Energy and Technology April 2021 San Jacinto College, Pasadena, TX
Co-Chairs: API/ASNT – John Nyholt - [email protected] NASA – George Studor - [email protected] ISIW 2017 - https://www.asnt.org/isiw 31 In Space and Oil & Gas Inspection Workshop 2021:
Objectives: 1. Explore the most difficult spaceflight and oil & gas challenges and solutions in damage/flaw detection and measurement: a. Concealed Damage b. Difficult to Access Areas c. Extreme/Hazardous Environments d. Remote Operations e. Autonomous Detection/Inspection. f. Optimizing Inspection-Detection Approaches. g. Modularity and Maintainability 2. Identify Synergetic technologies and needs between industries 3. Enable new Relationships and Partnerships 4. Exhibit and Demonstrate Emerging Capabilities & Technologies - Large Indoor High Bay for Exhibits and Competitions
- Outdoor Real-world Test Articles for Demonstrations 32 Back-up
Leveraging Drone Capabilities in Industry
33 Drones for Inspection – It’s a new tool for many jobs! http://www.dronesglobe.com/guide/bridge-inspection https://hazonsolutions.com Bridge Inspection Power Line Inspection Intel Falcon 8+ • Stone Arch Bridge in Minnesota • Daniel Carter Beard Bridge
www.industrialrobotics.com San Diego Gas & Electric – www.SDGE.com https://mohawkdrones.com/use-of-drones-in- https://dronelife.com/2018/01/15/2018-already-busy- construction-and-building-inspections/ police-drones/ Construction Sites First Responders and Police
34 Drones for Inspection – It’s a new tool for many jobs! Crop Evaluation Land Survey Wildfires
Harris Corp Precision Pass: https://www.cnet.com/videos/ how-drones-are-helping-fight- www.sensefly.com; www.sentera.com https://www.harrisgeospatial.com/ www.aerobatics.com Software-Technology/PrecisionPass californias-wildfires/ Railroads/Railways Confined Spaces News Broadcasts Skypersonic Real Estate Advertising www.Skypersonic.net
Interactive Aerial https://commercialdrones.fm/ https://www.youtube.com/watch Birhle “Railvision” podcast/cnn-air-greg-agvent- ?v=Lz7MUY7D_gI www.bihrle.com/railvision/index.dna drone-journalism/ Oil & Gas Pipelines Disaster Response - FEMA
http://crasar.org/
Honeywell’s UAV The Center for Robot-Assisted Inspection Service Search and Rescue (CRASAR) 35 FAA: UAS Info - www.faa.gov/uas FAA: UAS Integration Pilot Program (post “pathfinder”) https://www.faa.gov/uas/programs_partnerships/integration_pilot_program/
Purpose: craft new rules that support more complex low-altitude operations by • Identifying ways to balance local and national interests related to drone integration • Improving communications with local, state and tribal jurisdictions • Addressing security and privacy risks • Accelerating the approval of operations that currently require special authorizations.
Seven UAS Test Sites: Drone Advisory Committee (Partial List – June 2018) • North Dakota Dept of Commerce Dan Elwell, FAA Acting Administrator • State of Nevada James Burgess, Project Wing, Google X • New Mexico State University Michael Chasen, PrecisionHawk USA Inc. • University of Alaska Fairbanks Martin Gomez, Aeronautical Platforms, Facebook • Texas A&M Univ, Corpus Christi Gur Kimchi, Amazon Prime Air • Virginia Polytech & State Univ Brendan Schulman, DJI Technology • Griffiss International Airport (NY) Greg Agvent, National News Technology, CNN Todd Graetz, UAS Program, BNSF Railway Nan Mattai, Eng & InfoTech, Rockwell Collins Help with FAA Part 107 Remote Pilot exam Phil Straub, Aviation Division, Garmin Dart Drones Training Jaz Banga, Airspace Systems https://uavcoach.com/part-107-qa/
36 Drone Advisory Committee – Some Key Players in the industry
Google X Delivery Service Amazon Prime Air https://wing.com/ https://www.amazon.com/b?node=8037720011 CNN AIR Hurricane Damage https://twitter.com/cnn_air?lang=en
Drone Services: www.precisionhawk.com Oil & Gas, Utilities, Wind, Solar Agriculture Construction BNSF Beyond Line-of-Sight Autonomous Track Inspection
37 Drone Industry Trends https://www.unmannedsystemstechnology.com/ https://www.ohiodronellc.com/ Edge Processing -“Live Map” by DroneDeploy Image Analysis Softare – Harris “PrecisionPass” https://dronelife.com/2018/09/12/dronedeploy-jono-millin- talks-about-current-and-future-directions-for-the-industry/ https://www.harrisgeospatial.com/Software-Technology Autonomous UAVs - Clobotics Field Analytics - Sentera AG - 1 to 3 mm defects in Wind Turbines https://sentera.com/ www.skymatics.com Safe UAVs: Fleye www.gofleye.com www.DJI.com Copenhagen Drone Detect/Neutralization New Sensors - Kespry - ECSI - IR Drone for Roof Inspections - Audio, RF, Visual https://dronelife.com/2018/07/25/high- http://drone-hunter.com/en/detection/ resolution-thermal-imaging-for-drones-changing- the-game-on-building-inspections/ SkyX “SkyOne” – Range 100km Longer Range - BShark, SkyX - BShark Hydrogen powered drone https://dronelife.com/2018/09/16/bshark-brings- commercial-grade-hydrogen-drones-to-the-masses/ Bshark – 30km Precision Nav - Prenav Interrogate Passive Wireless Sensors - PWST Workshop - cm and mm scale location http://sites.ieee.org/wisee/program/workshops/ - Humatics Spatial Intelligence Platform™ Connecting Local Inspection to Orbital Insight38 http://www.prenav.com https://orbitalinsight.com/#slider-3 Autonomous Inspection Services: Wind Turbine Inspections: Skyspecs: www.skyspecs.com Damage Map/Characterization Deloitte: www.youtube.com/watch?v=-LOPGn7UFpk Clobotics: www.clobotics.com/en/newsEn/1901 Roof Inspection: Kespry Image: www.kespry.com Image Analysis & Hail Detection Algorithm in “the cloud” OffshoreWind.Biz Loveland Inno: www.lovelandinnovations.com Sep 2018 https://www.youtube.com/watch?v=N-XgRjEZD1A
Railway Infrastructure: BNSF - Bihrle “Railvision”: www.Bihrle.com BVLOS - NavPath UAS long range flight planner https://www.uasvision.com/2018/03/01/bnsf-railway-processes-terabytes-of- imagery-captured-during-bvlos-operations/ Union Pacific: https://www.youtube.com/watch?v=AtQgy4QzMVk
Multi-Infrastructure: 2008 –> now 25 countries Cyberhawk: https://www.thecyberhawk.com/ Utilities, Renewables, Rail, Oil & Gas Oil Tanker 39 Autonomous Features in Drones: Object Following: Skydio www.skydio.com R1 has 13 cameras, Streaming Video - Onboard Realtime Processing: NVIDIA Jetson 256 Core AI Visual SLAM, Deep Learning, Motion Planning Yuneec Typhoon H Pro – Autonomous flight Modes http://us.yuneec.com/typhoon-h-intel-realsense-technology Buildings/Confined Spaces: IR mode Shield AI: www.shield.ai - Hivemind AI Nova Prodrone: www.prodrone.com/release-en/2845/ Precision Hover - Navigation: Indoor/Outdoor – Obstacle Avoidance DJI Phantom-4 PRO V2: www.dji.com/phantom-4 DJI Mavic 2 PRO & Zoom: www.youtube.com/watch?v=7SembcsxrQw MultiPurpose: SenseFly: www.clipfail.com/watch/r5z4v2k3t4n4m4q5n3b4a3.html Professional Film-making: DJI Inspire 2 - https://www.youtube.com/watch?v=m5WvcXp2dqU comparison with Mavic: https://www.youtube.com/watch?v=wUvdPSXpawU BVLOS and VTOL: Ferrovial - AmeyVTOL “Flying Wing” http://www.vtol-technologies.com/ www.youtube.com/watch?v=OmU71886Cb0 www.ferrovial.com/en/projects/autonomous-drones/ 40 More Autonomous Features in Drones: Coordinated Large Area/Perimeter Inspection and Monitoring: Accelerated Dynamics: AD Platform v1.2 https://www.accelerateddynamics.co/ https://www.youtube.com/watch?v=b8UrjX1v_z8 Stand-alone Industrial complete Autonomous System Airobotics: Video: ww.airoboticsdrones.com/applications/inspection - configures & maintains itself remotely - indoor or outdoors
Powerline Inspection and Search and Rescue: DJI: Matrice 200/210 – Vis + IR, Obstacles, Weather & Cold https://www.youtube.com/watch?v=-ZeaqAWKvoY https://www.youtube.com/watch?v=Bj5ByZKVNao https://www.youtube.com/watch?v=GkIJ2NJQHys 41 Aircraft Inspections with Drones Autonomous close proximity defect characterization Boeing-Sensefly Albris: www.youtube.com/watch?v=mYKrR8pihAQ
Airbus-Testia: www.youtube.com/watch?v=f0VtWczvU6s
www.youtube.com/watch?v=36fchBYC8SM Donecle: www.donecle.com – EasyJet video mandatory Decals and Lightning Strikes
Output 42 “RAPID” www.output42.com/ • Blade Fix • Dent and Buckle • Cabin Survey
Ravinspec Tech (Turkey): www.ravinspect.com; www.youtube.com/watch?v=ZI6L92P5NT4 42 Sources to keep up with drone capability Training/Education/STEM: Association for Unmanned Vehicle Systems Intl(AVUSI): www.auvsi.org Sinclair College: http://uas.sinclair.edu/ Robonation: www.robonation.org Drone U: https://www.thedroneu.com/ UAV Digest: Competitions: Intl Aerial Robotics Competition: www.aerialroboticscompetition.org AVUSI Student Unmanned Aerial Systems Competition www.auvsi-suas.org
Conferences: Army Autonomy & AI Nov 28-29, 2018 www.ausa.org/army-autonomy-ai-symposium AirWorksConference (2018): https://enterprise.dji.com/airworks Construction, Energy, Agriculture, Public Safety, Infrastructure Energy Drone and Robotics Summit & Expo: Woodlands, TX June 12-13, 2019 http://stonefortgroup.com/energy-drone-coalition-2018/ Interdrone Conf & Expo: Sep 3-4, 2019 Las Vegas – www.interdrone.com Remotely Piloted Aircraft Systems: Jan 2019, Madrid https://rpas-civops.com/ Comm UAV Expo: Amsterdam(April); LasVegas(Oct) https://www.expouav.com/
Communities/News: Drone Enthusiast: www.dronethusiast.com Drone Life: https://dronelife.com UASVision: www.uasvision.com UAS Magazine: www.uasmagazine.com UAV Coach: www.uavcoach.com/news 43 Commercial Drone.FM: https://commercialdrones.fm/podcast NASA Autonomy Programs – some examples • NASA Space Mission Technology Directorate(STMD) Autonomous Systems Capability Leadership Team: AMES/Terry Fong Senior Scientist Overview Presentation: https://www.nasa.gov/sites/default/files/atoms/files/nac_tie_aug2018_tfong_tagged.pdf • 2018 Workshop on Autonomy for Future Science Missions • October 10-11, 2018 at Carnegie Mellon • Ames Research Center - Safe Autonomous System Operations Project • 2016 Report: https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20170000385.pdf • Armstrong Flight Research Center – Autonomous Systems R&D • International Space Station • REALM – RFID-Enabled Autonomous Logistics Management https://www.nasa.gov/mission_pages/station/research/experiments/2137.html • Autonomous Mission Operations EXPRESS 2.0 • Kennedy Space Center • Asteroid drone technology demo • Jet Propulsion Lab • Autonomous Sciencecraft Experiment https://ase.jpl.nasa.gov/ • Autonomous Landing on Europa
44