National Aeronautics and Space Administration

DRAFT Robotics, Tele-Robotics and Autonomous Systems Roadmap Technology Area 04

Rob Ambrose, Chair Brian Wilcox, Chair Ben Reed Larry Matthies Dave Lavery Dave Korsmeyer

November • 2010 DRAFT This page is intentionally left blank

DRAFT Table of Contents Foreword Executive Summary TA04-1 1. General Overview TA04-2 1.1. Technical Approach TA04-2 1.2. Benefits TA04-5 1.3. Traceability to NASA Strategic Goals TA04-5 1.4. Technology Push TA04-5 2. Detailed Portfolio Discussion TA04-6 2.1. Technical Area Breakdown Structure (TABS) Diagram TA04-6 2.1.1. Sensing TA04-6 2.1.2. Mobility TA04-7 2.1.3. Manipulation Technology TA04-8 2.1.4. Human-Systems Interfaces TA04-9 2.1.5. Autonomy TA04-10 2.1.6. AR&D TA04-11 2.1.7. RTA Systems Engineering TA04-12 2.2. Subtopics and Mission Diagram TA04-13 2.3. Mission by Mission Assessment TA04-13 2.3.1. SOMD Missions TA04-13 2.3.2. ESMD Missions TA04-15 2.3.3. SMD Missions TA04-19 2.3.4. ARMD Missions TA04-23 3. Conclusions TA04-24 3.1. Top Technical Challenges TA04-24 3.2. Overlap with other Technical Areas. TA04-25 3.3. Summary of Findings for Robotics, Tele-Robotics and Autonomous Systems TA04-25 Acronyms TA04-26 Acknowledgements TA04-27

DRAFT Foreword NASA’s integrated technology roadmap, including both technology pull and technology push strategies, considers a wide range of pathways to advance the nation’s current capabilities. The present state of this effort is documented in NASA’s DRAFT Space Technology Roadmap, an integrated set of fourteen technology area roadmaps, recommending the overall technology investment strategy and prioritization of NASA’s space technology activities. This document presents the DRAFT Technology Area 04 input: Robotics, Tele-Robotics and Autonomous Systems. NASA developed this DRAFT Space Technology Roadmap for use by the National Research Council (NRC) as an initial point of departure. Through an open process of community engagement, the NRC will gather input, integrate it within the Space Technology Roadmap and provide NASA with recommendations on potential future technology investments. Because it is difficult to predict the wide range of future advances possible in these areas, NASA plans updates to its integrated technology roadmap on a regular basis.

DRAFT Executive Summary rendezvous and docking research has focused on Ongoing human missions to the Internation- coupled sensing and range measurement systems al Space Station have an integrated mix of crew for vehicle pose estimation across short and long working with IVA and EVA robots and supporting ranges relative navigation sensors for various con- autonomous systems on-board spacecraft and in straints, autonomous GN&C algorithms and im- mission control. Future exploration missions will plementation in flight software, integrations and further expand these human-robot partnerships. standardization of capabilities,, docking mecha- Unmanned science missions are exclusively robot- nisms that mitigate impact loads that can increase ic in flight, but are integrated with Earth based allowable spacecraft mass, electrical/fluid/atmo- science and operations teams connected around spheric transfer across docked interfaces. Systems the globe. Autonomous unmanned aircraft used engineering topics identified for the RTA domain in military operations are now seeing civilian and include the required tolerance to environmental science applications, and the line between piloted factors of vacuum, radiation, temperature, dust, aircraft cockpits and telerobotic command con- and system level modular design philosophies that soles continues to blur. Robots, telerobots and provide for interoperability and support interna- autonomous systems are already at work in all of tional standards. Interfaces will exist between the NASA’s Mission Directorates. NASA will see even RTA domain and other roadmap domains, in- more pervasive use of these systems in its future. cluding power, destination systems, and informa- The RTA Roadmap effort has focused on the tion/modeling/simulation, habitation, and com- classical areas of sensing & perception, mobility, munications technology. manipulation, human-systems interfaces, auton- Sensing and Perception metrics include resolu- omous rendezvous and docking, and system au- tion, accuracy, range, tolerance of environmental tonomy. An additional sub-topic was added for conditions, and power. Mobility system metrics RTA-specific systems engineering such as human include range, payload, speed, life and mass. Ma- safety in proximity to robots. Functional capabil- nipulation system metrics include strength, reach, ities were identified within each of these sub-top- mass, power, resolution, minimum force/posi- ics where advances in processors, communication, tion, and number of interfaces handled. Human batteries and materials have enabled major leaps systems interface metrics include efficiency indi- forward in the past decade. Sensing and percep- ces such as mean time for a human to intervene tion research seeks new detectors, instruments and in a system. Autonomous system metrics include techniques for localization, proprioception, obsta- number of humans per system, mean time be- cle detection, object recognition and the process- tween human interventions, and number of func- ing of that data into a system’s perception of itself tions performed per intervention. Autonomous and its environment. Mobility research includes rendezvous and docking metrics include near and surface, subsurface, aerial and in-space locomo- far range, resolution, accuracy, mean docking im- tion, from small machines to large pressurized sys- pact impulse, mean docking alignment error at tems that can carry crew for long excursions, using contact, and capture envelope. modes of transport that include flying, walking, The roadmap ( see Figure R) evaluates dozens of climbing, rolling, tunneling and thrusting. Con- NASA missions of the four Mission Directorates temporary manipulation research is focused on over the next few decades and maps technology force control, compliance, eye-hand coordination, push and pull elements from the RTA core disci- tactile control, dexterous manipulation, grasping, plines into those missions, identifying ~100 indi- multi-arm control and tool use. Autonomous sys- vidual technologies that are enabling or strongly tems research seeks to improve performance with enhancing for those missions. a reduced burden on crew and ground support The top technical challenges for sensing and personnel, achieving safe and efficient control, perception are object recognition and pose esti- and enabling decisions in complex and dynamic mation and fusing visual, tactile and force sensors environments. RTA human-systems interface re- for manipulation. The top technical challenges for search includes classical areas of tele-robotics such mobility are achieving human-like performance as haptics, human-systems interfaces, and aug- for piloting vehicles and access to extreme ter- mented reality with newer topics that include hu- rain in zero, micro and reduced gravity. The top man safety, human-robot teams, crew decision technical challenges for manipulation are grap- support, interaction with the public, and super- pling and anchoring to asteroids and non-coop- vision across the time delays of space. Automated erating objects and exceeding human-like dexter- DRAFT TA04-1 ous manipulation. The top technical challenges in supporting autonomous systems on-board space- human-robot interfaces are full immersion telep- craft and in mission control. Future exploration resence with haptic, multi sensor feedback, under- missions will further expand these human-robot standing and expressing intent between humans partnerships. Unmanned science missions are ex- and robots, and supervised autonomy of dynam- clusively robotic in flight, but are integrated with ic/contact tasks across time delay. The top techni- Earth based science and operations teams con- cal challenge in Autonomy is verification of au- nected around the globe. Autonomous unmanned tonomous systems. The top technical challenge aircraft used in military operations are now see- for autonomous rendezvous and docking is prox- ing civilian and science applications, and the line imity operations culminating in its successful ac- between piloted aircraft cockpits and tele-robot- complishment despite the expected extreme con- ic command consoles continues to blur. Robots, ditions of harsh lighting, unknown near-Earth tele-robots and autonomous systems are already asteroid gravity and other unknown environmen- at work in all of NASA’s Mission Directorates. tal conditions like dust. NASA will see even more pervasive use of these The benefits to NASA of RTA technology in- systems in its future. clude extending exploration reach beyond human The RTA Roadmap effort has focused on the spaceflight limitations, reduced risks and cost in classical areas of sensing & perception, mobil- human spaceflight, enabling science, explora- ity, manipulation, rendezvous & docking, hu- tion and operation mission performance, increas- man-systems interfaces, autonomous rendezvous ing capabilities for robotic missions, use of robots and docking, and system autonomy. An addition- and autonomy as a force multiplier (e.g. multi- al sub-topic was added for RTA systems engineer- ple robots per human operator), and autonomy ing. Specific functional capabilities were identified and safety for surface landing and flying UAV’s. within each of these sub-topics where advances in The benefits outside NASA include bringing man- processors, communication, batteries and materi- ufacturing back to America; electric vehicles, wind als have enabled major leaps forward in the past turbine control, smart grids, and other green tech- decade. Sensing and perception research seeks new nology; synergy with other government agency detectors, instruments and techniques for local- robotics programs; in orbit strategic asset inspec- ization, proprioception, obstacle detection, object tion, repair and upgrade; automated mining and recognition and the processing of that data into a agriculture; prosthetics, rehabilitation, surgery, system’s perception of itself and its environment. tele-surgery, assistive robotics; undersea robotics Mobility research includes surface, subsurface, for exploration and servicing; educational robotics aerial and in-space locomotion, from small ma- for stimulating Science, Technology, Engineering chines to large pressurized systems that can carry and Mathematics inspiration; household robotics crew for long excursions, using modes of transport and automation; emergency response, hazardous that include flying, walking, climbing, rolling, materials, bomb disposal; and automated trans- tunneling and thrusting. Contemporary manipu- portation via land, air, and sea. lation research is focused on force control, compli- In summary, NASA’s four Mission Director- ance, eye-hand coordination, tactile control, dex- ates are depending on Robotics, Tele-Robotics terous manipulation, grasping, multi-arm control and Autonomy Technology. Over the next few de- and tool use. Autonomous systems research seeks cades, this technology should aim to exceed hu- to improve performance with a reduced burden man performance in sensing, piloting, driving, on crew and ground support personnel, achiev- manipulating, rendezvous and docking. This tech- ing safe and efficient control, and enabling de- nology should target cooperative and safe human cisions in complex and dynamic environments. interfaces to form human-robot teams. Autono- RTA human-systems interface research includes my should make human crews independent from classical areas of tele-robotics such as haptics and Earth and robotic missions more capable. augmented reality with newer topics that include human safety, human-robot teams, crew decision 1. General Overview support, interaction with the public, and super- vision across the time delays of space. Automat- 1.1. Technical Approach ed rendezvous and docking research has focused Ongoing human missions to the Internation- on coupled sensing and range measurement sys- al Space Station have an integrated mix of crew tems for vehicle pose estimation across short and working with IVA and EVA robots teamed with long ranges, relative navigation sensors for vari-

TA04-2 DRAFT Figure R: Robotics, Tele-Robotics and Autonomous Systems Technology Area Strategic Roadmap (TASR)

DRAFT TA04–3/4 This page is intentionally left blank ous constraints, autonomous GN&C algorithms and implementation in flight software, integra- tion and standardization of capabilities, docking mechanisms that mitigate impact loads that can increase allowable spacecraft structure and mass, electrical/fluid/atmospheric transfer across docked interfaces. Systems engineering topics identified for the RTA domain include the required toler- ance to environmental factors of vacuum, radia- tion, temperature, dust, and system level modular design philosophies that provide for interoperabil- ity and support international standards. Interfac- es will exist between the RTA domain and other roadmap domains, including power, destination systems, and information/modeling/simulation, habitation, and communications technology. Figure 1. Cover of the National Space Policy of the 1.2. Benefits United States of America Spaceflight is costly across the development, “Maintain a sustained robotic presence” in the so- flight unit production, launch and operation lar system to conduct science, demonstrate tech- phases of missions. Spaceflight is also risky to nologies and scout locations for future human both man and machine. Each of the RTA subtop- missions (page 11). ics is focused on research to reduce cost and risk. The Policy also establishes the use of space nu- An even greater benefit is when new technologies clear power systems to safely enable or significant- increase capabilities, or add whole new functions ly enhance space exploration and operational ca- that truly “change the game”. pabilties. The use of nuclear electric and nuclear So for each subtopic within the RTA domain we thermal power will require a significant improve- seek to make spaceflight safer and more econom- ment in autonomous system functions for the ical, while looking for improvements and break- management of nuclear power sources providing throughs as measured with quantifiable metrics. electrical power, thermal power, and propoulsion. Sensing and Perception metrics include resolu- The maturation of autonomous systems will re- tion, accuracy, range, tolerance of environmental quire demonstration missions in interplanetary conditions, and power. Mobility system metrics space. This policy directly drives the need for im- include range, payload, speed, life and mass. Ma- mediate and sustained development and matura- nipulation system metrics include strength, reach, tion of autonomous system technologies. mass, power, resolution, minimum force/position, 1.4. Technology Push and number of interfaces handled. Human sys- tems interface metrics include efficiency indicies Robotics, Tele-Robotics and Autonomous Sys- such as mean time to intervene in a system. Au- tems represent an exploding domain of research tonomous system metrics include number of hu- with broad investments beyond NASA and the mans per system, mean time between intervention US. This roadmap seeks to identify technologies by humans, and number of functions performed that can be integrated for flight missions over the per intervention. Autonomous rendezvous and next 25 years. The conventional approach to road docking metrics include near and far range, res- mapping involves technology pull, where mission olution, accuracy, mean docking impact impulse, needs are used to guide technology investment mean docking alignment error at contact, capture and development. The following sections use this envelope. approach, identifying missions over the next 25 years and proposing technology needs. At the 1.3. Traceability to NASA Strategic Goals same time the panel recognized the need for tech- Robotics, Tele-Robotics and Autonomous Sys- nology push, where major breakthroughs enable tems are prominently mentioned in the US Space new missions or potentially change missions plan- Policy released June 28, 2010 (see Figure 1). One ning with new capabilities. Therefore some invest- of its goals is to “Pursue human and robotic ini- ment in basic research is encouraged to invent and tiatives” to develop innovative technologies (page mature new approaches that are not today seen as 4). In the US Space Policy, NASA is directed to credible.

DRAFT TA04-5 2. Detailed Portfolio Discussion pact three areas of capability: autonomous naviga- tion, sampling and manipulation, and interpreta- 2.1. Technical Area Breakdown Structure tion of science data. (TABS) Diagram In autonomous navigation, 3-D perception has The Robotics, Tele-Robitics and Autonomous already been central to autonomous navigation of Systems Technology Area Breakdown Structure planetary rovers. Current capability focuses on ste- (TABS) is shown in Figure 2. reoscopic 3-D perception in daylight. Active opti- 2.1.1. Sensing cal ranging (LIDAR) is commonly used in Earth- This area includes sensors and algorithms need- based robotic systems and is under development ed to convert sensor data into representations for landing hazard detection in planetary explora- suitable for decision-making. Traditional space- tion. Progress is necessary in increasing the speed, craft sensing and perception included position, resolution, and field of regard of such sensors, re- attitude, and velocity estimation in reference ducing their size, weight, and power, enabling frames centered on solar system bodies, plus sens- night operation, and hardening them for flight. ing spacecraft internal degrees of freedom, such Imagery and range data is already in some use for as scan-platform angles. Current and future devel- rover and lander position and velocity estimation, opment will expand this to include position, atti- though with relatively slow update rates. Real- tude, and velocity estimation relative to local ter- time, onboard 3-D perception, mapping, and ter- rain, plus rich perception of characteristics of local rain-relative position and velocity estimation ca- terrain — where “terrain” may include the struc- pability is also needed for small body proximity ture of other spacecraft in the vicinity and dy- operation, balloons and airships, and micro-in- namic events, such as atmospheric phenomena. spector spacecraft. For surface navigation, sensing Enhanced sensing and perception will broadly im- and perception must be extended from 3-D per-

Figure 2. RTA Technical Area Breakdown Structure (TABS) Diagram

TA04-6 DRAFT ception to estimating other terrain properties per- mobility to high speeds and progressively more tinent to trafficability analysis, such as softness of extreme terrain. For microgravity mobility, the soil or depth to the load-bearing surface. Many Manned Maneuvering Unit (MMU), tested in types of sensors may be relevant to this task, in- 1984 and, more recently, the SAFER jet pack pro- cluding contact and remote sensors onboard rov- vide individual astronauts with the ability to move ers and remote sensors on orbiters. and maneuver in free space, or in the neighbor- Sampling generally refers to handling natural hood of a Near-Earth Asteroid. The AERCam sys- materials in scientific exploration; manipulation tem flew on STS-87 in 1997 as the first of future includes actions needed in sampling and handling small free-flying inspection satellites. man-made objects, including sample contain- We can expect in the next few decades that ro- ers in scientific exploration and handling a vari- botic vehicles designed for planetary surfaces will ety of tools and structures during robotic assembly approach or even exceed the performance of the and maintenance. 3-D perception, mapping, and best piloted human vehicles on Earth in travers- relative motion estimation are also relevant here. ing extreme terrain and reaching sites of interest Non-geometric terrain property estimation is also despite severe terrain challenges. Human driv- relevant to distinguish where and how to sample, ers have a remarkable ability to perceive terrain as well as where and how to anchor to surfaces hazards at long range and to pilot surface vehi- in micro-gravity or to steep slopes on large bod- cles along dynamic trajectories that seem nearly ies. Additional needs include recognizing known optimal. Despite the limitations of human sens- objects, estimating the position and orientation ing and cognition, it is generally observed that ex- of those objects, and fusing measurements from perienced drivers can pilot their vehicles at speeds force-torque, tactical, and visual sensors to exe- near the limits set by physical law (e.g. friction- cute grasping operations and mating/de-mating al coefficients, tipover and other vehicle-terrain of pairs of objects. kinematic and dynamic failures). This fact is re- Onboard science data analysis is important in at markable given the huge computational through- least two broad situations: (1) where large data sets put requirements needed to quickly assess subtle must be searched to find things of interest, and it terrain geometric and non-geometric properties is impractical to downlink the entire data set to (e.g. visually estimating the properties of soft soil) Earth, and (2) where time-sensitive phenomena at long range fast enough to maintain speeds near must be detected before the phenomena end (e.g. the vehicle limits. This ability is lacking in today’s eruptions) or the spacecraft moves beyond range best obstacle detection and hazard avoidance sys- of further observation. Success stories already in- tems. clude onboard detection of dust devils and clouds For free-flying vehicles, either in a microgravi- by Mars rovers; future examples could include de- ty environment or flying through an atmosphere, tecting dynamic events on Earth, comets, or Titan we can similarly expect that robotic vehicles will and onboard analysis of large, hyperspectral data become capable of utilizing essentially all avail- sets. Perception tends to be very computational- able vehicle performance, in terms of acceleration, ly intensive, so progress in this area will be close- turn rate, stopping distance, etc. without being ly linked to progress in high performance onboard limited either by the onboard sensors, computa- computing. tional throughput, or appropriate algorithms in 2.1.2. Mobility making timely decisions. Future missions may be identified for small or nano satellites, where break- The state of the art in robotic space mobility throughs in miniaturization of electronics, cam- (e.g. not including conventional rocket propul- eras and other sensors will allow functions per- sion) includes the Mars Exploration Rovers and formed by large spacecraft at less than 1% the the upcoming Mars Science Laboratory, and for mass and cost. human surface mobility the Apollo lunar roving NASA in particular has the need to reach sites of vehicle used on the final three Apollo missions. scientific interest (e.g. on the sides of cliffs) that are Recently, systems have been developed and test- of less interest to other agencies. So NASA needs ed on Earth for mobility on planetary surfaces to focus especially on those aspects of extreme- including the Space Exploration Vehicle and the terrain surface mobility, free-space mobility and ATHLETE wheel-on-leg cargo transporter. Both landing/attachment that will not be developed by feature active suspension. A series of grand chal- anyone else. Also, NASA has environmental con- lenges have extended the reach of robotic off-road straints such as thermal extremes and rad-hard

DRAFT TA04-7 computing that may make solutions developed for berthing, deploying, sampling, bending, and even others in-applicable to NASA. While high speed positioning the crew on the end of long arms are operations are of only limited use to NASA, mis- tasks considered to be forms of manipulation (see sion success will often depend on reliable, sus- Figure 3). Arms, cables, fingers, scoops, and com- tained operations, including the ability to move binations of multiple limbs are embodiments of through the environment long distances without manipulators. Here we look ahead to missions’ re- consuming too much of the mission timeline. quirements and chart the evolution of these capa- Mass, and to some degree power, generally need bilities that will be needed for space missions. Ma- to have a much greater degree of emphasis in the nipulation applications for human missions can design process for NASA missions than others. As be found in Technology Area 7 as powered exo- a result, mobility systems that can "brute force" skeletons, or payload offloading devices that ex- a solution to a difficult problem may need to be ceed human strength alone. accomplished by "finesse" in a NASA context. A Sample Handling- The state of the art is found good example of this is the use of tank-treads of in the MSL arm, Phoenix arm, MER arm, So- high-mobility military vehicles. Such systems pro- journer arm, and Viking. Future needs include vide good mobility but tend to entrain debris into handling segmented samples (cores, rocks) rath- the running gear in a way that requires lots of mass er than scoop full of soil, loading samples into on- and power to crush and expel. As a result, NASA board devices, loading samples into containers, has invested in alternatives such as multi-wheel sorting samples, and cutting samples. vehicles that have similarly low ground pressures Grappling- The state of the art is found in the but much lower power requirements. This trend SRMS, MFD, ETS-VII, SSRMS, Orbital Express, of NASA needing to invest in specialized systems and SPDM. Near term advances will be seen in that meet its own unique needs will presumably the NASA Robonaut 2 mission. Challenges that continue. A mobility system is highly dependent will need to be overcome include grappling with a on its power subsystems, especially for long tran- dead spacecraft, grappling a natural object like an sit, working against friction, or when carrying asteroid, grappling in deep space, and assembly of heavy payloads. In particular the specific power a multi-stack spacecraft. and specific energy metrics will dominate a mo- Eye-Hand Coordination- The state of the art is bility system’s range, speed and payload capacity. placement of MER instruments on rocks, Orbit- Technology Area 3 addresses new technologies for al Express refueling, SPDM ORU handling and improved space power and energy storage systems. Phoenix digging. Challenges to be overcome in- Other agencies have made significant develop- clude working with natural objects in micro grav- ment and investment in Unmanned Aerial Vehi- ity (asteroids), operation in poor lighting, cali- cles (UAVs). These capabilities can be adapted and bration methods, and combination of vision and applied to exploration of planetary surfaces and touch. weather. In particular, NASA will need to develop EVA positioning- The EVA community has fully autonomous and automated UAVs for oper- come to rely on the use of large robot foot re- ations on distant planets. Coordination of mul- straints versus having crew climb. The state of the tiple robotic systems is an active area of research. art is found in the SRMS and SSRMS. These arms Combinations of heterogeneous systems, such as were originally designed for handling inert pay- flying and roving systems is potentially useful for loads, and no controls were developed for control surface missions, pairing long range sensing on by the crew on the arm. Challenges to be over- the flyer with higher resolution surface sensing on come involve letting crew position themselves the rover. without multiple IV crew helping, safety issues, Mobility applications for human missions are and operation of these arms far from Earth sup- described in Technical Area 7, Human Explora- port. tion Destinations Systems. These include rovers, Assembly and Servicing - The state of the art is hoppers, docking spacecraft and EVA mobility in only large scale assembly of spacecraft modules aids such as exoskeletons and jetpacks. with the SRMS and SSRMS, and servicing is lim- 2.1.3. Manipulation Technology ited to ORU handling with Orbital Express and SPDM. Challenges to be overcome include open- Manipulation is defined as making an intention- ing and closing boxes, handling flexible materi- al change in the environment. Positioning sensors, als, force controlled mating of structure, mating handling objects, digging, assembling, grappling, electrical and fluid connectors, fastening, rout-

TA04-8 DRAFT Figure 3. Orbital Express, Phoenix Arm, MSL Arm, Robonaut 2, SSRMS & SPDM, JAXA MFD, ETS-VII ing cables and hoses, tether/capture management, ating “Holodeck”-like virtual environments that cutting and bending, and working with EVA in- can be naturally explored by the human opera- terfaces. tor with “Avatar”-like telepresence. These interfac- Tool Use - The state of the art in space tool use es will also more fully engage the aural and tactile is found in the Sojourner and MER instruments. senses of the human to communicate more infor- Near term advances will be seen with the Mars mation about the state of the robot and its sur- Science Lab and uses of the SPDM RMCT tool roundings. As robots grow increasingly autono- on ISS. Challenges to be overcome include use of mous, improved techniques for communicating a minimum set of tools, common too interfaces, the “mental state” of robots will be introduced, as common robot-EVA-tools, robot dexterity to han- well as mechanisms for understanding the dynam- dle tools built for humans, and smart tools that go ic state of reconfigurable robots and complex sen- beyond mechanical pass throughs. sor data from swarms. 2.1.4. Human-Systems Interfaces Current human-robot interfaces typically al- low for two types of commands. The first are sim- The ultimate efficacy of space systems depends ple, brief directives, sometimes sent via specialized greatly upon the interfaces that humans use to op- control devices such as joysticks, which interrupt erate them. The current state of the art in human- existing commands and immediately affect the system interfaces is summarized below along with state of the robot. A few interfaces allow the issu- some of the advances that are expected in the next ance of these commands through speech and ges- 25 years. tures. These immediate commands are ineffective Human operation of most systems today is ac- when significant communications delay is pres- complished in a simple pattern reminiscent of the ent. The second are higher-level goals, frequently classic “Sense – Plan – Act” control paradigm for presented in a sequence, that engage autonomous robotics and remotely operated systems. The hu- behaviors on the robot and require a significant man observes the state of the system and its en- amount of time to complete. Interfaces that allow vironment, forms a mental plan for its future ac- the latter type of commanding offer modeling and tion, and then commands the robot or machine to simulation capabilities that attempt to predict the execute that plan. Most of the recent work in this outcome of a series of commands. field is focused on providing tools to more effec- Future interfaces will make use of the steady ad- tively communicate state to the human and cap- vances in mobile devices and the entertainment ture commands for the robot, each of which is dis- industry to provide more intuitive and natural ro- cussed in more detail below. bot command devices. Robots will also be able to Current human-system interfaces typically in- more accurately interpret human speech and ges- clude software applications that communicate in- tures in the absence of these devices, enabling ro- ternal system state via abstract gauges and read- bots to more effectively work with humans. In outs reminiscent of aircraft cockpits or overlays many cases, commands will be automatically gen- on realistic illustrations of the physical plant and erated based on observations of the human’s natu- its components. Information from sensors is avail- ral interactions with a virtual or real environment, able in its native form (for instance, a single image removing the need for the human to explicitly is- from a camera) and aggregated into a navigable sue commands at all. model of the environment that may contain data Additional progress in two crosscutting areas is from multiple measurements and sensors. Some necessary to enable the advances described above. interfaces are adapted to immersive displays, mo- First, the lack of developed standards and conven- bile devices, or allow multiple distributed opera- tions for the control of robotic systems is having a tors to monitor the remote system simultaneously. negative impact on usability and limits the lever- Future interfaces will communicate state aging of one robot’s interface for another. Second, through increased use of immersive displays, cre- as humans interact more closely with robotic sys-

DRAFT TA04-9 tems (in some cases riding inside them), increased gree of autonomy can be varied from essentially focus on the need for safe physical interactions be- none to near or complete autonomy. For example, tween robots and humans is necessary. in a human-robot system with mixed initiative, While human-machine interfaces are complex, the operator may switch levels of autonomy on- the interfaces between multiple machines should board the robot. Controlling levels of autonomy is be more easily defined and consistent. This will tantamount to controlling bounds on the robot's be of importance when humans are working with authority, response, and operational capabilities. multiple machines. The complete research do- The Mars Exploration Rovers are the state-of- main includes combinations of “n” humans paired the-practice for operational mixed initiative au- with “m” machines. tonomy in a space robotic system. Multi-day top- 2.1.5. Autonomy level plans of rover actions are developed on the Earth for execution on-board the Rovers. On- Autonomy, in the context of a system (robot- board autonomy enables hazard avoidance while ic, spacecraft, or aircraft), is the capability for the driving, provides limited on-board automation of system to operate independently from external the acquisition of science data, and provides limit- control. For NASA missions there is a spectrum ed failure mode behaviors and actions when faults of Autonomy in a system from basic automa- occur in the system. tion (mechanistic execution of action or response Variable levels of autonomous systems can to stimuli) through to fully autonomous systems be applied to virtually any NASA system: air- able to act independently in dynamic and uncer- craft (autonomous, remotely-piloted, commer- tain environments. cial free-flight systems); spacecraft (robotic sys- Two application areas of autonomy are: (i) in- tems performing exploration, crewed spacecraft creased use of autonomy to enable an indepen- with decision support systems); and ground-based dent acting system, and (ii) automation as an automation to support science discovery, vehi- augmentation of human operation. Autonomy’s cle system management, and mission operations. fundamental benefits are; increasing a system op- Greater use of highly adaptable and variable au- erations capability, cost savings via increased hu- tonomous systems and processes can provide sig- man labor efficiencies and reduced needs, and nificant time-domain operational advantages to increased mission assurance or robustness to un- robotic systems or crewed systems that are limit- certain environments. ed to human planning, decision, and data man- An “autonomous system” is defined as a system agement speeds. Crew centered operations is a that resolves choices on its own. The goals the sys- complex challenge because it means that the crew tem is trying to accomplish are provided by an- must be able to track and modify daily activity other entity; thus, the system is autonomous from plans, monitor key systems, isolate anomalies, and the entity on whose behalf the goals are being select and perform any required recovery proce- achieved. The decision-making processes may in dures. All this requires significant on-board auto- fact be simple, but the choices are made locally. In mation and system autonomy to support the crew. contrast, an “automated system” follows a script, Achieving these gains from use of autonomous albeit a potentially quite sophisticated one; if it systems will require developing new methods to encounters an unplanned-for situation, it stops establish “trusted autonomy” through verification and waits for human help, e.g. “phones home”. and validation (V&V) of the near-infinite state The choices have either been made already, and systems that result from high levels of adaptabili- encoded in some way, or will be made external- ty; state management and system diagnostic/prog- ly to the system nostic technologies to enable complex systems Key attributes of such autonomy for a robot- to operate across a range of functional capabili- ic system include the ability for complex decision ties; and for human decision support systems that making, including autonomous mission execution manage multivariate plans and constraint optimi- and planning, the ability to self-adapt as the envi- zations. ronment in which the system is operating chang- Autonomous fault detection, isolation, and re- es, and the ability to understand system state and covery are critical for overall system autonomy. react accordingly. Variable (or mixed initiative) These real-time health management functions re- autonomy refers to systems in which a user can quire assessments and decisions in a timeframe of specify the degree of autonomous control that the milliseconds to minutes. Representative capabil- system is allowed to take on, and in which this de- ities include crew escape and abort decisions as

TA04-10 DRAFT well as on-board diagnostics and recovery. ploration, as well as satellite servicing/rescue, and Autonomous systems research involves the inte- is an essential capability for the future of human gration and implementation of several advanced and robotic missions. However, NASA and the autonomy techniques. Autonomous systems re- U.S. space industry have yet to develop and dem- quire unambiguous knowledge of the vehicle onstrate a robust Automated/Autonomous Ren- states, including location of failures and future dezvous and Docking (AR&D) capability suite states. ISHM provides the state determination, di- that can be confidently utilized on human space- agnostics, and prognostics of the systems and ve- flight and robotic vehicles over a variety of design hicle. Buidling upon this state information, on- reference missions. While Space Shuttle rendez- board mission executive and mission planning vous activities have been 100% successful, they autonomoy provides the decision making neces- have been limited to LEO operations and relied sary to manage the mission, vehicle, and failure heavily on ground operators and flight crew to in- responses. Radiation hardened, high performance crease robustness and probability of mission suc- processors are essential to enable this level of func- cess. Operational ISS transport and re-supply is tionality (see TA 11 for more detail on process- currently provided by AR&D systems in the form ing). Different autonomous algorithms may prove of ATV, HTV, and Progress, and in the future, to perform different functions better and differ- by US commercial vendors. These systems are op- ent vehicle systems (i.e., life support, propulsion, timized to take advantage of LEO infrastructure, thermal control, electrical power) may require dif- such as GPS, and ground controllers, and are there- ferent algorithmic approaches. The integration of fore not extensible to the different mission class- different algorithms to provide a consistent man- es beyond LEO without significant NRE. Other agement function has not yet been accomplished. recent AR&D technology demonstrators such as Verification and Validation of non-deterministic Orbital Express and XSS-11 have flown success- algorithms (e.g., dynamic neural networks, infer- ful or partially successful Rendezvous, Proximity ence engines, fuzzy logic) will require new meth- Operations, and Docking (RPOD) missions with ods in order to verify and validate the safe oper- some limited human involvement from ground ation of these algorithms in all possible vehicle controllers. However, much of the hardware dem- conditions. onstrated on these missions is no longer available With crew sizes of perhaps 4 – 6, the availabil- to support future flights. Since full autonomy and ity of crew is limited to perhaps 2 crew members automation has not been required for rendezvous managing the entire vehicle at any given time. and docking missions as yet, NASA does not have Some believe that the level of complexity of an in- a ready-to-fly AR&D capability that would be terplanetary spacecraft will be similar to that of considered routine, reliable, versatile, and cost-ef- a nuclear attack submarine which has 134 crew. fective, especially for missions beyond Low Earth Managing a vehicle of this complexity with only 2 Orbit (LEO). crew members will require significant automation AR&D is a capability requiring many vehicle of the vehicle management functions. In addi- subsystems to operate in concert. It is important tion, as the vehicle moves beyond 5 light minutes to clarify that AR&D is not a system and cannot from Earth, response time becomes a limiting fac- be purchased off the shelf. This strategy focuses on tor. The vehicle will have to respond to unexpect- development of a certified, standardized capability ed conditions such as solar flares or system failures suite of subsystems enabling AR&D for different without input from terrestrial control centers or mission classes and needs (see Figure 4). This suite operators. Autonomous and automated operation will be incrementally developed, tested and inte- for this level of complexity and crew size has not grated over a span of several missions. This tech- been demonstrated nor fully understood. nology roadmap focuses on four specific subsys- tems required for any AR&D mission. 2.1.6. AR&D1 The ability of space assets to autonomously ren- 1) Relative Navigation Sensors – During the course dezvous and dock enables human and science ex- of RPOD, varying accuracies of bearing, range, and relative attitude are needed for AR&D. 1 Summary from “A Strategy for the U.S. to Develop and Current implementations for optical, laser, Maintain a Versatile Mainstream Capability for Automated/Autono- and RF systems are mid-TRL (Technology mous Rendezvous and Docking in Low Earth Orbit and Beyond”, Readiness Level) and require some development Draft August 2010, Authored by the AR&D Community of Practice, a collaboration among ARC, DFRC, GRC, GSFC, HQ, JSC, JPL, and flight experience to gain reliability and LaRC, MSFC, NESC, and the HQ Office of the Chief Engineer. operational confidence. Inclusion of the ability DRAFT TA04-11 for cooperating AR&D pairs to communicate Although none of the subsystems are low TRL, directly can greatly improve the responsiveness some are not mature enough to be considered and robustness of the system. routine and cost-effective, and all need some lev- 2) Robust AR&D GN&C Real-Time Flight el of development, ground testing and/or on-orbit Software (FSW) – AR&D GN&C algorithms demonstration as well as incorporation of lessons are maturing, however, implementing these learned through integration with other subsys- algorithms into FSW is an enormous challenge. tems on a variety of spacecraft before they can A best practice based implementation of all be considered part of a versatile mainstream automated/autonomous GN&C algorithms U.S. capability. By evaluating mission character- into real-time FSW operating systems needs to istics against mission class, the above figure illus- be developed and tested. trates the essential nature of AR&D as an enabler of NASA missions over the next 20-25 years. The 3) Docking/Capture – NASA is planning for the main challenges of AR&D are a) the integration imminent construction of a new low-impact of subsystems to form a robust, cohesive, auton- docking mechanism built to an international omous system and b) standardization of AR&D standard for human spaceflight missions to suites between vehicles that operate in the same ISS. A smaller common docking system for theater so that vehicles built by different Pro- robotic spacecraft is also needed to enable grams/projects can interoperate. The Advanced robotic spacecraft AR&D within the capture Video Guidance System (AVGS) represents NA- envelopes of these systems. Assembly of the SAs current AR&D capability. The system has large vehicles and stages used for beyond successfully completed test flights on the Shuttle, LEO exploration missions will require new the Demonstration of Autonomous Rendezvous mechanisms with new capture envelopes Technologies (DART), and Orbital Express (OE- beyond any docking system currently used 1). This capability employs the use of reflectors on or in development. Development and testing the target spacecraft and provides for safe docking of autonomous robotic capture of non- and mating operations. cooperative target vehicles in which the target While landing on planetary bodies is the do- does not have capture aids such as grapple main of TA9 (Entry, Descent and Landing Sys- fixtures or docking mechanisms is needed to tems) there may be some overlap with Auton- support satellite servicing/rescue. omous Rendezvous and Docking when the 4) Mission/System Managers – A scalable spacecraft planetary body is small, as in the case of an aster- software executive that can be tailored for oid. For very small objects the gravity forces ap- various mission applications, for the whole proach zero and the landing approaches docking. vehicle, and various levels of autonomy and Measuring spin rate, matching rates and the use of automation is needed to ensure safety and anchoring devices will likely have much in com- operational confidence in AR&D software mon with the previously described technologies. execution. Numerous spacecraft software Additional challenges will arise with a dusty and executives have been developed, but the heterogeneous landing surface. necessary piece that is missing is an Agency- 2.1.7. RTA Systems Engineering wide open standard which will minimize the Many advances in robotics and autonomy de- costs of such architectures and its ability to pend on increased computational power. There- evolve over time to help overcome general fears fore, advances in high performance, low power about autonomy/automation.

Figure 4. Notional AR&D Capabilities vs. Mission Class and Order of Difficulty TA04-12 DRAFT onboard computers are central to more capable space robotics. Current efforts in this direction in- clude exploiting high performance field program- mable gate arrays (FPGAs), multi-core processors, and enabling use in space of commercial grade computer components through shielding, hard- ware redundancy, and fault tolerant software de- sign. Further pushes in these or other directions to achieve greater in-space computing power are needed. Modular interfaces are needed to enable tool change-out for arms on rovers and for in-space ro- botics assembly and servicing. When robots and Figure 5. Robonaut 2 concept for working inside humans need to work in close proximity; sensing, ISS planning, and autonomous control system for the evolving from Earth to in-space experiments. The robots, and overall operational procedures for ro- main objectives are to explore dexterous manipu- bots and humans, will have to be designed to en- lation in zero gravity, test human-robot safety sys- sure human safety around robots. tems, test remote supervision techniques for oper- Developing modular robotic interfaces will also ation across time delays, and experiment with ISS allow multiple robots to operate together. These equipment to begin offloading crew of housekeep- modular interfaces will allow structural, mechan- ing and other chores. The R2 was built in a part- ical, electrical, data, fluid, pneumatic and oth- nership with General Motors, with a shared vision er interaction. Tools and end effectors can also be of a capable but safe robot working near people. developed in a modular manner allowing inter- The R2 has the state of the art in tactile sens- changeability and a reduced logistics footprint. ing and perception, as well as depth map sensors, Modular interfaces will be the building block for stereo vision, and force sensing. The R2 will be modular self-replicating robots, and self-assem- deployed initially on a fixed pedestal with zero bling robotic systems. mobility, but future upgrades are planned to al- Reconfigurable system design offers the ability low it to climb and reposition itself at different to reconfigure mechanical, electrical and comput- worksites. Robonaut 2’s dexterous manipulators ing assets in response to system failures. Recon- are the state of the art, with three levels of force figurable computing offers the ability to internal- sensing for safety, high strength to weight ratios, ly reconfigure in response to chip level failures compliant and back drivable drive trains, soft and caused by environmental (i.e. space radiation), life smooth coverings, fine force and position control, limitations, or fabrication errors. dual arm coordination, and kinematic redundan- System verification will be a new challenge for cy. Human interfaces for the R2 include direct human rated spacecraft bound for deep space. force interaction where humans can manually po- New V&V approaches and techniques will be re- sition the limbs, trajectory design software tools, quired, and in-flight re-verification following a re- and script engines. R2 is designed to be directly pair may be necessary. tele-operated, remotely supervised, or run in an automated manner. The modular design can be 2.2. Subtopics and Mission Diagram upgraded over time to extend the Robonaut ca- The subtopics and mission diagram is shown in pabilities with new limbs, backpacks, sensors and the Figure R foldout. software. 2.3. Mission by Mission Assessment 2.3.1.2. ISS Refueling 2.3.1. SOMD Missions The Robotic Refueling Dexterous Demonstra- tion (R2D2) is a multifaceted payload designed 2.3.1.1. Robonaut 2 mission to ISS for representative tasks required to robotically re- During FY11 the Robonaut 2 system (see Fig- fuel a spacecraft. Once mounted to the Interna- ure 5) will be launched on STS-133 and deliv- tional Space Station, the demonstration will uti- ered to the ISS in what will become the Perma- lize the R2D2 payload complement, the Special nent Multipurpose Module (PMM). Robonaut 2 Purpose Dexterous Manipulator (SPDM) robotic (R2) is the latest in a series of dexterous robots arms, and 4 customized, interchangeable tools to built by NASA as technology demonstration, now simulate the tasks needed to refuel a spacecraft us- DRAFT TA04-13 ing its standard ground fill‐and‐drain valve. Dur- obtained from fixed cameras, cameras on robot- ing the mission, operators at JSC will maneuver ic manipulators, or cameras carried by crewmem- the SPDM robotic arms which will interact with bers during EVA. Similar tasks are anticipated for the R2D2 payload box and complete its robot- exploration missions: Inspect descent vehicle ther- ic tasks. Using SPDM’s end effector and inter- mal protection system before entry and landing; changeable R2D2 tools, operators will locate and provide alternative to EVA inspection during lu- access the fuel valve on the R2D2 payload box, nar cruise or interplanetary flight; provide visual uncap it, open the manual valve, and then transfer inspection cues to aid in developing repair plans a liquid, simulated fuel through the tool interface and procedures for EVA tasks, including in-space into the fill‐and‐drain valve. A “busy board” on assembly and maintenance. the R2D2 payload box will support the demon- The free-flyer represents the state of the art in stration of general robotic operations relevant to remote sensing instruments, with a 10x reduc- space servicing. Four advanced tools were devel- tion in mass and power due to the small scale of oped for this mission. Each tool is equipped with the system (<20Kg). The mobility requires com- two integral cameras to help operators guide their plete 6 axis motion control using a cold jet sys- maneuvers. The cameras use the OTCM umbili- tem that can be refueled IVA, relative navigation cal connector for power and data. in space, proximity loiter for inspection, obstacle 2.3.1.3. Free-flyer Inspection Robot avoidance and trajectory planning. Human inter- This proposed technology push mission is based faces include direct handling, teleoperation, and on an ISS utilization proposal titled “ISS Free fly- remote supervision. Autonomous skills for hover, er for Inspection, Remote Viewing, Science and relative trajectories, and autonomous rendezvous Technology”. The small free-flyer would be taken and docking with the JAXA airlock are required to ISS and flown outside through the JAXA air- to retrieve the free-flyer. lock, then back inside for refurbishment. The de- 2.3.1.4. Astronaut JetPack sign is based on a mix of results from the AER- As humans extend their reach to asteroids or Cam flight DTO (STS-87), as well as technology other in-space destinations, the ability to fly will work done for Mini-AERCam (JSC) (see Figure become essential for human mobility and locomo- 6), Inspector (JPL), PSA (MIT,ARC) and oth- tion. The ISS is designed with handrails and teth- er free flyer work in universities and other agen- er points to assist EVA, but asteroids and satellites cies. The free-flyer benefits both current human have no such features. EVA for the Shuttle and ISS spaceflight (ISS) and future exploration missions. missions relies on the SAFER Jetpack (see Figure In near-term operational applications, an exter- 7) for emergency recovery if an astronaut falls off nal free flyer provides beneficial views of on-orbit the spacecraft, but the SAFER is single string, and maintenance and servicing tasks that cannot be can only be used if the astronaut looses grip and a tether system fails. This technology push mis- sion would develop a multi-string Jet- pack able to be used for nominal flight during EVA on ISS, as well be applied to asteroid and high orbit missions where handrails and tether points are not avail- able. Sensing and per- ception require- ments are minimal, Figure 7. Existing Simplified Aid Figure 6. Mini AERCam Image and Top Level for EVA Rescue Assembly (SAFER) Jetpack TA04-14 DRAFT though the jetpack could be augmented with fu- during which time it performs pose estimation to ture sensor payloads for specific missions. The mo- accurately determine its position and attitude rel- bility system is complete 6 axis motion control, ative to the customer. The servicer then executes with >10 m/s delta V for the combined mass of a series of maneuvers to acquire and translate the crew, suit and jetpack. The base jetpack would down a capture axis, maneuvers the robotic arms have no manipulation, but be modular so that fu- to within approximately 1 meter of the custom- ture upgrades are possible. The human interface er (placing the arms in a predefined capture box), will provide suit interaction, as well as access to and finally autonomously grasping the customer. pre recorded waypoints and system data/status. The servicer then refuels the customer or boosts The nominal mode of control will be by the crew the stack into a super-synchronous orbit (GEO + member wearing the Jetpack, but remote control 350 km) as per NASA-STD-8719.14 (Process for will be possible to rescue injured crew. Rendez- Limiting Orbital Debris). It then releases the cus- vous and docking will allow the crew member to tomer satellite and waits until the next customer is return to airlocks, suit ports and worksites with an ready for refuel or removal. in-space version of cruise control. 2.3.1.7. HST 2.3.1.5. ISS DPP The mission launches a deorbit module into The Dexterous Pointing Payload (DPP) is a low Earth orbit to rendezvous and berth with the demonstration payload that will be installed and Hubble Space Telescope at the end of the life of subsequently exercised on the ISS. In preparation the observatory and deorbits it. The grappling of for servicing missions requiring greater dexterity HST upon approach would require a closed loop and tracking capability, the DPP will demonstrate autonomous rendezvous and capture system. Af- the algorithms and control mechanisms to locate ter capture, the servicing vehicle would perform and point at a specific location on Earth or a celes- a series of maneuvers to deorbit the observatory. tial object. DPP performs attitude determination However, prior to those maneuvers, the servicer using a star tracker and an Inertial Measurement could perform servicing technology demonstra- Unit (IMU). It will receive target parameters via tions on a well understood serviceable platform commands from a ground terminal, and will send before the deorbit occurs. This would reduce risk rate requests to the ISS Robotic Workstation Soft- of future robotic servicing missions in a low risk ware (RWS) to achieve desired instrument point- environment as the telescope is being deorbited ing. This closed-loop control of Dextre (SPDM) anyway. enables real-time pointing and disturbance reduc- 2.3.2. ESMD Missions tion that is beneficial for a wide range of servicing architectures. 2.3.2.1. Near Earth Asteroid (NEA) Robotic 2.3.1.6. Geo Fuel Precursor The mission consists of a servicer spacecraft that At the time of this writing three classes of robotic can sequentially capture and control several lega- missions are being proposed as precursors to Near cy, non-cooperative satellites in nearly co-planar Earth Asteroids. These classes are a survey observa- geosynchronous orbits, refuel them, or relocate them to a disposal orbit 350 km above the GEO belt. The servicer spacecraft launches into geosyn- chronous orbit and then executes sorties to multi- ple customer satellites. At the start of the mission, the customer satellites (near the end of their mis- sion life for refuel; at the end of their life for super- sync) are on-orbit waiting for fuel or a boost to a disposal orbit. The servicer spacecraft is equipped with all hardware, algorithms and fuel necessary for supervised autonomous rendezvous and cap- ture (AR&C), and refueling or supersync of the customers. The servicer is launched and inserted directly into GEO in the plane of the first custom- er satellite. The AR&C sequence puts the servicer Figure 8. Near Earth Asteroid Rendezvous (NEAR) onto a safety ellipse about the customer spacecraft mission to Eros 1996-2001 DRAFT TA04-15 tory in space, a rendezvous mission that does not land on the asteroid, and a robotic mission to ex- plore the asteroid’s surface. NASA’s Wide-field In- frared Survey Explorer (WISE) has observed over 100,000 asteroids, only 90 of which are in the near Earth class, leaving uncertainty that a survey mission can better resolve. A robotic rendezvous with a NEA will position sensors closer to the sur- face to collect better images, record spin rate, mea- sure magnetic properties, and build a complete 3D surface map. Robotic missions that contact an asteroid’s surface will be coupled with standoff im- aging, providing the same data as the rendezvous mission but with additional surface data and the Figure 9. Concept for a Crew Transfer Vehicle ability to study dust/rock ejecta. These missions ed. If missions choose to grapple the CTV, a ma- are unlikely to be before 2015. nipulator grapple fixture can be added. Human The survey mission has little requirement for interfaces will be minimal, with launch and en- Robotics, Tele-robotics and Autonomy beyond try being highly automated functions. The Crew contemporary spacecraft engineering. The rendez- Transfer Vehicle (CTV) will require onboard au- vous mission would benefit from new mapping tonomous systems in order to achieve the reliabil- sensors and can be used as a test demonstation of ity and affordability required by the President and autonomous rendezvous algorithms and sensors at Congress. The CTV will need autonomous sys- mid-range. tems to manage the spacecraft, requiring ground Relative navigation sensors, GN&C algorithms, assistance only when a significant state change has and mission manager subsystems can be integrat- occurred which affects mission success or when a ed and demonstrated. An asteroid contact mission state change has occurred beyond the limits of the that includes controlled landing and sampling, onboard systems. Autonomous systems need to be and perhaps anchoring will challenge sensing able to minimize the need for operator assistance (depth maps, materials), perception (map sensor thereby limiting the size of ground based opera- fusion), mobility (landing), manipulation (an- tions teams and minimizing operations costs. If choring and sampling), autonomy (remote ops), a strong ground operator dependence is required and be a more complete test of rendezvous at mid then the CTV will not be able to be transported and near range down to contact. beyond the cis-lunar system as crew time will not 2.3.2.2. Crew Transfer Vehicle (CTV) be available to monitor the vehicle. The CTV will Design Reference Missions (DRM’s) produced need a modular docking interface that is compati- by the Human Exploration Framework Team ble with the MMSEV and other spacecraft. (HEFT) identified a need for a Crew Transfer Ve- 2.3.2.3. Multi Mission Space Exploration hicle (CTV) for ascent and entry capabilities (see Vehicle Figure 9). This road mapping team studied several Designed to compliment capsules used for of the HEFT DRM’s and found roles for either a launch and re entry, the MMSEV (see Figure 10) CTV or commercial capsule in all cases. The CTV is designed to provide in-space functions such as is designed to return an exploration crew of up to EVA support, habitation, and exploration of man- 4 crew members from an interplanetary trajecto- made satellites or asteroids. It will be capable of re- ry directly to the Earth (water landing). The CTV fueling, and could be tested initially with missions is based on the Orion crew module design. Ac- to ISS or HEO. The cabin will have commonal- tive duration is on the order of 36-40 hours. First ity with future surface rovers, providing multi- launch for the CTV varies from 2019 to 2023 ple applications and refinement of the technolo- across the HEFT DRM’s, or never if a commer- gy on a flexible path to Mars. The cabin is design cial capsule option is pursued. to nominally support 2 crew for 2 weeks, or more Little or no sensing, perception or manipula- crew or duration in contingency or with addition- tion is needed for the CTV. The spacecraft will al logistics modules. The cabin has suit port in- need to support autonomous rendezvous and terfaces for supporting EVA, a grappling manip- docking, where sensor technology will be need- ulator to dock or anchor on asteroid, dexterous

TA04-16 DRAFT real-time control and support from the Earth. For missions to Lagrange Points, lunar orbit and near- vicinity NEAs, robotics and autonomous systems work will focus enabling robotic capabilities to perform precursor exploration and autonomous operations in support of the Crew. The evolution in robotics and autonomous systems capability for high earth orbit will focus on crew-system auton- omy to support exploration in uncertain and dy- namic environments. 2.3.2.5. Human HEO Low-thrust, solar electric vehicles will travel from low-earth to a High Earth Orbit (HEO) car- rying cargo using autonomous guidance, naviga- Figure 10. MMSEV Concept Image tion and control to orient the electric propulsion arms for sampling and servicing functions, solar system over the continuous thrusting period. Au- arrays for power generation, an RCS system for lo- tonomy will be required to manage the complex cal motion control, and iLIDS docking interfaces spacecraft’s system state. To dock with the car- to mate with other spacecraft. go, autonomous rendezvous and docking will be Sensors include depth mapping radar, LIDAR, required at the beginning and end of the HEO and multispectral imagers. Mobility requires a full transfer. 6 axis RCS motion control system, but with limit- The Human Rated Autonomy may be essential ed delta V for only local navigation. Manipulation for Human HEO missions. Limited crew sizes requires grappling that will likely be customized and the need to abort to Earth require proactive for ISS, Satellites or asteroid missions, and small- autonomous systems to manage the vehicle. These er arms for dexterous manipulation like sampling, capabilities require unambiguous determination servicing and carrying objects. Human interfac- of vehicle states, quick response to vehicle anoma- es include cockpit controls (displays, joysticks, au- lies, and the ability to abort the crew to Earth well dio), EVA interfaces, and remote interfaces for fly- in advance of life threatening failures. Autono- ing and operating the MMSEV as an unmanned mous systems will need to implement vehicle state vehicle. Autonomy includes the full range of space- determination, diagnostics, prognostics, mission craft systems automation, as well as support for executives, and mission planning functions. Ra- flying, manipulation, EVA support, and the ren- diation hardened avionics will be necessary to en- dezvous and docking required to mate the hatch- sure the vehicle can maintain crew functions dur- es with other vehicles. The Multi Mission Space ing solar radiation events. A variety of intelligent Exploration Vehicle (MMSEV) will require on- algorithms will need to be integrated to accom- board autonomous systems in order to achieve the plish these functions. New verification and vali- reliability and affordability required by the Pres- dation methods will also be required for these hu- ident and Congress. The MMSEV will need au- man rated autonomy and automated functions. tonomous systems to manage the spacecraft, re- 2.3.2.6. HEO Utilization Flight 1 quiring ground assistance only when a significant state change has occurred which affects mission 2.3.2.7. HEO Utilization Flight 2 success or when a state change has occurred be- For flexible human mission architectures, the yond the limits of the onboard systems. Auton- crewed vehicle will require several basic on-board omous systems need to be able to minimize the capabilities in order to complete the fundamental need for operator assistance thereby limiting the scientific and technical objectives. As missions ex- size of ground based operations teams and mini- plore farther and farther from the Earth, going to mizing operations costs. Earth-Moon Lagrange points, then Sun-Earth La- 2.3.2.4. Test HEO grange points, and onward to Near Earth Objects (NEOs), the crew will need to be increasingly au- The requirements for Robotics and Autonomy tonomous (see Section 2.3.2.5) from the Earth. technologies to support near-term High Earth The crew and spacecraft systems will need to be Orbit (HEO) missions will focus on the autono- operationally autonomous from real-time ground mous capabilities necessary to operate outside of

DRAFT TA04-17 control support due to distance based communi- added to sensors and perception associated with cations delays. Proximity operations to any targets supporting crew. Mobility needs include space- (space telescopes, or small NEAs) will require au- craft 6 axis motion as well as EVA mobility for tonomous rendezvous and coupling technologies. crew in a micro gravity environment. The mission This would require significant on-board capabili- will need manipulation technology for stack as- ties beyond what has been planned for low-earth sembly, grappling and anchoring to an asteroid, orbit or even lunar missions. The crew would need EVA crew positioning, and sample handling. The to have the equivalent of flight directors on-board human interfaces will span the spectrum of Earth to support real-time operations, EVA, robotic sys- supervision, cockpit command and control, and tems, and Proximity Ops. In addition, the crew EVA suit interfaces. The crew will be required to would be a scientific vanguard to the destination operate their spacecraft far from Earth, so autono- (Lagrange Points, NEOs, etc), needing all the my will be needed to reduce the system overhead equipment and tools of any first explorers. and make the crew independent. The ability to re- 2.3.2.8. Near Earth Asteroid Human Mission configure vehicle systems to maintain the crew in The first human mission to an asteroid will chal- the event of major system losses will need to be lenge our ability to live and work in deep space, addressed. New verification and validation meth- perform EVA in space, rendezvous with distant ods will also be required for these human rated au- objects, and support an independent crew work- tonomy and automated functions. These abilities ing far from Earth (see Figure 11). The mission build on the requirements for the HEO missions. will begin with stack assembly, likely even with Multiple autonomous rendezvous and docking heavy launch capability. Trans-rock injection will steps are likely, from stack assembly to proximity be followed by a lengthy cruise phase, then inser- ops, and ranging from near earth locations to deep tion and rendezvous with the asteroid’s orbit. De- space docking. pending on the scale of the asteroid, proximity op- 2.3.2.9. Human Mars Orbit/Phobos erations may have little in common with orbiting For a Human mission to Phobos or Mars Orbit, a planet, but more like hovering next to a satellite requirements for robotics and autonomy would or other small object. Crew-centered operations involve equipment and techniques supporting re- will be the norm rather than the exception here. mote sensing, deployment/re-deployment of ro- Mission control will be in an advisory function botic surface experiment packages, and robotic due to light-time communication delays. On- surface sampling. Previous ground-based observa- board automation must exist to support the crew tions and precursor mission data should have ad- operations for the mission. The crew will perform equately characterized the surface and local space mapping and survey tasks, and then attempt to environment to reduce risk to the spacecraft and make contact with the asteroid surface with either its assets (i.e., the crew and equipment). Hence, their spacecraft or going EVA. Trans-Earth injec- the majority of spacecraft operations should be tion will be followed by a second lengthy cruise able to take place in close proximity (~a few to phase, then Earth capture and re entry. several hundred meters) to the surface of Pho- All mapping and sensing technologies developed bos. Such operations have been found to be chal- for precursors will be reused with refinement; then lenging for remotely controlled spacecraft due to round trip light delay times of several tens of sec- onds or minutes, but should be much more trac- table for the crew with humans directly in the loop. The crew and spacecraft should be able to match the rotation of Phobos, or hover over its surface, while maintaining a stable attitude from which they can conduct a detailed scientific ex- ploration of the surface. This capability will ide- ally have been validated during a crewed NEO mission previously. Proximity operations and au- tomated rendezvous technologies will be critical. Additional autonomous systems technology needs become more demanding with missions to Mars orbit, Phobos, Deimos, more remote NEAs and Figure 11. Concepts for a Human Asteroid Mission TA04-18 DRAFT Venus orbit. Automation to support cryogenic struments, biomedical instruments, and sensors to fluid management for long-duration transfer and support navigation and mobility will be required management of liquid hydrogen, other propel- with redundancy in numbers and type. Mobili- lants and life support fluids, and ECLSS systems ty will include in-space flying, surface roving, and will be needed as well. Autonomy and other func- EVA mobility. The crew will be far from Earth and tions will build upon capabilities cited as essential will need interfaces to all systems, and those sys- for the previous missions to HEO and asteroids. tems will be highly autonomous to avoid consum- 2.3.2.10. Human Mars Mission ing crew time. Autonomous systems will need to There is much debate about the path to Mars, integrate across vehicle stages or platforms. The but general agreement that Mars is the ultimate Human Mars vehicle will have a complicated con- destination for humans in the inner solar system figuration including aerocapture, landers, and as- (see Figure 12). Volumes have been written on cent stage recovery. The mission will involve com- Mars mission architectures, and our science mis- plex stack assembly with manipulation, grappling, sions have greatly informed our plans with knowl- rendezvous and docking, EVA/robotic assembly, edge of the surface and experience in landing, and must be able to conduct those operations ei- operating and exploring. Standout differences be- ther near Earth, in deep space, Mars orbit, or on tween a human mission to Mars and our previ- the surface. ous experiences are the long duration of the mis- 2.3.3. SMD Missions sion for crew, the combination of zero gravity and 2.3.3.1. Mars Science Laboratory (MSL)/ reduced gravity, the large scale of entry and de- Extended scent vehicles, and long surface stay and mobili- Extended phases of the MSL mission are oppor- ty range required. A human mission to Mars may tunities for insertion and testing of software up- be preceded by missions to near Mars, its moons grades representing technology push items. Exam- or Mars orbit. ples of potential new onboard capabilities include Pre-deployed robotic assets could potentially be faster implementations of visual odometry algo- used to help produce propellant from the massive rithms for slip estimation, onboard visual terrain amounts of sub-surface water ice that are expect- classification for improved path planning, esti- ed to be present. By excavating water ice from the mating parameters of soil mechanics models for regolith, oxygen and hydrogen propellant can be improved trafficability analysis, automated mid- produced by electrolysis without transporting hy- sol and end-of-sol position estimation using or- drogen to Mars, as is needed for the production of bital imagery, automated instrument pointing, Methane. Landing pads may be robotically pre- science operations while driving, and automat- pared to reduce the risk of a bad landing. Robot- ed site survey and downlink of site maps annotat- ic assistants can connect the crew lander to a pow- ed with science observations. Some functions cur- er plant by deploying and mating a power cable rent performed on the ground could be migrated to the power plant before the lander runs out of onboard, including motion planning and colli- stored power. The robots will use sunlight energy sion checking for the sampling arm. Since MSL for solar electric propulsion, and then the humans is powered by RTGs, not solar panels, new oper- will use the propellants that are produced by the ational modes could include driving in the dark robotic mining systems. This is a good example of through terrain determined in advance to be free enabling human-robotic exploration. of obstacles by examination of onboard and orbit- Human missions generate massive amounts of al sensor data. data, and humans augmented with sensors are now Technology push opportunities exist for upload- the baseline for exploration. Mapping, science in- ing new perception software, that while utilizing the existing sensors, will expand capabilities and productivity per sol. Software upgrades can also provide for improved mobility and manipulation performance. Autonomy upgrades include more efficient data handling, and fault detection and re- covery.

Figure 12. Human Mars Mission Concepts DRAFT TA04-19 2.3.3.2. Mars Sample Return (MSR) Mission 1 biter would be sent nominally two opportunities 2.3.3.3. MSR 2 (four years) later. It is projected to launch before the MSR Lander, so that it could provide telecom- 2.3.3.4. MSR 3 munications infrastructure for the lander and its A definitive answer to whether there is or has fetch rover and Mars Ascent Vehicle (MAV). In been life on Mars or, if not, why not, requires re- the next opportunity (two years later), the MSR turn of carefully selected samples from one or Lander would be sent, also using an MSL-style more well-characterized, high-priority sites. Anal- EDL system to get the lander platform, includ- ysis of returned samples allows measurements us- ing the MAV, to the surface. The lander would ing complex analytical techniques (i.e., occupying dispatch a fetch rover to retrieve a sample cache large laboratories), provides necessary opportu- previously deposited on the surface by the 2018 nities for follow-up measurements, and enables NASA Caching Rover. The cache would be aug- subsequent analyses using techniques not yet de- mented by a lander-collected sample and insert- veloped at the time of sample return. Properly in- ed into the OS that would be launched into a 500 terpreting evidence related to life requires mul- km orbit by the MAV. The orbiter—having mon- tiple approaches, and it is not possible to select itored the launch and release of the OS—would discrete and unique criteria ahead of time. An- rendezvous and capture the OS. On the orbit- swers will come only through multiple analyses er the process of “breaking the chain of contact” of returned samples. Analysis of returned samples with Mars would take place, sealing the OS into would also contribute to most disciplines at Mars an Earth Entry Vehicle (EEV). The orbiter would and is necessary for advancing our understanding then return to Earth, release the EEV a few hours of many of them, including through comparison before entry, and divert into a non-Earth return with Earth. There is high relevance to topics in- trajectory. Because of Planetary Protection re- cluding planetary formation, geophysical evolu- quirements, the EEV seals would have to be ver- tion, surface geology, climate and climate history ified before targeting the Earth. The EEV would of all the terrestrial planets. hard land on the surface and then be transferred Sample return is also thought to be a necessary to a secure SRF for quarantine before samples are step along the path toward potential human mis- extracted. sions to Mars, in order to understand the environ- All three MSR missions have a broad set of tech- ment prior to human arrival. nology pull opportunities. MSR-1 Has potential- The proposed MSR would be a campaign of ly the most complex manipulation and mobility three missions: requirements of any mission yet attempted. This 1. 2018: sample caching mission, which would robot will need sensing and perception to assist cache rock cores for later retrieval. Earth science teams and augment their visualiza- tion of the geologic units and transitions. Cou- 2. 2022: MSR Orbiter Mission, which would pled with a long term mission life, the robot will augment the planetary communications be responsible for long term cache management. network and return the orbiting sample MSR-3 will perform the first Automated Rendez- container (OS) to the Earth’s surface after vous and Docking task ever attempted on the sur- 2024. face. 3. 2024: MSR Lander Mission, which would MSR Key Requirements retrieve the sample and places it in Mars orbit Numerous science advisory groups have met in an orbiting sample container (OS). over the past decade to define proposed science A fourth component is the Mars Returned Sam- objectives for MSR and to address the balance of ple Handling element that would include a Sam- objectives and mission difficulty and cost. While ple Receiving Facility (SRF) and a curation facil- the science would be ultimately performed in lab- ity. oratories here on Earth, the following goals reflect The campaign would entail three launches. The the latest thinking on the MSR missions. current baseline for the first mission is the Joint • Return >500 g of sample consisting of: NASA/ESA Mars 2018 mission, which would use »»1. Rock cores from multiple geological units a Mars Science Laboratory (MSL)-style entry, de- »»2. Regolith from a single location, but scent, and landing (EDL) system to land both a potentially from multiple locations NASA Caching Rover and the ESA ExoMars Rov- er on a single platform. The proposed MSR Or- »»3. A compressed atmospheric sample TA04-20 DRAFT • Use a suite of in situ instrumentation to the return trip at ≤ –10°C. After SRV recovery, the carefully select coring targets and document samples will be transferred to Johnson Space Cen- context of the cores ter astromaterials analytical laboratories that will • Minimize organic and inorganic contamination have been upgraded with capabilities to store, an- • Package samples to minimize cross alyze and characterize frozen samples. The reliance contamination and sample alteration (which on heritage spacecraft design wherever possible is might or might not include hermetic sealing) intended to minimize risk. The following critical • Maintain temperature control of samples to technologies will require development to Technol- <20°C (except potentially higher for a short ogy Readiness Level (TRL) 6: period after landing at Earth) • Ballistic-type sample return vehicle (SRV) will require new mobility and control technologies. 2.3.3.5. Comet Surface Sample Return (CSSR) • UltraFlex solar array Sample Acquisition System The fundamental CSSR mission scientific objec- (SAS) will require new sensing technologies. tives are as follows: • Height and Motion System (H&MS) will • Acquire and return to Earth for laboratory require manipulation, sensing and control analysis a macroscopic (at least 500 cc) sample technologies. from the surface of the nucleus of any comet. • Collect the sample using a “soft” technique 2.3.3.6. Comet Nucleus Sample Return that preserves complex organics. (CSNR) • Do not allow aqueous alteration of the sample SMD will propose a technological development at any time. program to enable a CNSR mission in the subse- quent decade (2021–2030). This Technology De- • Characterize the region sampled on the surface velopment Program will address technology needs of the nucleus to establish its context. for CNSR, mitigate mission development risks, • Analyze the sample using state-of-the-art and verify promising technologies and mission laboratory techniques to determine the nature concepts via a test and evaluation program. and complexity of cometary matter, thereby The overriding objective is to provide assurance providing fundamental advances in our that the key CNSR-required technologies can all understanding of the origin of the solar system be raised to at least Technology Readiness Level 5 and the contribution of comets to the volatile (TRL 5, full-scale prototype testing) in the com- inventory of the Earth. ing decade. The baseline CSSR mission scientific objectives It is assumed that by the time a CNSR mis- will also provide revolutionary advances in come- sion is launched, a Comet Surface Sample Return tary science: (CSSR) mission will have been accomplished and • Capture gases evolved from the sample, will have demonstrated how to obtain a surface maintaining their elemental and molecular sample. The primary interest for the CNSR mis- integrity, and use isotopic abundances of the sion is to obtain a sample at depth(s) from beneath gases to determine whether comets supplied the surface layer and to maintain it cold enough to much of the Earth’s volatile inventory, return material to the Earth in the ice phase. including water. The following set of top-level science goals is as- • Return material from a depth of at least 10 cm sumed for the mission study with highlighted tech (at least 3 diurnal thermal skin depths), if the opportunities: sampled region has shear strength no greater • Floor: Return one sample from a single site, than 50 kPa, thereby probing compositional with water ice and less volatile organics intact variation with depth below the surface. (i.e., no water ice melting or loss of “moderately • Determine whether the sample is from an volatile” species to vacuum). [manipulation active region of the nucleus because those areas and control technology] may differ in composition from inactive areas. • Baseline: Return one sample from a single After the mission spacecraft travels to Comet site, with >20% water ice by mass, with 67P/C-G and collects images to characterize the water ice, most volatile organics preserved, comet’s nucleus, a sample return vehicle (SRV) and stratigraphy intact. (It is noted that the will return ≥500 cc of material to Earth for labora- preservation of stratigraphy is highly desired, tory analysis. The payload will collect the samples but it is recognized to be difficult to achieve). using 4 drills. Samples will be maintained during [sensing, perception and manipulation DRAFT TA04-21 technology] descent and at the surface. While collecting data • Desired: Return up to several kilograms of at the first site, the bellows are filled with helium samples from multiple sites on the nucleus, and when buoyant, rise with the gondola, leav- with stratigraphy and all ices intact, and no ing the helium pressure tank on the surface. Driv- cross-contamination of collected samples. en by the ambient winds, the gondola floats with [sensing, perception, manipulation and the bellows for ~220 minutes, conducting addi- mobility technology] tional science. At the completion of the 8–16 km aerial traverse, the bellows are jettisoned and the 2.3.3.7. Venus Mobile Explorer (VME, aka gondola free falls back to the surface, where final Venus Aerobot) surface science measurements are performed. The The Venus Mobile Explorer (VME) mission total mission time in the Venus atmosphere is 6 concept affords unique science opportunities and hours, which includes 5 hours in the near surface vantage points not previously attainable at Venus. environment. The VME probe transmits data to The ability to characterize the surface composition the flyby carrier spacecraft continuously through- and mineralogy in two locations within the Ve- out the 6-hour science mission. After losing con- nus highlands (or volcanic regions) will provide tact with the VME probe, the carrier spacecraft essential new constraints on the origin of crustal then relays all data back to Earth. material, the history of water in Venus’ past, and This mission represents a completely new ap- the variability of the surface composition within proach to surface exploration. Technology needs the unexplored Venusian highlands. As the VME span sensing, perception, mobility, sample manip- floats (~3 km above the surface) between the two ulation, and spacecraft autonomy. surface locations, it offers new, high spatial reso- lution, views of the surface at near infrared (IR) 2.3.3.8. Titan Aerobot wavelengths. These data provide insights into the A mission launched in the 2018–2022 time- processes that have contributed to the evolution frame would provide a unique opportunity to of the Venus surface. The science objectives are measure a seasonal phase complementary to that achieved by a nominal payload that conducts in observed by Voyager and by Cassini, including its situ measurements of noble and trace gases in the extended missions. atmosphere, conducts elemental chemistry and Recent discoveries of the complex interactions mineralogy at two surface locations separated by of Titan’s atmosphere with the surface, interior, ~8–16 km, images the surface on descent and and space environment demand focused and en- along the airborne traverse connecting the two during observation over a range of temporal and surface locations, measures physical attributes of spatial scales. The Titan Saturn System Mission the atmosphere, and detects potential signatures (TSSM) two-year orbital mission at Titan would of a crustal dipole magnetic field. sample the diverse and dynamic conditions in the The VME design includes an elegant, volume ionosphere where complex organic chemistry be- efficient cylindrical gondola to accommodate the gins, observe seasonal changes in the atmosphere, science payload in a thermally controlled environ- and make global near-infrared and radar altimet- ment. An innovative, highly compact design sur- ric maps of the surface. This study of Titan from rounds the gondola with a toroidal pressure tank orbit with better instruments has the potential of capped with the bellows, enabling the entire land- achieving a 2–3 order-of-magnitude increase in er system to fit in an aeroshell with heritage geom- Titan science return over that of the Cassini mis- etry. The thermal design uses heat pipes and phase sion. Chemical processes begin in Titan’s upper at- change material that enable the gondola electron- mosphere and could be extensively sampled by an ics and instruments to survive 5 hours near the orbiting spacecraft alone. However, there is sub- Venus surface, thus providing sufficient time for stantial additional benefit of extending the mea- surface chemistry and an aerial traverse >8 km in surements to Titan’s lower atmosphere and the the current- like winds. surface. Titan’s surface may replicate key steps to- Launched on an Atlas V 551 in either 2021 or ward the synthesis of prebiotic molecules that may 2023, the carrier spacecraft carries the VME probe have been present on the early Earth as precursors to Venus on a Type II trajectory. After release to life. In situ chemical analysis, both in the atmo- from the carrier, the VME probe enters the atmo- sphere and on the surface, would enable the as- sphere, descends on a parachute briefly, and then sessment of the kinds of chemical species that are free-falls to the surface. Science is conducted on present on the surface and of how far such pu-

TA04-22 DRAFT tative reactions have advanced. The rich invento- ditional missions that will provide both push and ry of complex organic molecules that are known pull technology opportunities for robotics, tele- or suspected to be present at the surface makes robotics and autonomous systems technology. new astrobiological insights inevitable. In situ el- These include: ements also enable powerful techniques such as • Europa Lander subsurface sounding to be applied to exploring Ti- • Venus Sample Return tan’s interior structure. Understanding the forces that shape Titan’s diverse landscape benefits from • ATLAST detailed investigations of various terrain types at • 30m Space Telescope different locations, a demanding requirement any- 2.3.4. ARMD Missions where else, but one that is uniquely straightfor- ward at Titan using a Montgolfière hot-air bal- 2.3.4.1. Small UAV loon. TSSM’s Montgolfière could circumnavigate Small UAV Unmanned Aerial Vehicles (UAVs) Titan carried by winds, exploring with high reso- are aircraft that are either fully- or semi-autono- lution cameras and subsurface-probing radar. The mous (mixed initiative) robotic vehicles. UAVs combination of orbiting and in situ elements is are fundamentally similar in concept to space- a powerful and, for Titan, unprecedented oppor- craft. They are used for NASA science missions tunity for synergistic investigations— synthesis of in uncertain and rapidly changing environments, data from these carefully selected instrumentation and they drive much the same set of autonomy suites is the path to understanding this profound- requirements (pilot automated monitoring, diag- ly complex body. nosis, planning and execution, reliable software, The flight elements would be launched onan and advanced controls) that NASA finds in in- Atlas V 551 launch vehicle in 2020 using a gravi- ter-planetary exploration missions. Additionally, ty-assist SEP trajectory to achieve a trip time of 9 they can naturally be used to investigate issues in years to Saturn. Following Saturn orbit insertion, multi-agent cooperation (e.g., for surveillance by the orbiter would conduct a Saturn system tour, fleets of UAVs, or planetary robots) that NASA including 7 close Enceladus flybys and 16 Titan will need to solve for various future missions. flybys. This phase would allow excellent opportu- 2.3.4.2. Wind Turbines nities to observe Saturn, multiple icy moons and Many challenges exist for the efficient and safe the complex interaction between Titan and Sat- operation of wind turbines due to the difficulty in urn’s magnetosphere. The Montgolfière would be creating accurate models of their dynamic char- released on the first Titan flyby, after Saturn or- acteristics and the turbulent conditions in which bit insertion, and would use an X-band relay link they operate. A promising new area of wind tur- with the orbiter for communications. The lander bine research is the application of adaptive con- would be released on the second Titan flyby and trol techniques, which are well suited to problems communicate with the orbiter during the flyby where the system model is not well known and the only. This 24-month period would also mark the operating conditions are unpredictable. mission phase when all of the Titan in situ data is relayed back to Earth. Following its tour of the 2.3.4.3. Wildfire UAV Saturn system, the orbiter would enter into a high- Full automation of the Wildfire UAV system ly elliptical Titan orbit to conduct a two-month will enable free-flight within the National Air- concurrent Aerosampling and Aerobraking Phase space. This included the autonomous filing of in Titan’s atmosphere, sampling altitudes as low flight plans and execution of the same, based upon as 600 km. The orbiter would then execute a fi- satellite sensor data indicating where fires exist in a nal periapsis raise burn to achieve a 1500-km cir- geographic area. On-board system health manage- cular, 85° polar-mapping orbit. This Circular Or- ment and adaptive control will enable the UAV to bit Phase would last 20 months. operate in degraded modes. On-board data anal- This mission represents a completely new ap- ysis and science discovery will permit the UAV to proach to surface exploration. Technology needs report operational fire targets to ground firefight- span sensing, perception, mobility, sample manip- ers and to support remote sensing of fire progress. ulation, and spacecraft autonomy. 2.3.4.4. Air Cargo 2.3.3.9. Additional SMD Missions Advanced adaptive control technologies and sys- The RTA panel is continuing to investigate ad- tem state monitoring and management capabili- ties will be required to enable real-time feathering, DRAFT TA04-23 engine control, and system health management of tion applications. Major challenges include cali- variable speed rotorcraft operations. V&V of the bration of highly dissimilar sensors, dissimilar res- flight critical systems will also be necessary. olution, noise, and 1st principles of physics in the development of new sensors. 3. Conclusions Achieving human-like performance for pilot- 3.1. Top Technical Challenges ing vehicles The RTA panel identified multiple top techni- Machine systems have the potential to outper- cal challenges, and these will be described in or- form humans in endurance, response time and der of their associated location in the WBS, not a number of machines that can be controlled simul- particular priority. Each represents the top priori- taneously. Humans have safety limits on flight or ty within its WBS sub topic. drive-time that do not exist in machines. Human response time, coupled with human machine in- Object Recognition and Pose Estimation terfaces, results in significant delays when faced Object recognition requires sensing, often fus- with emergency conditions. Humans are poor at ing multiple sensing modalities, with a percep- parallel processing the data and command cycles tion function that can associate the sensed object of more than a single system. But machines are with an object that is understood a priori. Sens- currently far behind humans in handling extreme- ing approaches to date have combined machine ly rare cases, improvising solutions to new condi- vision, stereo vision, LIDAR, structured light, and tions never anticipated, and learning new skills on RADAR. Perception approaches often start with the fly. CAD models or models created by a scan with the same sensors that will be used to identify the ob- Access to extreme terrain in zero, micro and ject later. Pose estimation seeks to locate an object reduced gravity relative to a sensor coordinate frame, computing Current crew rovers cannot access extreme Lu- the six axis pose using sensing data. Pose estima- nar or Martian terrain, requiring humans to park tion is often preceded by object recognition, or and travel on foot in suits. In micro gravity, loco- presumes an object so that its pose can be estimat- motion techniques on or near asteroids and com- ed and tracked. There is a special case of identify- ets are undeveloped and untested. Access to com- ing humans as objects of interest, tracking human plex space structures like the ISS is limited to motion, gestures, and doing human recognition. climbing or positioning with the SSRMS. Chal- Major challenges include the ability to work with lenges include developing robots to travel into a large “library” of known objects (>100), identi- these otherwise denied areas, or building crew fying objects that are partially occluded, sensing in mobility systems to move humans into these chal- poor (high, low and sharply contrasting) lighting, lenging locations. estimating the pose of quickly tumbling objects, Grappling and anchoring to asteroids and non and working with objects at near and far range. cooperating objects This technology is important for object manip- Grappling an object in space requires a manipu- ulation and in mobility for object following and lator or docking mechanisms that form a bi direc- avoidance. Human tracking is important in ma- tional 6 axis grasp. Grappling an asteroid and then nipulation for safely working with human team- anchoring to it is an all-new technology. Grap- mates, and in mobility for avoiding collisions with pling approaches attempted on man- made ob- pedestrians. jects may not apply to asteroids, since these tech- Fusing vision, tactile and force control for niques count on specific features such as engine manipulation bells that will not be available on a natural object. The field of mobile robotics has matured with Similarly, grappling an object that is tumbling has the advance of safe, fast and deterministic motion not been attempted. control. This success has come from fusing many Exceeding human-like dexterous manipulation sensors to avoid contacting hazards. Manipulation The human hand is generally capable. A robot- requires forming contact, so the breadth of sens- ic equivalent, or superior, grasping ability would ing will require proximity, then tactile, and ulti- avoid the added complexity of robot interfaces mately force sensing to reach, grasp and use ob- on objects, and provide a sensate tool change-out jects like tools. Vision requires sensors that are not capability for specialized tasks. Dexterity can be blocked when limbs reach for objects, but that can measured by range of grasp types, scale, strength be pointed and transported for mobile manipula- and reliability. Challenges include fundamental TA04-24 DRAFT 1st principles of physics in the development of ac- tion approaches to overlay predicted, committed tuation and sensing. Other challenges include 2 and commanded states, and the ability to work point discrimination, contact localization, extrin- ahead of real-time. sic and intrinsic actuation, backdrivability vs com- Rendezvous, proximity operations and dock- pliance, speed/strength/power, hand/glove cover- ing in extreme conditions ings that do not attenuate sensors/motion but are Rendezvous missions include flybys of destina- rugged when handling rough and sharp objects. tions without landing or docking. Proximity oper- Full immersion, telepresence with haptic and ations require loiter at destinations with zero rela- multi modal sensor feedback tive velocity. Docking drives latching mechanisms Telepresence is the condition of a human feel- and electrical/fluid couplings into a mated condi- ing they are physically at a remote site where a tion. Major challenges include the ability to ren- robot is working. Technologies that can contrib- dezvous and dock in all ranges of lighting, work ute to this condition include fully immersive dis- across near to far range, and achieve a docked state plays, sound, touch and even smell. Challenges in all cases. include 1st principles of physics in the develop- Mobile manipulation that is safe for working ment of systems that can apply forces to human with and near humans fingers, displays that can be endured for long pe- Merging manipulative capabilities with gener- riods of telepresence immersion, and systems that al mobility is sought to allow robots to go to the can be used by people while walking or working work site, rather than require the work be deliv- with equipment concurrently with the telepres- ered to the robot. Manipulator arms and mobili- ence tasks. ty drives each pose hazards to people. Combined, Understanding and expressing intent between they present many risks. Challenges include track- humans and robots ing humans in the workspace, responding deter- Autonomous robots have complex logical states, ministically to inadvertent contact, compliance, control modes, and conditions. These states are and providing redundant sensor and software sys- not easily understood or anticipated by humans tems. working with the machines. Lights and sounds 3.2. Overlap with other Technical Areas. are helpful in giving cues as to state, but need to Table 1 summarizes overlaps that have been iden- be augmented with socially acceptable behaviors tified with between TA4 and the other technology that do not require advanced training to interpret. areas. These have been sorted into the two class- Likewise, robots have difficulty in understanding es of either being technologies needed by TA4, or human intent through gesture, gaze direction or technologies from TA4 needed by the other area. other expressions of the human’s planned behav- ior. 3.3. Summary of Findings for Robotics, Verification of Autonomous Systems Tele-Robotics and Autonomous Large software projects have such complex soft- Systems ware that exhaustive and manual exploration of all 1) NASA’s four Mission Directorates are possible cases is not feasible. Human rated auton- depending on Robotics, Tele-Robotics and omous systems are particularly challenging. Veri- Autonomy Technology. fication techniques are needed to more fully con- 2) Technology should aim to exceed human firm system behavior in all conditions. performance in sensing, piloting, driving, Supervised autonomy of force/contact tasks manipulating, rendezvous and docking. across time delay Tasks have time constants that vary greatly, with 3) Technology should target cooperative and safe the shortest time constants involving motions that human interfaces to form human-robot teams. form contacts with the environment and force 4) Autonomy should make human crews controlled actions. These tasks require high speed independent from Earth and robotic missions local control loops. As time delays approach these more capable. tasks time constants the ability to tele-operate the . machine degrades. Supervision is the manage- ment of a robot with autonomous skills, working along a sequence of tasks. Challenges include run time simulation to predict future states, visualiza-

DRAFT TA04-25 Table 1. Overlap with other Technical Areas showing technologies needed by TA4 and technologies from TA4 needed by the other area.

Acronyms AERCam Autonomous EVA Robotic Camera LEO Low Earth Orbit ARC Ames Research Center LIDAR Light Detection and Ranging AR&D Autonomous Rendezvous and Docking LIDS Low Impact Docking System ARMD Aeronautics Research Mission Directorate MER Mars Exploration Rover ATP Authority to proceed MMU Manned Maneuvering Unit ATV Autonomous Transfer Vehicle MSL Mars Science Laboratory CTV Crew Transfer Vehicle MSR Mars Sample Return ESMD Exploration Systems Mission Directorate MMSEV Multi Mission Space Exploration EVA Extra Vehicular Activity Vehicle DPP Dexterous Pointing Payload NEA Near Earth Asteroid DRM Design Reference Mission NEAR Near Earth Asteroid Rendezvous FPGA Field Programmable Gate Array NEO Near Earth Object FSW Flight Software NRE Non recoverable Engineering GEO Geosynchronous Earth Orbit OCT NASA’s Office of the Chief Technologist GN&C Guidance, Navigations and Control ORU Orbital Replacement Unit GSFC Goddard Space Flight Center OTCM ORU Tool Changeout Mechanism HEFT Human Exploration Framework Team R2 Robonaut 2 HEO High Earth Orbit R2D2 Robotic Refueling Dexterous Demonstration HST Hubble Space Telescope RMCT Robotic Micro Conical Tool HTV H-II Transfer Vehicle RTAs Robotics, Tele-Robotics and Autonomous ILIDS International Low Impact Docking System systems ISS International Space Station RWS Robotics Work Station IVA Intra Vehicular Activity SAFER Simplified Aid for EVA Rescue JPL Jet Propulsion Laboratory SMD Science Mission Directorate JSC Johnson Space Center

TA04-26 DRAFT SOMD Space Operations Mission Directorate SPDM Special Purpose Dexterous Manipulator SRMS Shuttle Remote Manipulator System SSRMS Space Stations Remote Manipulator System TA Technology Area TRL Technology Readiness Level TSSM Titan Saturn System Mission UAV Unmanned Aerial Vehicle V&V Verification and Validation WBS Work Breakdown Structure

Acknowledgements The NASA technology area draft roadmaps were developed with the support and guidance from the Office of the Chief Technologist. In addition to the primary authors, major contributors for the TA04 roadmap included the OCT TA04 Road- mapping POC, Maria Bualat; the reviewers pro- vided by the NASA Center Chief Technologists and NASA Mission Directorate representatives, and the following individuals: Brittany Kimball, David Dannemiller, Sharada Vitalpur, Jack Braz- zel, and Mimi Aung.

DRAFT TA04-27 November 2010

National Aeronautics and Space Administration

NASA Headquarters Washington, DC 20546 www.nasa.gov TA04-28 DRAFT