“Exploration Telepresence: Value and Challenges”

Dan Lester University of Texas

FISO Telecon

May 27, 2015 Human cognition makes use of “presence” in distant places. Cell phone, Skype, etc.

Once achievable only with human bodies. Congress endorses human “presence” in spaceflight.

Telepresence!

The cognitive mandate for telepresence depends on

• control latency • control bandwidth

Telepresence and human space flight as enablers for space exploration.

Human-robot collaboration

FISO Telecon 5/27/15 2 The idea of space telepresence was considered in some detail at the Exploration Telerobotics Symposium, May 2-3, 2012 at GSFC.

This symposium explored

Space Exploration via Telepresence: A New Paradigm for Human-Robotic Cooperation

See presentations and final report at http://telerobotics.gsfc.nasa.gov

FISO Telecon 5/27/15 3 Recipe for Telepresence

Control bandwidth for quality telepresence? • Human vision ~10 Mb/s per eye • A bit more with haptics, dexterity, and audio

(with compression, and some generosity, letʼs say 1 Mb/s)

Control latency for quality telepresence? • Threshold for latency detection is ~50 ms but … • Human reaction time for visual stimulus is ~200 ms

That defines a “cognitive horizon” of c * 200 ms / 2 ~ 30,000 km

But everywhere we want to go is way beyond 30,000 km from Earth!

• Earth-Moon 2.6 seconds • Earth-Mars 8-40 minutes

WE CANʼT HAVE “QUALITY TELEPRESENCE” AT THESE LOCATIONS FROM THE EARTH. BUT WE CAN IF WE GET CLOSE.

FISO Telecon 5/27/15 4 But letʼs talk about latency

Dave Scott’s Apollo 15 stopwatch

FISO Telecon 5/27/15 5 Whatʼs wrong with a little control latency, anyway?

For driving around, and avoiding obstacles, not much. (10-20s latency endured by Lunokhod controllers made driving around hard, but they did it)

Few careful assessments of the latency penalty on cognition for >1 second.

Oscillopscia (perceptual instability relative to inertial and gravitational frame of reference) and dynamic head-tracking errors. Latency screws with your brain.

Allison et al. (2001, 2004) assessed 22.5 deg/s perceived stability by a user with a stereo head-mounted display while panning head. 45 deg/s “Additional tracking delay” (abcissa) is the added latency between the motion of their head, and the motion of the image.

100ms two-way latency isnʼt that bad. 90 deg/s Just go slow.

FISO Telecon 5/27/15 6 What’s wrong with a little control latency, if we’re trying to do something? Well, it depends on what you’re trying to do.

If youʼre just trying to guide a 7 DOF arm to do a positioning task, itʼs not that hard, but there are latency penalties.

Lane 2001 saw that the required completion time using an “unmitigated” situation, where the user simply saw what was going on, rises steeply with latency.

It helps if you have predictive assistance, and also if you have a graphical indication of what you commanded.

FISO Telecon 5/27/15 7 What’s wrong with a little control latency, if we’re trying to do something? It depends on how hard it is.

Piece of telerobotic history -- work of Bill Ferrell, back in the 1960s.

Simple 2D “minimal manipulator”.

Task completion time versus latency and “difficulty”.

“Difficulty” defined as how far you had to move, divided by how precisely you got there.

Not surprisingly task completion time depends critically on latency and difficulty.

FISO Telecon 5/27/15 8 Whatʼs wrong with a little control latency, if weʼre trying to do something? How about doing really hard stuff?

Telerobotic surgery is exquisitely difficult. High-dexterity w/ precision cutting and stitching of compliant material. Low error tolerance.

A LOT of work assessing latency penalty. 500ms is considered practical limit.

Assessment by Rayman 2006 – task time for average of four laproscopic surgery manipulations.

Things get bad fast beyond 500ms.

Errors also go up quickly.

Lesson of telerobotic surgery is for high dexterity operations, 500ms is a practical limit for latency.

FISO Telecon 5/27/15 9 … and then there is easy stuff that is hard (unlike surgery)

The NEEMO crew experimented with undersea habitat-to-shore telerobotics, and found that tying a shoelace with Earth-Moon latency (2.6 seconds) took almost ten minutes.

FISO Telecon 5/27/15 10 What’s wrong with control latency, if we’re trying to do something? What “hard stuff” are we talking about?

Picking up rocks is hard with latency. Different sizes, random shapes. Turning over to inspect.

Assembly and maintenance is hard with latency. Drilling, digging, stacking and cabling.

In general, dexterity is hard with latency. Situation awareness as well.

Haptic feedback extremely intolerant of latency.

For field geology, we have NO experience with low latency telerobotics.

Lessons from high latency telerobotics (MER, MSL) donʼt necessarily transfer well to low latency telerobotics.

Analog studies on the Earth will be essential to this understanding.

Latency makes life HARD. So letʼs do something about it. (But landing humans in gravity wells is hard too.)

FISO Telecon 5/27/15 11 So what are we talking about? On-orbit telerobotics. Human “presence” from a (short) distance away. LOW LATENCY. (images from Boeing, GSFC, etc.) FISO Telecon 5/27/15 12 Controlling telerobots from space microgravity may pose some challenges …

Human factors are an important element of exploration telepresence.

• Sensorimotor deficits in eye-hand coordination in microgravity, “space-fog”? • Cognitive complication of control in a gravity field by an astronaut in microgravity. • Space habitat fatigue and work overload on cognitive functionality. • Teleoperation control station design in microgravity – display screens, dexterous manipulators. • Candidate screening & skill assessment.

HARI – NASA ARC/JSC Otmar Bock/DLR Chuck Oman/MIT PMDIS - Barry Fowler/York U.

Doing telerobotics in space is serious issue for exploration telepresence. ISS probably at least as important for assessing human factors as technical factors.

(Canadarm/DEXTRE not that relevant anymore.)

FISO Telecon 5/27/15 13 6 Crew Human Exploration Using Real-time Robotic Operations (HERRO)

Schmidt, Oleson, Landis et al. Glenn Research Center

FISO Telecon 5/27/15 14 Low latency telerobotic control process?

Undersea oil, gas, cable

Drones/UAVs Mining Lessons about telepresence architecture from terrestrial applications. Surgery

FISO Telecon 5/27/15 15 Evolution of space telepresence

Lunar workstation at Lunokhod control from Earth cis-lunar habitat - latency ~few x 100 ms (control latency ~5-10 seconds)

Space "##! telepresence ISS Remote enabled Workstation control latency ~ ms)

Boeing FISO Telecon 5/27/15 16 Low Latency Telerobotics - not a new concept for space exploration

Paine Report (1986) “Pioneering the Space Frontier” "We recommend that: NASA explore the limits of expert systems, and tele-presence or tele-science for remote operations including ties to spacecraft and ground laboratories. In working toward these goals, a broad examination of the non-space applications of tele-science should be included.”

“Telepresence: The use of real-time video communications coupled with remote control techniques which would provide an operator on Earth's surface or other location with the capability to carry out complex operations in space or on the surface of a planet or moon.”

Stafford Report (1991) “America at the Threshold: America's Space Exploration Initiative” “ ʻTelepresenceʼ robots can conduct some geologic field work. While such a technique might greatly enhance the scientific return, the details of how such robots might work with people remain to be developed. Operators for these telepresence robots need near-instant radio contact with the robots. This may be marginally obtainable by having the controllers on the Earth, but operators on or near the Moon have a near-zero time lag for robotic teleoperations.”

and more recently… (including HERRO!)

Mars DRM5 (2009) “Low-Latency telerobotics operation may be a useful strategy, particularly if human missions stay out of gravity wells for some time.”

Augustine Commission (2009) “Review of United States Human Space Flight Plans Committee” “The goal focuses human exploration on producing exciting new science at each step of the way. The emphasis would be on obtaining multi-kilogram samples from a variety of solar system bodies through tele-robotic exploration in concert with The human missions. In the case of the Moon and Mars, humans would remain in orbit.”

“A Phobos-based teleoperated exploration of the Martian surface, returning with samples from that surface, would likely precede a crewed Mars landing mission, and would provide dramatically more responsive remote control than with the communication delays incurred between Mars and Earth.”

NASA Technology Roadmap TA04 (2012) ”Top technical challenges in human-robot interfaces are full immersion telepresence with haptic, multi sensor feedback, understanding and expressing intent between humans and robots, and supervised autonomy of dynamic/contact tasks across time delay.” FISO Telecon 5/27/15 17 and most recently …

Human Spaceflight Architecture Team (HAT) DRM 8a (2013) Mars Orbital Mission (crew tele-operates surface assets)

Global Exploration Roadmap (2013) “New mission concepts, such as human-assisted sample return and tele-presence should be further explored, increasing understanding of the important role of humans in space for achieving common goals.” “Human-assisted sample return and tele-presence represent new and integrated approaches to space exploration with the potential to increase benefits.” “Tele-presence is a capability which could significantly enhance the ability of humans and robots to explore together, where the specific exploration tasks would benefit from this capability.”

Evolvable Mars Campaign (2015) “Exploration Augmentation Module: Tele-operations in space and on lunar surface”, report in progress

Humans Orbiting Mars workshop (2015) Teleoperation of Mars surface assets is presumed, but report in progress

Somewhat oddly, the 2013 NRC Human Spaceflight Report referred to this once in a footnote …

“The committee defines as human exploration ambiguous cases in which humans are in martian or lunar orbit telerobotically conducting surface operations because of the astronauts’ proximity to the target and their remoteness from Earth.”

FISO Telecon 5/27/15 18 Opportunities and value presented by space telepresence

• Reduced cost impact, by not landing humans in strong gravity well.

• One party of humans on one trip achieves presence at many sites.

• Exploration duration not EVA-limited.

• Better visual perception (compared with the view through a helmet visor)

• Better dexterity (compared with dexterity achievable with EVA gloves)

• Benefits from rich commercial terrestrial technology development investments.

• Credible forward planetary protection. Donʼt put people where youʼre looking for life.

• Lower human risk. Put presence on the surface where humans might not dare tread.

>> Increase in the number of potential destinations for human “presence”. Mercury? Venus? Scuba diving in the lakes of Titan?

Thatʼs not to say that we donʼt want to land humans! (thatʻs another subject entirely)

FISO Telecon 5/27/15 19 ISS in LEO TDRSS in GEO

2-5 seconds latency 24/7 KONTOUR2 in Zvezda

~50 ms Testing from ISS?! latency Exercising comm for ~10 min challenges.! Weilheim overpass DLR On-orbit, low-latency control has been proven on ISS,

e.g. METERON (shown here), HET Surface Telerobotics K-10 White Sands “Justin” FISO Telecon 5/27/15 20 So where do we try to use space telepresence?

LEO-to-Earth demo from ISS e.g. Ames K-10 (Fong) ESA METERON (Schiele)

Good feasibility demo, but incomplete operational trial for Mars or Moon applications. TDRSS-reliant or very brief connect-time.

Lunar orbit-to-Moon e.g. from Orion or cis-lunar hab

But from where exactly?

LLO? Not good. R~100 km, unstable, short duration LOS contact. EM L1/L2? Needs some stationkeeping, r~70,000km 500ms latency is a bit large. Elliptical Changing latencies, limited LOS durations

ARM SDRO? Similarly large 2-way latency. Strongly captured but “precarious”.

PSDRO “Proximal” SDRO – ARM orbit > r > 5,000km giving real-time connectivity.

FISO Telecon 5/27/15 21 Lunar human-assisted sample return ?

“OSCAR” Orion Sample Capture and Return / Orion-MoonRise is an excellent concept proposed by Lockheed and JPL (Leon Alkalai, FISO telecon, 2-27-13) (and others)

Orion at EM L2 controls sampler-cacher in the SPAB, stow samples, transfer them to Orion waiting at EM L2, which transports them back to Earth.

Concept worked out in some detail

• Getting Orion back and forth to EM L2 • Lunar ascent vehicle to bring rocks back • Rendezvous, docking, and sample transfer to Orion • Target sites for sampling

Sample selection procedure not obviously worked out. Grab & go? Sampling decisions – this one, and not that one?

Human-in-the-loop sample selection, or human taxi service?

That being said, about EM L2 operations, ….

FISO Telecon 5/27/15 22 The value of PSDROs for lunar surface telepresence, … better than EM Lagrange point orbits?

EM L1/L2 orbit

25,000km

70,000km

25k km PSDRO

PSDRO characteristics (Adamo et al., AIAA Space2014) and Jeff Parker at Colorado Close to Moon ; ~180ms 2-way latency at 25,000km.

Ro~25,000 km makes for a convenient 4-day orbit. Good launch flexibility. Both-side operations. PSDROs are slightly “chaotic”, but strongly captured – no stationkeeping. High inclinations possible, permitting polar operations. Retrograde orbit offers free-return capability. FISO Telecon 5/27/15 23 Suppose we had a habitat at a 25,000 km PSDRO – an enabling site for lunar telepresence

• Latencies 15x less than from Earth, real time operations • Continuous access at a site over a whole workshift, both sides of Moon • No orbit maintenance required • Convenient trajectories to lunar surface • Similar LOI deltaVs for small PSDROs and, like EM L1 and L2 • Very short eclipses of Earth and Sun • Control options for multiple surface sites

This habitat need not obviate human visits to the lunar surface! (But such visits can be expensive …)

Most potent long term goal of lunar on-orbit telerobotics is conops validation for future visits to Mars and beyond.

FISO Telecon 5/27/15 24 How could telepresence offer value on the Moon?

Things youʼd want humans to do there. Especially things that might be inefficient to do from the Earth.

• Inspection and assaying of regolith • Manipulation and collection of samples • Site surveying and reconnaissance • Instrument emplacement, maintenance • Regolith grading, piling, and “paving” for dust suppression • Deployment of habitats and support facilities

(Yes, you could do this all from the Earth, but it would take a lot longer for complicated tasks …)

FISO Telecon 5/27/15 25 Operationally, comm is easy at a small PSDRO (and not bad at L1 or L2)

habitat L1/2

25,000km habitat

surface

FISO Telecon 5/27/15 26 Where we are now (beyond ISS experiments) (reports being produced)

• NASA Evolvable Mars Campaign planning on-orbit telepresence for Mars, perhaps on martian moons, as informed by activities in cis-lunar space. (Crusan, Drake et al.)

• “Humans Orbiting Mars” workshop highlighted on-orbit telepresence for Mars exploration. (Hubbard, Ney, Naderi, Price et al.)

• ESA “HERACLES”, end-to-end plan for cis-lunar space consistent with GER, include exploration telepresence explicitly. Partnering with NASA, CSA, ROSCOSMOS, JAXA. (Hufenbach, Landgraf et al.)

• “Telepresence” study group formed by IAA Planetary Robotic Exploration Coordinating Group. Possible workshop organization. (Coradini)

• Informal Exploration Telepresence Working Group active for a year. Biweekly telecons. Proposal for KISS workshop on exploration telepresence. (Anderson)

FISO Telecon 5/27/15 27 So where do we go from here?

On-orbit telerobotics is important rationale for in-space habitats

!Hab-specific value! • Control functionality? • Optimal orbit?

!Telerobotics-specific value! • Modes for fully immersive telepresence? • Extensibility to other destinations? • Sharing low- and high-latency tasks?

Policy-specific value • Reconcile historical “exploration” with on-orbit telepresence – exploration without physically being “all the way there”?

FISO Telecon 5/27/15 28 Dan Lester University of Texas [email protected] The evolution of exploration?

FISO Telecon 5/27/15 29