NRI Collaborative Research: Human-Centered Modeling and Control of Cooperative Manipulation with Bimanual Robots Ruzena Bajcsy, Human-Assistive Robotic Technologies (HART) Lab, University of California, Berkeley Oussama Khatib, Stanford Lab, Computer Science Department,

Abstract Musculoskeletal Morphology Analysis

In physical human-robot collaboration, the robot Objective Auto-Segmentation Morphological Variation can use knowledge of its human partner’s physiological limitations and ergonomic pref- Evaluate erences to plan safer, more comfortable ac- tions that more easily accommodate the unpre- • morphological variation dictability of human motion. across subjects • ability of existing musculoskel- Our work addresses three first steps in endowing etal modeling frameworks to co-robots with this knowledge: account for this variation • construction of personalized musculoskeletal • extent to which variation models that accurately capture morphological impacts predicted dy- variation between individual humans namics (contact forces, joint • non-invasive human motion sensing to pro- torques, etc.) vide co-robots with an accurate estimate of a hu- SUBJECT 1 SUBJECT 2 SUBJECT 3 man collaborator’s state Preliminary segmentation results show • control strategies that generate safe and ergo- Using maximally stable extremal regions (MSER) for segmentation, significant that nomically optimal robot actions given an individual bones can be segmented in an automated manner (left) morphological variation across subjects cannot be modeled in existing frameworks. human’s physical state and physiological limits that significantly reduces manual cleanup time(right) .

Non-Invasive Human Motion Sensing and Dynamical Model Fitting Objective Simplified Model Preliminary Results

Assuming muscle force-length relation EMG for single Create a model of the human arm Data: ~400 × that F/T subject • accurately predicts contact forces { } and joint torques and normalized muscle activation and length Subject pressed upward on F/T sensor mounted to UR5 ro- • is trainable/customizable using non-in- bot while sEMG data were gathered from Myo arm bands. vasive sensing The muscle force-length relation was recovered via: • accommodates dynamically- and med- ically-relevant pathologies the dynamics relation of each pair is described by • avoids overfitting The generated surface is qualitatively reasona- • has no reliance on literature val- ble and fits the data well, and the predictedforce- ues or population measures length relation is biologically reasonable:

i.e., . , rad , Nm

Human-Robot Collaborative Control Strategies Future Work Objective Model Results Our future endeavors will build on the results Optimize physical human-robot We seek to influence the human to choose The robot shapes the set of available human actions so that the most above and will include collaboration with respect to ergo- a globally optimal action, using human grasp probable, greedy choices are also near-globally optimal. • statistical analysis of morphological vari- nomic cost and human safety and fea- configuration selection probability sibility constraints ation across subjects and its impact on pre- dicted dynamics • automated extraction of morphological ergonomic cost (from musculoskeletal model) features (including muscle volume and mus- cle-bone attachment points) • incorporation of additional non-invasive (we used ) Scenario: robot-human object sensor data (including AMG and ultrasound) handoffs to select robot grasp and handover pose with • further refinement and increase in com- minimum expected total cost Problem: Humans are plexity of dynamical model of the arm • non-deterministic: won’t reliably se- • extension of collaborative control algo- lect an optimal configuration even rithms to plan continuous human-robot cou- if it’s available pled trajectories (in addition to the static in- (optimize over • greedy: will select immediately low- all robot grasps, (expected handover cost) (expected goal cost) teractions considered in this work) cost option even if it’s suboptimal handover poses) (all feasible human overall grasps)