2005 IEEE International Workshop on Robots and Human Interactive Communication Determination of the Arm Orientation for Brain-Machine Interface Prosthetics Sam Clanton Joseph Laws Yoky Matsuoka Robotics Institute / School of Medicine Electrical and Computer Engineering The Robotics Institute CMU / Univ. of Pittsburgh Carnegie Mellon University Carnegie Mellon University Pittsburgh,PA, USA, 15213 Pittsburgh,PA, USA, 15213 Pittsburgh,PA, USA, 15213 [email protected] [email protected] [email protected] Controlling prosthetics with brain-machine injury. The substantial prevalence of these disorders, in interface will soon become the most natural way to combination with their severity, makes it imperative that restore limb function to those who suffer from satisfactory therapies for these conditions are developed. neurodegenerative disease or injury. Here, we first Current therapies for these patients include both various discuss the development of neural signal processing attempts at repair or reconnection of the dysfunctional systems for brain-machine interfaces which provide neuromuscular systems, as well as the use of artificial control within the Cartesian (extrinsic) frames of devices to replace the function of the defunct motion. Then we justify the development of systems extremities. that provide control using the kinematic (intrinsic) While some success has been made in regeneration frames of motion of the manipulator prosthetic of neuromuscular systems to restore the previous device. An experiment to create a general model of function of the body, our work focuses on the goal of natural arm motion is presented, along with its replacing the deficient motor system with a prosthetic application to brain-machine interfaces. system which attempts to reproduce the capabilities of the original. Current prosthetic systems commonly rely Index Terms – neuroprosthetics, kinematics, brain-machine on control signals generated from alternative signalling interface, biomechanics pathways from the motor system with the deficiency, e.g. the control of a prosthetic arm is based on muscular I. INTRODUCTION contractions at the shoulder. The development of more A significant cause of worldwide morbidity and sophisticated prosthetic systems which can reproduce the mortality is disease or injury which causes loss of the movement and control fidelity of the human extremities ability to actuate limbs. Conditions which fall into this must rely on more sophisticated control pathways that category include spinal cord injury, anterior lateral can incorporate the richness of signals inherent to the sclerosis (ALS, Lou Gehrig’s disease), and control systems of the human body. cerebellomedular disconnection (locked-in syndrome). These conditions cause varying degrees of dyskinesia, II. BRAIN-MACHINE INTERFACE WITH CARTESIAN ranging from a retardation of normal movement, to (EXTRINSIC) COORDINATE FRAMES quadriplegia, to the complete inability to exercise One potential modality of more sophisticated neural voluntary muscular control. In addition, conditions such control could employ a direct interface to the motor as limb injury and myopathy can cause dysfunction of cortex to provide control signals for prosthetic devices. the extremities themselves. Severe diseases of both of Some of the initial work in this area was pioneered by these types can render a fully conscious patient unable to Georgopoulos et al, who discovered that population of interact with his or her environment normally. In the neurons encode movement directions [20]. Since then, case of the more severe neuromuscular disorders, groups led by, for example, Schwartz, Donoghue, and patients often cannot live independently and rely on Nicolelis, have continued progress in this area by others for assistance with many basic daily activities. developing more sophisticated recording techniques and These diseases also lead to the death of patients, signal processing algorithms. Common signal especially for those less able to afford satisfactory health processing schemas for these control systems employ care and assistance. discrete classifiers, linear filters, and complex neural It is estimated that approximately 250,000 networks to develop reasonable estimates of the hand individuals in the United States alone have spinal cord motion [3], [7], [8], [9]. Two-dimensional control of injury [14]. This number is a fraction of those that have electronic interfaces with Cartesian coordinate frames lost limb function worldwide as a result of disease or has been commonly achieved with these methods. 0-7803-9275-2/05/$20.00/©2005 IEEE 422 Schwartz’s group employs population vectors in the high correlation of those intrinsic reference frames derived from Georgopoulos’s original work [20]. Using to the extrinsic coordinate frames for reaching tasks. this method, they demonstrated three-dimensional In addition to the difficulty with using joint Cartesian coordinate frame and has interfaced it with a angular population vectors, this technique is further robotic manipulator. Recent progress has allowed complicated by the fact that an artificial arm will (most primates to reach for orange slices placed in front of likely) not match the kinematics of the original arm of them with a brain controlled-robot arm [5]. the subject. Therefore, even if joint angles are While control methods based on three-dimensional accurately predicted from cortical signals, actuating the movement within Cartesian coordinate frames are novel arm with similar parameters would not establish the and crucial for brain-machine interface, they suffer from hand configuration nor link orientation in space as was inherent limitations when compared to the dexterous intended by the subject. Thus, a more sophisticated seven DOF arms in primates. These arms allow the system than simple joint angle or angular velocity primates to bring their hands to a particular location and prediction will need to be developed for the control of an orientation in space with the extra degree of freedom artificial 7-DOF prosthetic arm. that allows multiple arm configurations for the same hand location/orientation. The control of particular IV. APPROACHES TO BRAIN-CONTROLLED PROSTHETICS configuration of the arm is not only important for It has been shown that the activation levels of the keeping the arm out of the way when manipulating neurons that are correlated with arm movement change particular objects in space, but also is an important part when the transition is made from controlling a cursor of motion planning, where a different joint angle with real arm movements to no arm movements [12]. trajectory may be employed to reach the same target. This suggests that while a neuromotor model based on For instance, a particular arm position at an instant can correlations between cortical activation and actual arm be more suited for striking an object than reaching for it, movement are useful for training, the control model of a but simply knowing the Cartesian three coordinates and neuroprosthetic may not need to be based on the orientation of the hand cannot encode that information. predictive value of a model to the movement of an actual limb. In other words, the cortex may be able to control III. BRAIN-MACHINE INTERFACE WITH KINEMATIC joint-angle parameters of a 7-DOF device by employing (INTRINSIC) COORDINATE FRAMES a model without necessary direct correlation to the 7 Because of the limitations of the Cartesian-frame DOF control of an actual subject arm. system described, it is desirable to establish a robotic In addition, from the perspective of assisting the neuroprosthetic control system that operates through the disabled who have little or no ability to move their arms, 7-DOF kinematic (intrinsic) frames. This kinematic the closed-loop training phase with actual arm motion is coordinate frame approach is more likely to result in impossible to perform. For this reason, the development reconstructing the precise position and motion dynamics of a brain-computer interface system which does not rely of each robotic link as in the original appendage than the on motion data capture from the intended user of the Cartesian coordinate approach pursued. prosthetic will be important for the practical application Unfortunately, work in the brain-machine of these devices. interface with kinematic coordinate frames has met with Our work is focused on a system which relies on difficulty. So far, clear correlation between neural experimentally-derived statistical models of seven arm activity and joint angles have not been shown. Reina et joint angles to enable the control of a 7-DOF al. [13] demonstrated some ability to directly predict manipulator. Through the analysis of human subjects joint angular velocities of a primate arm with data with different arm kinematics reaching for objects at recorded from the motor cortex during reaching tasks known locations and orientations, we extrapolate using joint angular velocity population vectors. probability distributions of joint angle configurations for Predictions were significantly accurate for the joint reaching with manipulators of varied dimensions. These velocities of elbow flexion as well as shoulder data are then used to inform a model that maps neural flexion/extension and rotation, while predictions of other activation to 7-DOF reaching motions of a robotic arm. joint velocities of the arm were not. When Cartesian hand velocity was predicted using the same neuronal V. EXPERIMENTAL PROCEDURE data, the results were