Mini‐Symposia Title: 04. Computational Systems & Synthetic Biology; Multiscale modeling Augmented Neural Prostheses 05. Cardiovascular and Respiratory Systems Engineering 06. Neural and Rehabilitation Engineering 07. Biomedical Sensors and Wearable Systems 08. Biorobotics and Biomechanics Mini‐Symposia Organizer Name & Affiliation: 09. Therapeutic & Diagnostic Systems and Technologies James Weiland, University of Michigan, Tim Denison, Oxford 10. Biomedical & Health Informatics Mini‐Symposia Speaker Name & Affiliation 1: 11. Biomedical Engineering Education and Society Cynthia Chestek, University of Michigan 12. Translational Engineering for Healthcare Innovation and Mini‐Symposia Speaker Name & Affiliation 2: Commercialization Dustin Tyler, Case University Mini‐Symposia Synopsis— Max 2000 Characters Neural prosthesis and neural modulation systems can benefit from Mini‐Symposia Speaker Name & Affiliation 3: adding capability that integrates information from wearable sensors and/ or enables closed loop operation. Coordinated and James Weiland, University of Michigan complementary distribution of function between implanted and external systems will become increasingly important to the success of such systems. Neural implants have been created to Mini‐Symposia Speaker Name & Affiliation 4: treat blindness, parlysis, movement disorders, dearness, and othe neurological disorders. But the level of functional recovery Jennifer Collinger, University of Pittsburgh possible with these devices is limited. Wearable sensing and computing technology, as well as sophisticated robotic limbs, offers increasing levels of complextity and capability, and a S Mini‐Symposia Speaker Name & Affiliation 5: possible means to augment neural prostheses and increase the level of functional recovery. Provision of feedback to control Robert Gregg, University of Michigan neural stimualation will result in fewer side-effects and more naturalistic control. In this mini-symposium, we will present neural prostheses for vision restoration, control of paralyzed or robotic Mini‐Symposia Speaker Name & Affiliation 6: arms, and mitigation of movement disorders, and discuss how Huiling Tan, University of Oxford these devices are and will be augmented through the deployment of wearable sensring/ computing/ actuation and the use of closed loop control as well as neuromodulation. Theme: 01. Biomedical Signal Processing 02. Biomedical Imaging and Image Processing 03. Micro/ Nano-bioengineering; Cellular/ Tissue Engineering & Augmenting neuroprosthetic control for individuated finger movements Cynthia Chestek, Associate Professor of Biomedical Engineering University of Michigan, Ann Arbor movements span the full range of highly coupled joints Abstract— Motor neuroprosthetics have the potential to one- rather than staying within a local linear approximation in a day restore fine movement of the hand and fingers to people limited working volume. As a result, the large improvements with upper limb amputations or paralysis. There have been promising proof of concept demonstrations, but performance over the past twenty years have come about by augmenting has so far been far below able-bodied hand movement. In our this basic underlying linear fit with additional information. group we have explored controlling individuated finger The Kalman filter, and its many variants add an underlying movements with muscle-amplified nerve signals from physical model to neural prediction [5]. Similarly, recurrent amputated nerves in humans, as well as with neural activity neural networks have begun to show promise in brain from primary motor cortex in nonhuman primates. Most machine interfaces and have their own underlying dynamics motor decoders in the literature would naturally improve with that could theoretically infer missing information from a the quality and quantity of neural signals recorded, but these weak neural signal [7],[8]. types of hardware improvements translate very slowly to While we routinely augment our algorithms with human use. Fortunately, machine learning has provided a range of methods for augmenting a simple underlying linear information beyond what we get directly from the neurons, model linking neural activity to movement. These include augmentation in terms of going beyond what could be Kalman filter trajectory models, enforced smoothing, mode accomplished with able-bodied hands is far more selection schemes, and recurrent neural networks. In the challenging. All of the high performance BMI systems future, neural networks have the potential to further embed currently require a brain surgery. Also, in terms of end intelligence about what movements should look like directly effector, it is still extremely difficult for even a state of the into our decoding algorithms. art robotic manipulator to fold a shirt. However, to quote Bill Gates, “we always overestimate the change that will Within the past decade, there have been numerous occur in the next two years and underestimate the change demonstration of cortical brain-machine interfaces (BMIs) that will occur in the next ten.” As intracortical high channel controlling upper limb movement in people with spinal cord count BMIs enter their third decade with promising paths to injuries. [1], [3], [4], [6]. While this is promising further improvement, things may progress faster than preliminary work, performance is not yet sufficient for a expected. disabled patient to be without a caregiver for long periods of time. Increased independence would likely be a prerequisite 1. Ajiboye, A. Bolu, et al. "Restoration of reaching and grasping for wide-scale clinical BMI deployment. There is strong movements through brain-controlled muscle stimulation in a interest in the spinal cord injury (SCI) community to have person with tetraplegia: a proof-of-concept demonstration." The Lancet 389.10081 (2017): 1821-1830. cortical neuroprostheses, particularly to regain control of 2. Blabe, Christine H., et al. "Assessment of brain–machine their native arm [2]. However, BMI performance must be interfaces from the perspective of people with high enough to produce meaningful life improvements, for paralysis." Journal of neural engineering 12.4 (2015): 043002. example through increased autonomy or restoration of the 3. Bouton, Chad E., et al. "Restoring cortical control of functional movement in a human with quadriplegia." Nature 533.7602 ability to work. (2016): 247. Intracortical BMI performance has increased dramatically 4. Collinger, Jennifer L., et al. "High-performance neuroprosthetic since the early 2000s. Interestingly, the underlying map control by an individual with tetraplegia." The Lancet 381.9866 between neural firing rates has remained basically the same (2013): 557-564. 5. Gilja, Vikash, et al. "Clinical translation of a high-performance since early work. Most human and monkey brain machine neural prosthesis." Nature medicine 21.10 (2015): 1142. interface experiments today still use linear regression often 6. Hochberg, Leigh R., et al. "Reach and grasp by people with augmented by regularization (e.g. [4]. This is simple to use tetraplegia using a neurally controlled robotic and generalizes well to movements not specifically in the arm." Nature 485.7398 (2012): 372. 7. Pandarinath, Chethan, et al. "Inferring single-trial neural training dataset. Nonetheless, the linear prediction itself is population dynamics using sequential auto-encoders." Nature actually very poor, requiring heavy “fixing” to produce a methods (2018): 1. smooth movement. Also, the problem becomes harder as we 8. Sussillo, David, et al. "A recurrent neural network for closed-loop add degrees of freedom or move to the fingers, where intracortical brain–machine interface decoders." Journal of neural engineering 9.2 (2012): 026027. Putting the Human in the Loop for Prosthetics following Limb Loss Dustin J. Tyler, Case Western Reserve University and Cleveland VA Medical Center Abstract— The loss of a hand or foot results in significant I. INTRODUCTION loss of function and significant loss of connection to objects The goal of a limb loss prosthesis is for user to experience and others. When asked, the first response from each of our the prosthesis as though it is their limb (Figure 1). This research subjects it that they most miss the ability to feel the requires agency and visual-tactile synchrony. Agency refers hands of a loved one. They also talk about their prosthesis as to the user’s “effortless” control of the limb where a user a tool at the end of their residual limb. When using the thinks about moving the device and it “just moves” as prosthesis, they are heavily dependent on visual feedback intended. Visual-tactile synchrony refers to the user feeling for task function. All of this results in a very unnatural and tactile information that is simultaneous to the visual ineffective replacement for the lost limb. To change this experience of a tactile interaction and matches the expected paradigm, the prosthesis’ interaction with the world must location, intensity, and quality based on the visual input. To feel like the user’s hand interacting with the world and the feel like part of the user, the delay loop from subject control of the prosthesis should engage the conscious and initiation of task to perceptual feedback must be less than pre-conscious human motor-control systems. This requires approximately 50 msec. If a system connects to natural that we take external information from the prosthesis, neural systems, it should become more incorporated