Research Collection Doctoral Thesis The use of performance feedback and reward for optimization of motor learning and neurorehabilitation of motor functions Author(s): Widmer, Mario Publication Date: 2017 Permanent Link: https://doi.org/10.3929/ethz-a-010870008 Rights / License: In Copyright - Non-Commercial Use Permitted This page was generated automatically upon download from the ETH Zurich Research Collection. For more information please consult the Terms of use. ETH Library DISS. ETH NO. 24106 THE USE OF PERFORMANCE FEEDBACK AND REWARD FOR OPTIMIZATION OF MOTOR LEARNING AND NEUROREHABILITATION OF MOTOR FUNCTIONS A thesis submitted to attain the degree of DOCTOR OF SCIENCES of ETH ZURICH (Dr. sc. ETH Zurich) presented by MARIO WIDMER MSc ETH HMS, ETH Zürich born on 22.04.1986 citizen of Gränichen (AG) accepted on the recommendation of Prof. Dr. Nicole Wenderoth Prof. Dr. Andreas Luft Dr. Kai Lutz 2017 The Use of Performance Feedback and Reward for Optimization of Motor Learning and Neuro- rehabilitation of Motor Functions Doctoral Thesis MARIO WIDMER Acknowledgments Writing my dissertation, and finally completing it, could not have been done without the help of some enthusiastic and intelligent people around me. First, and foremost, I want to express my sincere gratitude to Prof. Dr. Andreas Luft for giving me the opportunity to work in this interesting research field and to conduct this thesis. His enormous scientific knowledge and experience were highly influential for my development over the course of my PhD. I would also like to sincerely thank Prof. Dr. Nicole Wenderoth for agreeing to be the head of my committee and for giving me the freedom to perform my research outside of ETH Zurich. Moreover, I am deeply grateful to Dr. Kai Lutz, my mentor, for his indispensable support dur- ing the last few years. I am indebted for his scientific advice, but also for his encouragement and comprehension in work-related as well as in private matters. I would like to express my thankfulness for the great support from our research team. A spe- cial thank you goes to my teammate Jeremia Held, who supported me in every situation, in research and in daily life. I thank him for his helpful advice and for being a friend. Many cor- dial thanks go to Belen Valladares, who is always having an open ear for me (never forget that you make Switzerland a better place), but also to Robinson Kundert, José López Sánchez, Irene Christen, Carola Bade-Daum, and all other members of my study team who have helped me over the years. Furthermore, I owe a big thank you to Samara Stulz for being an excellent Masters student, for her contribution to our "fMRI Reward Assessment" project and for her patience in the data acquisition, data analysis and the entry of the data in our electronic database. Many thanks also for taking care of our office plant, which is facing a very insecure future, now that you have left our group. In addition, I would like to take this opportunity to express my sincere appreciation to all participants for their time and enthusiasm during the studies of this thesis. That includes all healthy young and elderly subjects as well as all stroke patients who, at times, needed to bring along a lot of patience. v Acknowledgments The biggest thank you goes to my friends and my girlfriend for their unconditional love and support - Without you, I have nothing. But with you, I have everything! – and last but not least, I would like to offer my gratitude to family for being there for me. This research was carried out in collaboration with the University of Zurich, the University Hospital of Zurich, the cereneo - center for neurology and rehabilitation, and ETH Zürich. My position was funded by the Clinical Research Priority Program (CRPP) Neuro-Rehab of the University of Zurich. I am deeply grateful for their financial support. I would like to dedicate this thesis to Nadja Ziegler, who started her Master thesis in our lab at around the same time as I started my PhD. Nadja sadly passed away in July 2014. “Funny how someone can come into your life for such a brief time but leave such a lasting impres- sion” - Monica Murphy. Through all the pain of losing you I know that I am better for having known you! vi vii viii Abstract Intrinsic motivation refers to doing something because it is inherently interesting or enjoya- ble. Extrinsically motivated actions, on the other hand, are performed because they lead to an outcome. Similar to motivation, reward can be classified as extrinsic or intrinsic. Extrinsic reward refers to the receipt of material (e.g., food or money) for a specific activity. The term "intrinsic reward", on the other hand, refers to reward derived from task inherent stimulation (e.g., information about an achieved performance). This includes stimuli that signal perfor- mance accuracy, usually termed feedback, which can then be used to modify future perfor- mance. Generally, learners strive for positive feedback, which means that positive feedback fulfills the definition as a reinforcer or a reward. The changes in neural activity in response to the processing of reward (and punishment) has been extensively investigated in healthy, but also clinical populations, using the so-called monetary incentive delay (MID) task. Typically, this task requires an individual to react to a target stimulus presented after an incentive cue to win or to avoid losing the indicated re- ward. The first part of this thesis (Chapter 2 ) offers an overview of different utilizations of the MID task by reviewing literature outlining the neuronal processes involved in distinct aspects of human reward processing. A special focus was laid on reward-based learning processes. For instance, in a motor experiment using a MID task combined with functional magnetic resonance imaging (fMRI), both intrinsic and extrinsic rewards have been shown to increase the neural activity in the ventral striatum, a key locus of reward processing. In a rewarded task, hemodynamic ventral striatal response correlates with dopamine release in the ventral striatum, which similarly correlates with the reward-related neural activity in the substantia nigra/ventral tegmental area, the origin of the dopaminergic projection. There is evidence from animal studies that dopaminergic projections from the midbrain to the primary motor cortex (M1) are necessary for the learning of a new motor skill. In M1, dopamine facilitates long-term potentiation, a form of synaptic plasticity that is critically involved in skill learning. Such synaptic plasticity in M1 similarly occurs during recovery/rehabilitation after stroke and likely contributes to its success. Thus, this opens the potential to use rewarding feedback in humans to promote motor skill learning and neurorehabilitation of motor functions. ix Abstract Based on this evidence, we conducted an fMRI study with healthy young subjects, relating striatal activity to performance feedback with or without monetary consequences during the training of a repetitive arc-tracking task (Chapter 3 ). The task required subjects to perform wrist movements to steer a cursor on a computer screen through a semicircular channel while undergoing fMRI. Our results demonstrate an influence of the feedback modality on motor skill learning. Adding a monetary reward after good performance led to better consol- idation and higher ventral striatal activation than knowledge of performance alone. In con- clusion, rewarding strategies that increase ventral striatal response during the training of a motor skill may be utilized to improve skill consolidation. In stroke survivors, activity of this dopaminergic pathway may not only be reduced because rewards are small, but also because, after stroke, rewarding feedback might not have the same capacity to increase dopaminergic activity as in healthy subjects. This has been demon- strated for cognitive tasks, and the hypothesis for the study presented in Chapter 4 was, that this also happens in motor tasks. To test this hypothesis, we applied a similar arc-tracking task, modified as motor MID task and using fMRI to measure striatal activity linked to perfor- mance dependent monetary reward. Results of nine stroke patients and nine age-matched healthy individuals show a tendency for reduced responsiveness of ventral parts of the stria- tum in stroke patients. This is of particular interest as in the study described above ventral striatal activation was found to be the key factor for successful overnight consolidation. We have learned from animal studies that proper functioning of the dopaminergic reward system is necessary for successful motor skill learning. Thus, a reduced responsiveness of the ventral striatum to a motor performance derived reward, be it extrinsic or intrinsic, could be an im- plication for a blunted motor learning ability in patients after stroke. The ability to learn, however, is supposed to support motor recovery. After stroke, about 50% of all survivors remain with functional impairments of their upper limb. As we were able to show that training with rewarding feedback improves motor learn- ing in humans, we hypothesize that rehabilitative arm training could also be enhanced by rewarding feedback. This amplification of reward during rehabilitative training might be a means to overcome a potentially deficient response to task inherent feedback in order to stimulate the dopaminergic system to improve recovery after stroke. Therefore, a further achievement of this thesis is the development of a clinical trial protocol, investigating re- wards in the form of performance feedback and monetary gains as ways to improve effec- tiveness of rehabilitative training (Chapter 5 ). This trial will be the first to directly evaluate x Abstract the effect of rewarding feedback including monetary rewards on the recovery process of the upper limb following stroke. A positive outcome could therefore pave the way for novel types of interventions with significantly improved treatment benefits.