Research Article: New Research | Sensory and Motor Systems Feedback adaptation to unpredictable force fields in 250ms https://doi.org/10.1523/ENEURO.0400-19.2020
Cite as: eNeuro 2020; 10.1523/ENEURO.0400-19.2020 Received: 3 October 2019 Revised: 12 March 2020 Accepted: 6 April 2020
This Early Release article has been peer-reviewed and accepted, but has not been through the composition and copyediting processes. The final version may differ slightly in style or formatting and will contain links to any extended data.
Alerts: Sign up at www.eneuro.org/alerts to receive customized email alerts when the fully formatted version of this article is published.
Copyright © 2020 Crevecoeur et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.
1 2 3 Feedback adaptation to unpredictable force fields in 250ms 4 5 Abbreviated Title: Feedback Adaptation in 250ms 6 7 8 9 Frédéric Crevecoeur1,2, James Mathew1,2, Marie Bastin3, Philippe Lefevre1,2 10 11 1Institute of Information and Communication Technologies, Electronics and Applied Mathematics 12 (ICTEAM), 2Institute of Neuroscience (IoNS), University of Louvain, Belgium 13 3DNAlytics, Louvain-la-Neuve, Belgium 14 15 16 17 18 19 20 21 22 23 Corresponding Author: 24 F. Crevecoeur, 25 Euler Building, 4 ave. Georges Lemaitre 26 1348 Louvain-la-Neuve, 27 Belgium 28 29 30 E-mail: [email protected] 31 Tel.: +3210478040 32 33 Acknowledgements: 34 The authors wish to thank the anonymous reviewers and the reviewing editor for their 35 continued and in-depth assessment of our work. FC and JM are supported by grants from F.R.S.- 36 FNRS (Belgium, grant numbers: 1.C033.18, PDR T.0048.19). 37 38
1
39 40 Abstract
41 Motor learning and adaptation are important functions of the nervous system. Classical
42 studies have characterized how humans adapt to changes in the environment during tasks such as
43 reaching, and have documented improvements in behavior across movements. Yet little is known
44 about how quickly the nervous system adapts to such disturbances. In particular, recent work has
45 suggested that adaptation could be sufficiently fast to alter the control strategies of an ongoing
46 movement. To further address the possibility that learning occurred within a single movement, we
47 designed a series of human reaching experiments to extract from muscles recordings the latency of
48 feedback adaptation. Our results confirmed that participants adapted their feedback responses to
49 unanticipated force fields applied randomly. In addition, our analyses revealed that the feedback
50 response was specifically and finely tuned to the ongoing perturbation not only across trials with the
51 same force field, but also across different kinds of force fields. Finally, changes in muscle activity
52 consistent with feedback adaptation occurred in about 250ms following reach onset. The adaptation
53 that we observed across trials presented in a random context was similar to the one observed when
54 the force fields could be anticipated, suggesting that these two adaptive processes may be closely
55 linked to each other. In such case, our measurement of 250ms may correspond to the latency of
56 motor adaptation in the nervous system.
57
58 Significance Statement
59 We measure the latency of feedback adaptation in a human reaching experiment by applying
60 force field trials randomly. In spite of the fact that these disturbances could not be anticipated, we
61 measured improvement in feedback corrections that paralleled standard adaptation. Correlates in
62 muscle recordings occurred within about 250ms following movement onset. Such a short timescale
63 of adaptation suggested that rapid adaptation complements feedback control of an ongoing
64 movement. To further test this hypothesis, we demonstrate that indeed participants are able to
2
65 adapt their feedback responses to different kinds of force fields and directions applied randomly.
66 These findings support the existence of very rapid, possibly online, adaptation in the nervous system.
3
67 Introduction
68 Humans and other animals can adapt motor patterns to counter predictable disturbances
69 across a broad range of contexts, including reaching, locomotion, and eye movements (Shadmehr et
70 al., 2010; Wolpert et al., 2011; Roemmich and Bastian, 2018). A central question in movement
71 neuroscience is to identify the time scales at which this process can influence behavior. In the
72 context of reaching movements, standard learning paradigms have focused on trial-by-trial learning,
73 such that changes in behavior were documented by contrasting early and late motor performances,
74 often separated by minutes to hours, or equivalently by hundreds of trials (Lackner and DiZio, 1994;
75 Shadmehr and Mussa-Ivaldi, 1994; Singh and Scott, 2003; Smith et al., 2006; Wagner and Smith,
76 2008). Thus, a clear benefit of motor adaptation is to improve behavior over these timescales, which
77 is of prime importance for instance when we deal with a new tool or environment. The associated
78 neural mechanism must also be beneficial for adaptation to changes occurring over slower time
79 scales such as development and long-term skill acquisition (Dayan and Cohen, 2011).
80 Besides the improvement of behavior over medium to long timescales, previous studies also
81 indicated that motor learning could be very fast. The presence of rapid adaptation was previously
82 established by observing after-effects induced by a single movement (Sing et al., 2013). Likewise,
83 unlearning was documented after a single catch trial when a force field was unexpectedly turned off
84 (Thoroughman and Shadmehr, 2000). Other studies showed that the timescale of motor learning
85 could be even faster (Braun et al., 2009; Crevecoeur et al., 2020). In the latter reference, it was
86 documented that healthy volunteers could produce adapted feedback responses to the
87 unanticipated force field perturbations during reaching, and after-effects were evoked within an
88 ongoing sequence of movements in less than 500ms when participants were instructed to stop at a
89 via-point.
90 These latter results contrasted with standard models of sensorimotor learning
91 (Thoroughman and Shadmehr, 2000; Baddeley et al., 2003; Smith et al., 2006; Kording et al., 2007),
92 which included multiple timescales but assumed that each movement was controlled with a fixed
4
93 representation as a consequence of the assumption that the fastest timescale in these models was
94 longer than the movement time. The expression of after-effects in less than 500ms challenged this
95 view, as it reveals that adaptation was potentially fast enough to influence movements of similar or
96 longer duration. Thus, motor adaptation could not only support learning across trials but also
97 complement online feedback control.
98 Evidence for online adaptation was interpreted in the context of adaptive control (Bitmead
99 et al., 1990): a least-square learning algorithm coupled with a state-feedback controller. This
100 technique is based on standard state-feedback control models that successfully capture humans’
101 continuous and task-dependent adjustments of voluntary movements (Todorov and Jordan, 2002;
102 Diedrichsen, 2007; Liu and Todorov, 2007), as well as feedback responses to mechanical
103 perturbations (Scott, 2016; Crevecoeur and Kurtzer, 2018). Intuitively, the state feedback controller
104 in the nervous system can be viewed as a parameterized control loop, and the goal of adaptive
105 control is to tune this loop in real-time by continuously tracking the model parameters (and errors).
106 This model captured both adjustments of control during un-anticipated perturbations, and the
107 standard single rate trial-by-trial learning observed across a few trials, assuming that changes in
108 anticipation across trials may come from rapid feedback adaptation of the previous one (Crevecoeur
109 et al., 2020).
110 To gain further insight into the timescales of motor adaptation in the brain, we designed this
111 study to address the following key questions: first, we sought to replicate previous findings of
112 adaptation to unpredictable disturbances, and measure precisely the latency of adaptive changes in
113 control from muscle recordings. Second, we sought to test a surprising prediction of the theoretical
114 framework of adaptive control: if the nervous system tracks model parameters in real-time, then, in
115 principle, it should be possible to handle simultaneously force fields not only of different directions
116 (clockwise or counterclockwise) but also of different kinds (i.e. with different force components).
117 First, our results showed that feedback responses to unanticipated perturbations became tuned to
118 the force field within ~250ms of movement onset. Second, we found that humans were indeed able
5
119 to produce adapted and specific feedback responses to different force fields randomly applied as
120 catch trials. Our results confirmed the existence of a very fast adaptation of feedback control during
121 movements and provide an estimate of ~250ms for the latency of motor adaptation.
6
122 Methods
123 Experiments
124 A total of 44 healthy volunteers were involved in this study (19 females, between 22 and 37
125 yrs) and provided written informed consent following procedures approved by the local Ethics
126 Committee (UCLouvain, Belgium). Eighteen participants performed the first experiment, another
127 group of 18 participants performed the second experiment, and the rest (n=8) performed the control
128 experiment. The data of the control experiment was published in our previous study and was re-used
129 here to underline the similarities between feedback adaptation and standard trial-by-trial learning.
130 In all experiments, participants grasped the handle of a robotic arm (KINARM, BKIN
131 Technologies, Kingston, ON, Canada), and were instructed to perform visually guided reaching
132 movements towards a virtual target. Each trial ran through as follows. Participants had to wait in the
133 home target (a filled circle with radius 0.6cm) for a random period uniformly distributed between 2s
134 and 4s. The goal was also displayed as a circle located 15cm ahead of the start. After the random
135 period, the cue was delivered to initiate the movement by filling the goal target (Fig. 1a). Participants
136 had between 600ms and 800ms (including reaction time) to reach the goal and stabilize in it for at
137 least 1s. Information about the time window was provided as follows: when participants reached the
138 goal too soon, it turned back to an open circle. When they reached it too late, it remained red. When
139 they reached it within the desired time window, it became green and a score displayed on the screen
140 was incremented. The scores and feedback about timing were provided to encourage consistent
141 movement times, but all trials were included in the dataset. The grand average success rate was
142 70±12% for Experiment 1, and 76±10% for Experiment 2. In all cases, the direct vision of the arm and
143 hand was blocked but the cursor aligned to the handle was always visible. These procedures were
144 identical across the three experiments, which only varied by the frequency and nature of mechanical
145 perturbations applied during movements.
146
7
147 Experiment 1. This experiment was designed to replicate previous results on the adaptation of
148 feedback responses to unpredictable perturbations (Crevecoeur et al., 2020), and to measure the
149 moment within a trial when the muscle activity started to show feedback tuning corresponding to
150 the force field. Participants performed six blocks of 60 trials, composed of unperturbed trials
151 (baseline) and force field trials. The and coordinates corresponded to lateral and forward
152 directions, respectively (Fig. 1a). In this experiment, the force field was defined as a lateral force
-1 153 proportional to forward velocity: =± ̇, with =±13 Nsm (the subscript “ ” refers to the
154 Orthogonal force field). There were 5 force field trials per block and per direction (counter-clockwise
155 and clockwise), which corresponded to a frequency of perturbation trials of 1/6, and a total of 30
156 force field trials for each perturbation direction. The sequence of trials was randomized within each
157 block, such that the occurrence and direction of the perturbations were unpredictable.
158
159 Experiment 2. The purpose of this experiment was to test further the hypothesis of online adaptive
160 control by alternating different kinds of force fields, which in theory could be handled by online
161 tracking of model errors (Bitmead et al., 1990). To investigate this, we performed an experiment
162 similar to Experiment 1, with the addition of curl force field trials randomly interspersed between
163 unperturbed and orthogonal force field trials. The orthogonal force field was identical to Experiment
164 1. For the curl field, both forward and lateral velocities were mapped onto lateral and forward
-1 165 perturbation forces with opposite signs, respectively: = ̇, and = − ̇ with =±15 Nsm
166 (the subscript “ ” refers to the Curl field). There were 5 perturbation trials per force field
167 (orthogonal and curl) and direction (clockwise and counterclockwise), summing to a total of 20
168 perturbations per block presented in a random sequence. As in Experiment 1, participants performed
169 six blocks of 60 trials, composed of 40 baseline trials and 20 perturbation trials (perturbation
170 frequency: 1/3).
171
8
172 Control experiment. In this experiment, we were interested to measure participants’ behavior in a
173 fully predictable context corresponding to a standard adaptation task. Participants performed a
174 series of baseline trials for training, followed by 180 force field trials (orthogonal force field, CW or
175 CCW for the entire series), followed by another series of 180 force field trials in the opposite
176 direction for the entire series. The two series were separated by 20 baseline trials to induce washout
177 between the two adaptation phases. We re-used previously published data for this experiment and
178 refer to (Crevecoeur et al., 2020) for complementary descriptions of the results.
179
180 Data Collection and Analysis
181 The 2-dimensional coordinate of the cursor aligned to the robotic handle, and the forces at
182 the interface between the participants’ hand and the handle were sampled at 1kHz, and digitally low-
183 pass filtered with a fourth-order dual-pass Butterworth filter with a cut-off frequency of 50Hz.
184 Velocity signals were obtained from numerical differentiation of position signals (4th order, finite
185 difference algorithm). We collected the activities of two of the main muscles recruited when
186 performing lateral corrections against the perturbations used in our experiment: Pectoralis Major
187 (shoulder flexor) and Posterior Deltoid (shoulder extensor). Muscle samples were recorded with
188 surface electrodes for Experiments 1 and 2 (Bagnoli Desktop System, Delsys, Boston, MA, US). EMG
189 signals were collected at 1kHz, digitally band-pass filtered (4th order dual-pass: [10, 400] Hz), and
190 rectified.
191 Two events were used as timing references. First, reach onset was defined as the moment
192 when the cursor aligned to the handle exited the home target. Second, we used a position threshold
193 located at 1/3 of the distance between the home and goal targets to re-align the EMG traces offline.
194 The crossing of this position threshold approximately coincided with the peak forward velocity,
195 which allowed reducing the trial-to-trial variability in EMG recordings. Similar conclusions were
196 obtained when all analyses were performed based on traces realigned with respect to reach onset.
9
197 Exponential fits were used to quantify the presence of learning on several parameters,
198 including the maximum lateral hand displacement, and maximum target overshoot for Experiment 1.
199 The quantification of learning from Experiment 2 was based on exponential fits of the path length
200 computed as the time integral of hand speed. We fitted the exponential functions to the raw data
201 from each participant as a function of the trial index and assessed whether the 99.9% confidence
202 interval for the parameter responsible for the curvature of the fit included or not the value of 0
203 (P<0.001). Variability across participants was illustrated on hand trajectories by calculating the
204 dispersion ellipses based on singular value decomposition of the covariance matrices at different
205 time steps evenly spaced.
206 We measured both the onset of changes in EMG responsible for changes in behavior across
207 early and late force field trials, as well as the onset of changes in EMG across force fields from
208 Experiment 2. To contrast early and late trials, EMG data were averaged for each participant across
209 the first four and last four trials. To contrast the feedback responses to orthogonal and curl fields in
210 Experiment 2, EMG data were averaged across the last 15 trials of each kind of force field. EMG
211 averages were then collapsed into a 30ms wide (centered) sliding window, and sliding comparisons
212 across time were performed with paired t-tests. We searched in the time series of P-values the
213 moment of strongest statistical difference across populations of EMG data (P<0.005), and then went
214 back in time until the threshold of P<0.05 was crossed. On the one hand, this test could identify early
215 differences since it included data from -15ms to +15ms relative to the center of the bin, but on the
216 other hand, we kept the threshold of significance instead of attempting to find the true onset of
217 changes in responses that must have occurred a little before. This criterion, along with the fact that
218 the crossing of the threshold of 0.05 was followed by highly significant differences, ensured reliable
219 conclusions. It should be noted that corrections for multiple comparisons do not apply here for two
220 reasons: first, the samples at each time step are involved in only one comparison, and second
221 consecutive samples are not statistically independent. Indeed, if there is a significant difference at a
10
222 given time step, it is very likely that there is also a significant difference at the next time step because
223 signals do not vary instantaneously. Hence, the risk of false-positive must not be controlled.
224 An index of motor adaptation was derived based on the relationship between the lateral
225 commanded force and the measured force along the same axis. Similar metrics were used previously
226 (Crevecoeur et al., 2020), and were based on the fact that these correlations were sensitive to
227 learning. A classical approach is based on error clamps or force channels producing virtual walls and
228 allowing an ideal measure of participants’ expectations. In our case, we are interested in online
229 control and force channels are not suitable. The correlations between commanded and measured
230 forces are also impacted by other factors including the robot dynamics (which putatively also
231 influences the force produced against a channel wall), and real-time sampling errors. Besides these
232 errors, the difference between measured and commanded force simply corresponds to lateral
233 acceleration. We documented previously that errors made by ignoring the robot dynamics and by
234 online sampling were on the order of ≤10% (Crevecoeur et al., 2020). Hence, the change in
235 correlation in the range that we observed with similar forward kinematics could be linked to
236 adaptation. It is clear that this metric is close to others that could be captured based on lateral
237 acceleration, velocity or displacement. However, we priviledged the correlation as this technique
238 allows extracting a single parameter from continuous curves, by taking their entire time-varying
239 profiles into account.
240 The data of the control experiment was also used to validate this argument empirically.
241 Specifically, for each trial, we computed a least-square linear regression between the commanded
242 force obtained from velocity signals, and the applied force measured with the force encoders. These
243 correlations were then averaged across perturbation directions for each participant (as they revealed
244 qualitatively similar effects), then across participants for illustration. Surrogate correlations were
245 obtained by calculating linear regressions between the measured force and the commanded force of
246 randomly selected trials with replacement. These surrogate correlations were calculated on 100
247 randomly picked trials with a replacement for each index and participant.
11
248 Results
249 Experiment 1
250 Our first experiment was designed to replicate previous findings of the adaptation of
251 feedback responses to unpredictable disturbances (Crevecoeur et al., 2020), and measure accurately
252 from EMG data the moment when the perturbation-related activity started to be tuned to the force
253 field. Importantly, a feedback response is expected in all cases (Milner and Franklin, 2005; Wagner
254 and Smith, 2008; Cluff and Scott, 2013). What we searched for was not just a feedback response, but
255 a change in feedback response across early and late force field trials indicating that the response
256 became adapted to the force field. The critical aspect of our experiments with respect to these
257 previous studies, which also documented changes in feedback response, is that in our case, the force
258 field perturbation could never be anticipated.
259 We measured a clear deviation in the lateral cursor displacement in the direction of the force
260 field as expected since the perturbations could not be anticipated (Fig. 1b). Although the maximum
261 hand displacement exhibited a small reduction across the first few trials, the exponential fits of this
262 variable as a function of the trial index did not display any significant curvature (P > 0.05 for both
263 directions). In other words the parameter in the exponential fit responsible for the curvature of the
264 function was not distinct from 0, reducing to a linear (possibly horizontal) best fit. In contrast, the
265 maximum target overshoot exhibited a clear and highly significant exponential decay across trials
266 (Fig. 1d, P<0.001). Hence participants initiated force field trials with a controller that would
267 otherwise produce a straight reach path (black traces in Fig. 1b), resulting in a clear perturbation-
268 related movement error, but then managed to improve their online correction across perturbation
269 trials.
12
270
271 Figure 1. a. Illustration of the workspace and task. Participants were instructed to perform forward 272 reaching movements towards a visual target. An open goal target was presented for a random delay 273 uniformly distributed between 2s and 4s before it was filled in. The cue to reach the target was given 274 by filling in the goal in red. The goal was turned red if the time between the go signal and the 275 stabilization in the target was comprised between 0.6s and 0.8s. b. Hand paths from the first force 276 field trials (top) and trial #25 selected for illustration (bottom) from each participant (n=18). Counter- 277 clockwise and clockwise perturbations are depicted in blue and red, respectively. The black traces 278 illustrate for each panel baseline trials selected randomly (1 baseline trial per participant). Osh.: 279 target overshoot; Max.: maximum displacement in the direction of the force field; Thresh.: The 280 positional threshold used to align EMG data. c. Maximum displacement in the direction of the force 281 field. The dashed trace illustrates that the exponential fit did not reveal any significant curvature 282 across force field trials (P>0.05). d. Maximum target overshoot in the direction opposite to the force 283 field. Solid traces revealed strongly significant exponential decay across trials (P<0.001). 284
285 It was shown previously that the reduction in target overshoot was associated with a
286 reduction in measured force near the end of the movement (Crevecoeur et al., 2020). This result was
287 also replicated below. As a consequence, we expected to measure a reduction in muscle response to
288 the perturbation. To observe this, we averaged EMG data across the first four and last four trials for
289 each perturbation direction. Our rationale was that fewer than four trials would likely be too small a
290 sample, whereas trials performed after 5 or more FF trials already exhibited significant adaptation,
291 thereby reducing the size of the effect under investigation. As expected, we found a significant
292 reduction in EMG responses to the perturbations (Fig. 2a-b). When traces were aligned to the
293 position threshold and averaged across directions (see Methods), we found that the onset of a drop
294 in the time series of P-values occurred on average at 122ms following threshold (center of the 30ms
295 bin, Fig. 2c-d). It could be observed that there was no clear difference across the first and last trials
296 prior to the moment when the P-value dropped (Fig. 2c), which indicated that there was no
297 systematic co-activation of this pair of antagonist muscles on average, and therefore no clear
298 reliance on limb impedance control.
13
a b c Counterclockwise Clockwise * ** P < 0.05 Trials [1:4] 2 2 Trials [27:30] 1 PM PD
0 0 EMG [a.u.] 0
EMG [a.u.] PM PD -2 -2 -1 Agonist Thresh. Average -0.20 0.2 0.4 0.6 -0.20 0.2 0.4 0.6 -0.20 0.2 0.4 0.6 Time [s] Time [s] Time [s] d e f 0 *** 1 Hand Paths
-1 P = 0.05 -2 -3 Log10(P) Thresh. -4
-5 Distributions Cumulative 0 -0.20 0.2 0.4 0.6 0.2 0.3 2cm 299 Time [s] Time re: Reach Onset [s]
300 Figure 2. a. Activity of Posterior Deltoid (PD, agonist, solid), and Pectoralis Major (PM, antagonist, 301 dashed) averaged across the first four (light blue) and last four counterclockwise (dark blue) 302 perturbation trials. The vertical arrows illustrate the moment when a sliding paired comparison of the 303 agonist activity averaged in a 30m window dropped below P<0.05. Traces were aligned to the 304 position threshold corresponding to one-third of the reach path to reduce variability. b. Same as panel 305 a for clockwise perturbation trials. The position threshold is also represented. c. Grand average of the 306 difference between the agonist activities of the first four and last four force field trials, aligned to the 307 position threshold and averaged across muscles and participants (n=18). The gray area corresponds 308 to the standard error of the mean. The dashed window is the first window that displays a significant 309 difference from sliding paired comparison (P<0.05, width=30ms). The solid window is the window 310 associated with the minimum P-value (P<10-4). d. P-value of the sliding paired comparison performed 311 on the data from panel c. All EMG traces were smoothed with a 5-ms sliding window for illustration 312 purposes. e. Cumulativeindividual distributions of the delay between movement onset, and the 313 moment when the P-value of panel d dropped below 0.05. This moment corresponds to the time of 314 threshold crossing plus 122ms. The median delay between movement onset and this time was 315 237±15ms (mean±SD across participants, n=18). f. Average hand paths and standard dispersion 316 ellipses for the first four and last four trials in each direction, Dispersion ellipses are displayed 50ms 317 (see Methods). The black dots represent the moment corresponding to the vertical arrows of panels a 318 and b. 319
320 Because there was some variability between reach onset (defined as the moment when the
321 cursor exited the home target) and the moment when participants’ hand crossed the position
322 threshold, we calculated for each subject distribution of elapsed time between reach onset and the
323 moment corresponding to threshold + 122ms. These distributions are reported in Fig. 2e and the
324 mean±SD of medians is shown (black cross): 237±15ms. For illustration, we reported in Fig. 2f the
325 mean latency of within-trial changes in feedback response on the average hand path represented for
14
326 the first and last four trials. The black circle illustrates the moment of a significant reduction in
327 perturbation-related EMG that could be linked to the reduction in target overshoot observed in
328 Figure 1. The analysis of antagonist muscle activity also revealed a decrease in activity in absolute
329 value, but the difference was delayed relative to the onset of change in agonist activity (Fig. 2,
330 dashed traces in panels a and b). Following the same technique of sliding window, we found that the
331 difference between early and late trials became significant at 400±15ms (medians across individual
332 distributions, mean±SD).
333 The reductions in perturbation related response in EMG and in target overshoot were
334 expected if participants learned to handle the force field. To further address whether their online
335 corrections reflected adaptation, we correlated the measured lateral force with the commanded
336 force calculated offline based on forward hand velocities. Average traces were represented in Fig. 3a
337 and b for CCW perturbations (normalized for illustration). Observe that the average correction in the
338 first trial was variable and the traces were irregular (Fig. 3c). In contrast, the same data plotted for
339 the last trials appeared more regular (Fig. 3b, d). Figure 3c and 3d show phase diagrams with
340 measured and commanded forces in the first and last trials for each participant. These traces were
341 taken from ~200ms following reach onset to 1000ms (gray rectangles in panels a and b), based on
342 the previous analysis revealing that there was no difference until ~240ms following reach onset, and
343 thus no expected improvement in correlation prior to this time. It can also be observed that the peak
344 terminal force occurring after 400ms decreased in absolute value, which is consistent with our
345 previous observation (Crevecoeur et al., 2020), and can be linked to the fact that participants were
346 able to reduce the lateral target overshoot when exposed to the force field.
15
347
348 Figure 3. a. Forward hand velocity (black) and measured lateral force (blue) normalized to the 349 average maximum calculated on the first trial. Shaded areas represent one standard error across 350 participants (n=18). Panels display data from the trials with counter-clockwise force field 351 perturbation. b. Same as panel a for the last trial normalized to the average maximum of the first 352 trial. c. Lateral force as a function of the forward velocity for CW and CCW perturbations (red and 353 blue, respectively). d. Same as panel c for the last trial. For panels c and d, there was one trace per 354 participant. Observe that the traces were smoother in the last trials. e. Mean±SEM of the linear 355 correlation (R2 statistics) across force field trials from Experiment 1 (CCW and CW perturbations 356 averaged). f. Difference between the correlations from Experiment 1 as calculated in panel c, and the 357 correlation between the lateral force of each trial with the forward hand velocity of a randomly 358 picked surrogate trial with replacement. The surrogate correlations were calculated 100 times per 359 trial and participants and averaged across. The shaded area is one SEM. 360
361 The correlations exhibited highly significant changes across trials. This was first assessed with
362 a repeated-measures ANOVA on the correlations with the trial indices as the main factor (rmANOVA,
16
-5 363 F(29,493)= 8.7, P<10 ), and a standard least-square linear regression highlighted a clear increase in this
364 variable (Fig. 3e, P<0.005). These correlations were compared to those obtained with randomly
365 picked surrogate profiles to see whether the measured force in each perturbation trial reflected
366 tuning to the ongoing perturbations (see also Methods, Fig. 3f), or whether a non-specific correction
367 pattern was produced, which could correlate as well with randomly picked surrogates as with the
368 experienced perturbation. We found that the true correlations were initially below the surrogate and
369 then became greater than the surrogates. In support of this observation, we found a significant
370 interaction between the the trial index and correlation type (i.e. true versus surrogate, rmANOVA,
371 F(29,493)=1.88, P=0.004).
372 This result must be interpreted as follows: when perturbations are associated randomly to
373 the measured forces, it produces surrogate correlations that are initially better than in the data due
374 to more regular profiles taken from late learning, and then it produces correlations that are weaker
375 than the data as early trials with more perturbed profiles are sometimes randomly picked in the
376 surrogates. As a consequence, the change in correlations is not simply due to the fact that the force
377 profiles become more regular, which explains the increase in the surrogate data, but the true
378 correlations are better than those obtained by random associations between commanded and
379 measured forces. In other words, the relationship between the specific perturbation and force
380 profiles from the same trials must be preserved to reproduce the observed correlations. To make
381 sure that this result was not too dependent on some very particular traces that occurred early during
382 learning, we performed the same analyses after removing the first four trials for the surrogate
383 correlations and found the same result (interaction between trial index and true or surrogate types
384 without the first four trials, rmANOVA, F(29,493)=1.9, P<0.002).
385 We verified with the data from the control experiment that an increase in the same
386 correlations between the measured and commanded forces occurred when the perturbations were
387 fully predictable (Fig. 4). The commanded and measured forces displayed initially variable traces with
388 a terminal increase in interaction force at the handle consistent with the production of a target
17
389 overshoot (black arrow), followed by more regular and similar profiles. Hence, the key observations
390 were that these correlations represented a sensitive metric of learning, and they increased across
391 trials in the random context of Experiment 1 similarly as in the standard context of trial-by-trial
392 adaptation. In all, the data from Experiment 1 highlighted that participants were able to adapt their
393 feedback responses to unanticipated force field disturbances within ~240ms following reach onset
394 and their force control during unpredictable force field trials paralleled the behavior observed in a
395 standard learning paradigm in spite of the lack of anticipation.
ab c CCW CW CW Perturbations 4 Trial #1 Trial #180 1 SEM] ± 0 0.8 Trial #1 Trial 0.6
Force [N] -4 Commanded 0.4 -8 Measured
5cm Correlations [mean Trial #180 00.51 00.51 1 60 120 180 396 Time [s] Trial Index
397 Figure 4. Control experiment, data from (Crevecoeur et al., 2020). a. Individual hand traces from the 398 first and last perturbation trials in each direction from the control experiment (one traces per 399 participant, n=8). b. Commanded (black) and measured (red) force profiles for the first (left) and last 400 (right) clockwise perturbations. The arrow highlights the increase in peak terminal force linked with 401 the target overshoot. Observe that the traces become very similar, which increases the temporal 402 correlation between them. c. Correlations between commanded force and measured force as in 403 Figure 3 against trial indices. Correlations were averaged across directions and participants. Displays 404 are mean±SEM across participants. 405
406 Experiment 2
407 This experiment was designed to investigate whether participants could learn to adapt their
408 feedback responses when exposed to four different force fields at the same time: either orthogonal
409 or curl force field, in clockwise or counter-clockwise directions. For the orthogonal field, we observed
410 the same behavior as in Experiment 1: minute changes in the maximum lateral displacement across
411 the first few trials, and highly significant exponential decay of the maximum target overshoot across
412 trials (not shown). Hand traces during curl field trials were distinct because, unlike the orthogonal
413 field, the anteroposterior component of the force field prevented a systematic target overshoot (Fig.
414 5a-b). For this reason, we used the path length to capture adaptation across the two force fields with
18
415 the same metric. We found a clear reduction in path length for each force field and each
416 perturbation direction (Fig. 5c). Exponential fits confirmed a very strong decay across trials (Fig. 5c,
417 P<0.001).
418 As for Experiment 1, we calculated the temporal correlations between the commanded and
419 measured lateral forces averaged across clockwise and counterclockwise directions. For the two
420 kinds of force fields, we found a clear impact of the trial index on the correlations, which confirmed
-6 421 the visible increase shown in Fig 5d (rmANOVA, F(29,493)>4, P<10 ). Furthermore, for the two kinds of
422 force fields, we found highly significant interactions between the trial index, and the difference
-10 423 between the true and surrogate correlations (F(29,493)>5, P<10 ). As for Experiment 1 we verified that
424 this result was not too dependent on the early phase of learning where force and velocity profiles
425 were more irregular and found also a strong interaction between the trial index and the correlation
-10 426 type after removing the first four trials (F(29,493)>5, P<10 ). Observe also that in this experiment, the
427 correlations in the last trials wer significantly stronger than for the surrogate (direct comparisons,
428 one sided paired t-test: orthogonal field: t(17)=3.7, P=0.0016; curl field: t(17) = 6.4, P<0.001). This
429 analysis indicated again that true correlations were lower first, then became greater, which
430 supported that online corrections were tuned to the specific force profile experienced during each
431 trial.
ab Orthogonal Field Curl Field
5cm Trial #1 Trial #25 Trial #1 Trial #25 Trial #1 Trial #25 Trial #1 Trial #25
cd Orthogonal Field Curl Field Orthogonal Field Curl Field 28 0.9 CCW CW P < 0.001 0.7 24
0.5 20 Correlations Path Length [cm] Data Surrogates 0.3 1 10 20 30 1 10 20 30 1 10 20 30 1 10 20 30 432 Trial Number Trial Number
19
433 Figure 5. a. Hand paths from the first (light colors) and 25th (dark colors) trials with an orthogonal 434 field chosen to illustrate changes in feedback responses. All data were taken from Experiment 2, and 435 each trace represents trials taken from each participant (n=18). Blue and red traces represent counter 436 clock-wise and clockwise perturbations. b. Same as panel a for the curl field. c. Path length across 437 force field trials. d. Trial by trial correlations between the lateral commanded force (proportional to 438 forward velocity) and the measured force. Correlations were averaged across counter-clockwise and 439 clockwise directions. The shaded areas represent one SEM across participants. The dashed traces 440 (mean±SEM) are the correlations between the measured lateral force and the commanded force 441 corresponding to the velocity of randomly picked trials with replacement. The procedure was 442 repeated 100 times for each participant, and the results were averaged across CCW and CW 443 perturbations directions. 444
445 Surface recordings during the orthogonal force fields gave similar results as those reported in
446 Fig. 2: we found that the initial responses to the perturbations were similar for Pectoralis Major and
447 Posterior Deltoid, until 100ms following the threshold, where we observed a reduction in activity
448 consistent with the production of an adapted response (Fig. 6a-c). As a consequence, the
449 adjustments, in this case, were even observed slightly earlier than during Experiment 1. Indeed, the
450 median time elapsed between reach onset, which was defined as the time when they exited the
451 home target, and the reduction in target overshoot was 218±10ms (Fig. 6d). The analysis performed
452 on curl field trials also revealed a significant reduction in perturbation-related activity occurring
453 104ms following the threshold, which corresponded to a delay between reach onset and changes in
454 muscle response of 225±11ms (Fig. 6e-h).
455 Again, there was no systematic change in co-contraction, which could have impacted the
456 mechanical impedance of the shoulder joint (Hogan, 1984; Burdet et al., 2001). Indeed, the average
457 traces in Fig 6 c and 6g represent the average difference in muscle activity aligned to the position
458 threshold for each muscle. The presence of co-contraction would have resulted in an offset observed
459 at the beginning of reach onset. To quantify this, we averaged the activity in each muscle from the
460 first 100ms of the window presented in Fig. 6 (threshold-200ms until threshold-100ms), and
461 compared the activity across the first and last four trials. We did not observe any statistical
462 difference (t(17)<1.1, P>0.3). Besides possible changes in limb impedance (but see (Crevecoeur and
463 Scott, 2014)), there is a known modulation of baseline activity evoked by unanticipated force field
464 trials identified previously on the same muscles and during a similar task (Franklin et al., 2008;
20
465 Crevecoeur et al., 2019). However, this effect did not impact the activity prior to the perturbations
466 systematically, likely due to the fact that perturbation trials were randomly interspersed and the
467 modulation of baseline activity averaged out.
468 We made similar observations as in Experiment 1 regarding the antagonist response: the
469 activity in the antagonist was statistically similar until later in the reach relative to the change in
470 agonist response, for both muscles and for the two force field conditions. We applied again the same
471 sliding analysis to the grand average as suggested in Fig. 2c but for the antagonist activity and found
472 that the changes relative to reach onsets were at 345±10ms for the orthogonal force field, and
473 351±11ms for the curl force field (medians of individual distributions, mean±SD). These changes in
474 antagonists were also characterized by a reduction in perturbation-related activity.
475
476 Figure 6. a-d. Same as Figure 2 for the orthogonal force field data from Experiment 2 (a distinct group 477 of 18 participants). Traces are first and last 4 trials in the force field for Pectoralis Major (a blue) and 478 Posterior Deltoid (b, red) aligned to the position threshold. The activity during unperturbed trials was 479 subtracted and the traces were smoothed with a 5ms moving average for illustration. The vertical 480 arrows show the moment with a sliding paired comparison of activity averaged in 30ms bins became 481 significant (P<0.05). c. Difference between early and late feedback responses (mean±SEM) averaged 482 across participants and muscles. The results of the sliding paired comparisons are shown: one star, 483 first 30ms-bin with P<0.05, two stars: minimum of P (P<0.005, see Methods). d. Individual 484 distributions of time elapsed between reach onset and the center of the first bin with P<0.05. The 485 cross is the median ± SD across participants. e-h. Same as a-d for the curl force field. 486
21
487 It remained to be elucidated whether the feedback responses to the orthogonal and curl
488 force fields were distinct, or whether participants used a single response pattern undifferentiated
489 across force field disturbance, should the perturbations be sufficiently close to be handled with a
490 single and non-specific response. Our data allowed us to reject this possibility. Indeed, we contrasted
491 the feedback response to curl and orthogonal force fields averaged across the last 15 force field trials
492 for each muscle. We found clear changes in EMG patterns, such that the curl force field evoked a
493 stronger response, and was followed by a second increase in activity near the end of the reach (Fig.
494 7a-b). Based on the difference between the activities from curl and orthogonal fields, we found that
495 the modulation of EMG activity was highly significant (Fig. 7c). Furthermore, the onset of changes in
496 EMG revealed that the feedback responses were tuned to the force field very early during
497 movement: this time corresponded to ~55ms prior to threshold on average (Fig. 7d), which was
498 denoted as a star in the average hand path displayed in Fig. 7d (black circle). This median of the
499 distributions of elapsed time from reach onset to this time across participants was 65±10ms
500 (mean±SD).
a b cd Posterior Deltoid (CCW) Pectoralis Major (CW) Grand Average Curl Orth. 3 2 0.8 Curl * ** Orth. 2 0.4 1 1 0 EMG [a.u.]
EMG: Curl -Orth. [a.u.] **
0 0 -0.4 -0.20 0.2 0.4 0.6 -0.20 0.2 0.4 0.6 -0.20 0.2 0.4 0.6 501 Time [s] Time [s] Time [s] 5 cm
502 Figure 7. a. Average activity of Posterior Deltoid in curl (thick) and orthogonal (thin) counterclockwise 503 perturbations across the last 15 trials of each type of force field. b. Same as a for Pectoralis Major 504 from clockwise perturbations. c. Difference between activities recorded during curl and orthogonal 505 force fields averaged across participants and muscles. The onset of significant changes based on a 506 30ms wide sliding window is highlighted with one star (P<0.05), followed by strongly significant 507 differences (two stars, P<0.005). d. Average hand paths during counter-clockwise perturbations. 508 Ellipses are 2-dimensional standard dispersion across participants every 50ms, and the impact of the 509 force field is illustrated with the gray arrow (a common scaling for the two force fields was applied for 510 illustration). The open dots are the moment when the activities started to differ across two types of 511 perturbations. 512
22
513 It is important to stress that this result was expected and must be used as a control analysis:
514 in fact the perturbations across orthogonal and curl fields were distinct, thus it is not surprising to
515 observe distinct responses. The goal of this analysis is to verify that the motor system responded to
516 the details of the perturbations across each trial, and as a consequence, there were no default
517 feedback responses triggered in all cases. The link with adaptive control was based upon the
518 observation that participants’ behavior had a similar signature as during a trial-by-trial learning
519 scenario (Fig. 4), in conjunction with the fact that there was no anticipation (by design), no
520 systematic co-contraction, and that the feedback responses were adjusted to each perturbation trial
521 (Fig. 7 and surrogate data analyses on Fig. 3 and 5).
522 A consistent observation across Experiments 1 and 2 was a decrease in perturbation-related
523 feedback response, which accounted for the improvement in reach control. We interpret this
524 reduction as an adaptation in the feedback circuits, because of similar changes in correlation as
525 observed in the control experiment with standard trial-by-trial learning. But could there be a
526 “default” downregulation of feedback gains? Our data suggest that it was not the case. We limited
527 this analysis to the orthogonal field for simplicity. Across experiments and perturbation directions,
528 we found no significant variation of forward velocity across trials (rmANOVA, all F(29,493)<1.36, all P >
529 0.1). Thus the commanded lateral force was statistically similar. A default downregulation of
530 feedback gains should then be visible in the peak of measured force applied to the handle. We found
531 such a decrease in absolute value in only one case (CCW in Exp. 1, F(29,493)<1.74, P=0.01), and a visual
532 inspection indicated that this happened for the first few trials. In the other cases, we found either no
533 change across trials (F(29,493)<1.2, P>0.2), or even an increase in the peak of lateral measured force
534 (CW in Exp. 2, F(29,493)=2.11, P<0.001). Thus there was no default or generic downregulation of
535 feedback gains, instead, it occurred typically later than the peak force during the reach. A precisely
536 timed downregulation with respect to the force field is expected to assume adaptation of feedback
537 motor responses.
23
538 To summarize, participants produced feedback responses tailored to the details of the
539 unanticipated perturbations for each kind of force field; these feedback responses improved and
540 exhibited similar traits as those of standard adaptation paradigms, namely the increase in correlation
541 between commanded and measured forces. These changes in behavior were linked to finely tuned
542 EMG activities, which indicated that feedback response was adapted to the force field as early as
543 250ms following reach onset.
24
544 Discussion
545 Current theories of motor learning have postulated that sensory feedback about movement
546 error is mapped to model updates for the next movement. Based on this idea, several seminal
547 studies have characterized learning across movements by means of trial-by-trial learning curves. We
548 recently argued that motor adaptation unfolded over faster time scales, potentially within a single
549 trial, which revealed a novel function of motor adaptation, which is to complement feedback control
550 online (Crevecoeur et al., 2020). To further test this hypothesis, we performed here two experiments
551 with the following aims: 1- to replicate our previous results on improvements in feedback responses
552 to unanticipated force fields, 2- to identify in muscle recordings the latency of adaptive changes in
553 control, and 3- to test whether healthy humans could adapt their feedback response to two different
554 force fields and two different perturbation directions randomly applied across trials. The results
555 confirmed our previous findings and highlighted a modulation of feedback responses within 250ms
556 of reach onset. Importantly, these feedback responses were also specific to each perturbation profile
557 (curl and orthogonal force fields).
558 Previous studies investigated learning in random environments and found partial to total
559 anterograde interference between different motor memories dependent on the time between two
560 adaptation sessions (Gandolfo et al., 1996; Shadmehr and Brashers-Krug, 1997; Karniel and Mussa-
561 Ivaldi, 2002; Caithness et al., 2004). To explore the neural basis of acquisition of multiple motor skills,
562 subsequent studies have highlighted that adapting to opposite velocity-dependent force fields was
563 possible if they were associated with different contextual cues, movement representations, initial
564 limb proprioceptive states, or planning conditions (Wada et al., 2003; Addou et al., 2011; Hirashima
565 and Nozaki, 2012; Howard et al., 2012; Green and Labelle; Sheahan et al., 2016). Our data contrast
566 with this literature at first glance: we claim there was feedback adaptation to opposite velocity-
567 dependent force fields applied randomly without any change in context or representation across
568 trials. We explain this discrepancy by the fact that most studies used indicators such as maximum
569 perpendicular error, based on which we would not conclude for adaptation either (see e.g. Fig. 1c). In
25
570 contrast, an indication of changes in feedback control occurred later in movement and was observed
571 when we analyzed the details of force, EMG and kinematic signals near the end of movements. Thus
572 we conclude that there is an adaptation of the feedback controller, which is evoked by the
573 perturbation experienced within the movement, and as a consequence takes some time during the
574 trial before it has a measurable impact on control. Our data suggested that motor execution could
575 form and recall distinct internal representations of dynamics during movement. This interpretation
576 based on adaptive control captures both interference and feedback adaptation: if the nervous
577 system tracks model parameters online, then the alternation of CW and CCW force field trials
578 produces an estimate of the force field that is zero on average. Thus, in this framework, there is no
579 anticipation of the force field type or direction, but the improvements in feedback corrections are
580 mediated by online adaptation. In addition, these individual perturbation responses were not
581 stereotyped. The differences between single-trial adaptive responses when orthogonal and curl field
582 perturbations were randomly intermingled indicated specificity of perturbation response at the
583 single trial level. Interestingly, a field specific adaptation was reported in a random perturbation
584 scenarios were position and velocity-dependent force fields were applied (Joiner et al., 2017). Thus
585 online responses and single-trial adaptation seem both highly sensitive to the details of the
586 perturbation profiles.
587 Of course, this adaptation may differ from anticipatory or feed-forward adaptation. In
588 standard learning paradigms, it was postulated that reaching movements consisted of a feed-forward
589 component, which we can define as an open-loop control sequence, along with a feedback
590 component (Kawato, 1999), and that adaptation was associated with changes in the internal forward
591 model (Shadmehr et al., 2010; Franklin and Wolpert, 2011). Several reports have then documented
592 that feed-forward and feedback control systems share their internal representations, based on the
593 observation that changes in feedback corrections correlated with the learning of a force field
594 (Wagner and Smith, 2008; Ahmadi-Pajouh et al., 2012; Cluff and Scott, 2013; Maeda et al., 2018). The
595 adaptation that we highlighted here may be a distinct process because there was no possibility for
26
596 the participants to change their anticipatory compensation for the force field trials, which were
597 unpredictable by design.
598 However, the observed feedback adaptation may not be functionally distinguishable from
599 the standard adaptation: it enabled adjustments of motor output suitable to the experienced force
600 field. Although feedback adaptation may be dissociable from feedforward adaptation (Yousif and
601 Diedrichsen, 2012; Kasuga et al., 2015), and may not engage the same circuits, there is no reason a
602 priori to dismiss it as a contributor to motor adaptation in general. A possible link between feed-
603 forward and feedback adaptation is that the online representation which is used to adapt the
604 feedback response in a force field trial may be stored and used as a new prior for the next
605 movement. This hypothesis is motivated by the observation that a single force field trial is sufficient
606 to produce significant after-effects in the next trial (Scheidt et al., 2001; Sing et al., 2013; Crevecoeur
607 et al., 2019) (also observed in the present dataset). Considering this, a challenging question arises as
608 to how much feedback adaptation contributes to trial-by-trial learning, and how these two putative
609 processes are implemented in neural circuits.
610 Our reasoning was based on the theory of adaptive control. The basic premise of this theory
611 is that the controller adjusts the parameters of the state-feedback control loop in real-time. In
612 principle, there is no lower bound on the time scale of this mechanism, but the instantaneous
613 learning rate should not be too large to prevent instability (Bitmead et al., 1990). The reduction in
614 target overshoot for the orthogonal field (data from Exp. 1) without impacting the maximum lateral
615 hand deviation was previously explained in this context (Crevecoeur et al., 2020). The ability to adapt
616 feedback responses to two different randomly applied force fields was also a prediction of this
617 model: if parameters can be tracked online, it does not matter which force field is applied (curl or
618 orthogonal), since sensory feedback of each specific trial can be used to produce a response adapted
619 to the ongoing perturbation.
620 Another candidate mechanism to counter unexpected disturbances is the use of co-
621 contraction to modulate the limb’s intrinsic properties (Hogan, 1984; Burdet et al., 2001; Franklin et
27
622 al., 2008), and the feedback gains in a non-specific way (Crevecoeur et al., 2019; Bian et al., 2020).
623 Such a strategy was previously demonstrated to influence feedback control gains across a few trials,
624 which also limits the extent of lateral displacement following disturbances. However, our current
625 dataset requires another explanation because the perturbations were applied randomly, and there
626 was no co-contraction observed on average. Furthermore, an increase in EMG activity expected if a
627 strategy based on co-contraction was used, whereas we documented a decrease in perturbation-
628 related response across the two experiments (as shown in Fig. 2a, b, 6a, b, e, f). The absence of a
629 systematic increase in baseline co-activation questioned the possibility that a default increase in
630 intrinsic limb impedance or control gains was responsible for the adjustments of control.
631 Contrary to an increase in limb mechanical impedance, our data showed that the change in
632 feedback correction was instead associated with a reduction in muscle activity, raising the question
633 whether this change was based on knowledge of the force field, or whether it was applied in a non-
634 specific manner. One should recall that the forward velocity was relatively constant by design, and
635 we did not measure any systematic change in interaction force applied to the handle until the peak
636 in this variable. Thus the reduction in control gain was not a default mechanism either, as it was
637 observed only after some time within the trial. In our view, it is difficult to imagine a precisely timed
638 modulation in control gains unrelated to adaptation (albeit not feedforward adaptation), since the
639 function of this mechanism, that is to improve control, and one of its behavioral signature (changes
640 in correlations) were the same across random and predictable contexts.
641 Online tracking of model parameters is a candidate model to explain the change in control
642 occurring within a movement. Indeed, movements were straight on average and force field trials
643 were often separated by several baseline trials. Thus, participants initiated the reaching movements
644 with a controller corresponding to a baseline trial (i.e. without force field), which would, in this case,
645 produce a straight reach path (see black traces in Fig. 1). Then, during perturbations, they changed
646 their control to produce feedback responses that became adapted to the force field. This transition
647 between a baseline controller and a controller adjusted to each force field, along with the
28
648 observation that each feedback response was better adapted without practicing in a predictable
649 context, constituted strong evidence for adaptive control in the motor system.
650 How much the controller changed within perturbation trials, or between two trials remains a
651 matter of debate. On the one hand, our previous study provided an upper bound of ~500ms within
652 which after-effects could be evoked (Crevecoeur et al., 2020). Our current measurements based on
653 EMG indicated that the change in feedback responses, likely based on the same mechanism,
654 occurred within 250ms. This time window leaves enough room for adjustments of the controller to
655 each force field within the movement. However, it is clear that changes in movement representation
656 also occurred offline, between two trials or over longer timescales (Krakauer and Shadmehr, 2006;
657 Smith et al.; Kording et al., 2007; Dayan and Cohen, 2011).
658 Further investigations are required to better characterize the components of adaptive
659 control. For instance, our experiments did not allow teasing apart how much vision and
660 somatosensory feedback contributed to feedback adaptation, as the cursor was visible all the time.
661 However, a strong contribution of proprioception is expected: first, this system could produce
662 detailed responses to the smallest perturbations with long-latency delays (50-60ms) (Crevecoeur et
663 al., 2012), which almost certainly contributed to the early changes in EMG data of Experiment 2
664 evoked by very small differences across orthogonal and curl force fields. Furthermore, previous work
665 highlighted that long-latency feedback engaged motor responses that are well captured in a state-
666 feedback control model (Crevecoeur and Kurtzer, 2018). This rapid state-feedback control loop is
667 supported by a distributed network through primary sensory and motor cortices, pre-motor cortex,
668 parietal regions, and cerebellum (Flament et al., 1984; Omrani et al., 2016). Hence, the fastest
669 adjustments to state feedback control could be achieved by tuning the long-latency feedback loop.
670 Besides the potential contribution of muscle afferent feedback, the fact that changes in
671 feedback responses were detected within 250ms leaves enough time to engage task-related
672 feedback responses mediated by touch (Pruszynski et al., 2016; Crevecoeur et al., 2017), and vision,
673 which participates in goal-directed feedback control (Franklin and Wolpert, 2008; Scott, 2016), as
29
674 well as in rapid changes in navigation strategies (Cross et al., 2019). Characterizing the specific
675 contribution of each sensory system constitutes an exciting challenge for future work.
30
676 References
677 Addou T, Krouchev N, Kalaska JF (2011) Colored context cues can facilitate the ability to learn and to 678 switch between multiple dynamical force fields. J Neurophysiol 106:163-183. 679 Ahmadi-Pajouh MA, Towhidkhah F, Shadmehr R (2012) Preparing to reach: selecting an adaptive 680 long-latency feedback controller. J Neurosci 32:9537-9545. 681 Baddeley RJ, Ingram HA, Miall RC (2003) System identification applied to a visuomotor task: near- 682 optimal human performance in a noisy changing task. J Neurosci 23:3066-3075. 683 Bian T, Wolpert DM, Jiang ZP (2020) Model-Free Robust Optimal Feedback Mechanisms of Biological 684 Motor Control. Neural Comput 32:562-595. 685 Bitmead RR, Gevers M, Wertz V (1990) Adaptive optimal control : the thinking man's GPC. New York: 686 Prentice Hall. 687 Braun DA, Aertsen A, Wolpert DM, Mehring C (2009) Learning optimal adaptation strategies in 688 unpredictable motor tasks. J Neurosci 29:6472-6478. 689 Burdet E, Osu R, Franklin DW, Milner TE, Kawato M (2001) The central nervous system stabilizes 690 unstable dynamics by learning optimal impedance. Nature 414:446-449. 691 Caithness G, Osu R, Bays P, Chase H, Klassen J, Kawato M, Wolpert DM, Flanagan JR (2004) Failure to 692 consolidate the consolidation theory of learning for sensorimotor adaptation tasks. J 693 Neurosci 24:8662-8671. 694 Cluff Y, Scott SH (2013) Rapid feedback responses correlate with reach adaptation and properties of 695 novel upper limb loads. Journal of Neuroscience 33:15903-15914. 696 Crevecoeur F, Scott SH (2014) Beyond Muscles Stiffness: Importance of State Estimation to Account 697 for Very Fast Motor Corrections. PLoS Computational Biology 10:e1003869. 698 Crevecoeur F, Kurtzer IL (2018) Long-latency reflexes for inter-effector coordination reflect a 699 continuous state-feedback controller. J Neurophysiol. 700 Crevecoeur F, Kurtzer I, Scott SH (2012) Fast corrective responses are evoked by perturbations 701 approaching the natural variability of posture and movement tasks. J Neurophysiol 107:2821- 702 2832. 703 Crevecoeur F, Scott SH, Cluff T (2019) Robust control in human reaching movements: a model-free 704 strategy to compesate for unpredictable disturbances. Journal of Neuroscience. 705 Crevecoeur F, Thonnard JL, Lefèvre P (2020) A Very Fast Time Scale of Human Motor Adaptation: 706 Within Movement Adjustments of Internal Representations during Reaching. eNeuro 7. 707 Crevecoeur F, Barrea A, Libouton X, Thonnard JL, Lefevre P (2017) Multisensory components of rapid 708 motor responses to fingertip loading. J Neurophysiol 118:331-343. 709 Cross KP, Cluff T, Takei T, Scott SH (2019) Visual Feedback Processing of the Limb Involves Two 710 Distinct Phases. J Neurosci 39:6751-6765. 711 Dayan E, Cohen LG (2011) Neuroplasticity subserving motor skill learning. Neuron 72:443-454. 712 Diedrichsen J (2007) Optimal task-dependent changes of bimanual feedback control and adaptation. 713 Curr Biol 17:1675-1679. 714 Flament D, Vilis T, Hore J (1984) Dependence of cerebellar tremor on proprioceptive but not visual 715 feedback. Experimental Neurology 84:314-325. 716 Franklin DW, Wolpert DM (2008) Specificity of Reflex Adaptation for Task-Relevant Variability. 717 Journal of Neuroscience 28:14165-14175. 718 Franklin DW, Wolpert DM (2011) Computational mechanisms of sensorimotor control. Neuron 719 72:425-442. 720 Franklin DW, Burdet E, Tee KP, Osu R, Chew CM, Milner TE, Kawato M (2008) CNS Learns Stable, 721 Accurate, and Efficient Movements Using a Simple Algorithm. Journal of Neuroscience 722 28:11165-11173. 723 Gandolfo F, Mussa-Ivaldi FA, Bizzi E (1996) Motor learning by field approximation. Proc Natl Acad Sci 724 U S A 93:3843-3846.
31
725 Green AM, Labelle JP (2015) The influence of proprioceptive state on learning control of reach 726 dynamics. Exp Brain Res 233:2961-2975. 727 Hirashima M, Nozaki D (2012) Distinct motor plans form and retrieve distinct motor memories for 728 physically identical movements. Curr Biol 22:432-436. 729 Hogan N (1984) ADAPTIVE-CONTROL OF MECHANICAL IMPEDANCE BY COACTIVATION OF 730 ANTAGONIST MUSCLES. Ieee Transactions on Automatic Control 29:681-690. 731 Howard IS, Ingram JN, Franklin DW, Wolpert DM (2012) Gone in 0.6 seconds: the encoding of motor 732 memories depends on recent sensorimotor states. J Neurosci 32:12756-12768. 733 Joiner WM, Sing GC, Smith MA (2017) Temporal specificity of the initial adaptive response in motor 734 adaptation. PLoS Comput Biol 13:e1005438. 735 Karniel A, Mussa-Ivaldi FA (2002) Does the motor control system use multiple models and context 736 switching to cope with a variable environment? Exp Brain Res 143:520-524. 737 Kasuga S, Telgen S, Ushiba J, Nozaki D, Diedrichsen J (2015) Learning feedback and feedforward 738 control in a mirror-reversed visual environment. J Neurophysiol 114:2187-2193. 739 Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 740 9:718-727. 741 Kording KP, Tenenbaum JB, Shadmehr R (2007) The dynamics of memory as a consequence of 742 optimal adaptation to a changing body. Nat Neurosci 10:779-786. 743 Krakauer JW, Shadmehr R (2006) Consolidation of motor memory. Trends Neurosci 29:58-64. 744 Lackner JR, DiZio P (1994) Rapid adaptation to Coriolis-force perturbations of arm trajectory. J 745 Neurophysiol 72:299-313. 746 Liu D, Todorov E (2007) Evidence for the flexible sensorimotor strategies predicted by optimal 747 feedback control. Journal of Neuroscience 27:9354-9368. 748 Maeda RS, Cluff T, Gribble PL, Pruszynski JA (2018) Feedforward and Feedback Control Share an 749 Internal Model of the Arm's Dynamics. J Neurosci 38:10505-10514. 750 Milner TE, Franklin DW (2005) Impedance control and internal model use during the initial stage of 751 adaptation to novel dynamics in humans. J Physiol 567:651-664. 752 Omrani M, Murnaghan CD, Pruszynski JA, Scott SH (2016) Distributed task-specific processing of 753 somatosensory feedback for voluntary motor control. Elife 5. 754 Pruszynski JA, Johansson RS, Flanagan JR (2016) A Rapid Tactile-Motor Reflex Automatically Guides 755 Reaching toward Handheld Objects. Curr Biol 26:788-792. 756 Roemmich RT, Bastian AJ (2018) Closing the Loop: From Motor Neuroscience to Neurorehabilitation. 757 Annu Rev Neurosci 41:415-429. 758 Scheidt RA, Dingwell JB, Mussa-Ivaldi FA (2001) Learning to move amid uncertainty. J Neurophysiol 759 86:971-985. 760 Scott SH (2016) A Functional Taxonomy of Bottom-Up Sensory Feedback Processing for Motor 761 Actions. Trends Neurosci 39:512-526. 762 Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor 763 task. Journal of Neuroscience 14:3208-3224. 764 Shadmehr R, Brashers-Krug T (1997) Functional stages in the formation of human long-term motor 765 memory. J Neurosci 17:409-419. 766 Shadmehr R, Smith MA, Krakauer JW (2010) Error Correction, Sensory Prediction, and Adaptation in 767 Motor Control. Annual Review of Neuroscience, Vol 33 33:89-108 768 769 Sheahan HR, Franklin DW, Wolpert DM (2016) Motor Planning, Not Execution, Separates Motor 770 Memories. Neuron 92:773-779. 771 Sing GC, Orozco SP, Smith MA (2013) Limb motion dictates how motor learning arises from arbitrary 772 environmental dynamics. J Neurophysiol 109:2466-2482. 773 Singh K, Scott SH (2003) A motor learning strategy reflects neural circuitry for limb control. Nature 774 Neuroscience 6:399-403. 775 Smith MA, Ghazizadeh A, Shadmehr R (2006) Interacting adaptive processes with different timescales 776 underlie short-term motor learning. Plos Biology 4:1035-1043.
32
777 Thoroughman KA, Shadmehr R (2000) Learning of action through adaptive combination of motor 778 primitives. Nature 407:742-747. 779 Todorov E, Jordan MI (2002) Optimal feedback control as a theory of motor coordination. Nat 780 Neurosci 5:1226-1235. 781 Wada Y, Kawabata Y, Kotosaka S, Yamamoto K, Kitazawa S, Kawato M (2003) Acquisition and 782 contextual switching of multiple internal models for different viscous force fields. Neurosci 783 Res 46:319-331. 784 Wagner MJ, Smith MA (2008) Shared Internal Models for Feedforward and Feedback Control. Journal 785 of Neuroscience 28:10663-10673. 786 Wolpert DM, Diedrichsen J, Flanagan JR (2011) Principles of sensorimotor learning. Nature Reviews 787 Neuroscience 12:739-751. 788 Yousif N, Diedrichsen J (2012) Structural learning in feedforward and feedback control. J 789 Neurophysiol 108:2373-2382. 790
33