J Comput Neurosci (2011) 30:555–565 DOI 10.1007/s10827-010-0278-8

Relating reflex gain modulation in posture control to underlying neural network properties using a neuromusculoskeletal model

Jasper Schuurmans · Frans C. T. van der Helm · Alfred C. Schouten

Received: 25 March 2010 / Revised: 1 September 2010 / Accepted: 14 September 2010 / Published online: 24 September 2010 © The Author(s) 2010. This article is published with open access at Springerlink.com

Abstract During posture control, reflexive feedback the neural, sensory and synaptic parameters on the joint allows humans to efficiently compensate for unpre- dynamics. The results showed that the lumped reflex dictable mechanical disturbances. Although reflexes gains positively correlate to their most direct neural are involuntary, humans can adapt their reflexive set- substrates: the velocity gain with Ia afferent velocity tings to the characteristics of the disturbances. Reflex feedback, the positional gain with muscle stretch over modulation is commonly studied by determining reflex II afferents and the force feedback gain with Ib afferent gains: a set of parameters that quantify the contribu- feedback. However, position feedback and force feed- tions of Ia, Ib and II afferents to mechanical joint be- back gains show strong interactions with other neural havior. Many mechanisms, like presynaptic inhibition and sensory properties. These results give important and fusimotor drive, can account for reflex gain mod- insights in the effects of neural properties on joint ulations. The goal of this study was to investigate the dynamics and in the identifiability of reflex gains in effects of underlying neural and sensory mechanisms experiments. on mechanical joint behavior. A neuromusculoskeletal model was built, in which a pair of muscles actuated a Keywords Reflexes · Afferent feedback · Reflex limb, while being controlled by a model of 2,298 spiking gains · Sensitivity analysis · System identification in six pairs of spinal populations. Identical to experiments, the endpoint of the limb was disturbed with force perturbations. System identification was 1 Introduction used to quantify the control behavior with reflex gains. A sensitivity analysis was then performed on the neu- During posture control, humans have two strategies romusculoskeletal model, determining the influence of to counteract unexpected disturbances and keep their equilibrium position. Co-contraction of the muscles increases joint viscoelasticity, while maintaining equi- Action Editor: Simon R. Schultz librium. Co-contraction provides instantaneous vis- The model presented in this article is available on our coelasticity at the expense of high metabolic energy department’s website: http://nmc.3me.tudelft.nl. consumption. Sensory feedback is energy efficient, al- J. Schuurmans (B) · F. C. T. van der Helm · A. C. Schouten though effectiveness is limited due to the inherent Department of Biomechanical Engineering, neural time delays. Delft University of Technology, Mekelweg 2, In the presence of fast unpredictable disturbances 2628 CD Delft, The Netherlands e-mail: [email protected] in the upper extremity, reflexes through afferent feed- back provided by muscle spindles and Golgi tendon F. C. T. van der Helm · A. C. Schouten organs are the major contributors to sensory feedback. Laboratory of Biomechanical Engineering, MIRA Institute Muscle spindles are located in the muscles and provide for Biomedical Technology and Technical Medicine, University of Twente, P.O. Box 217, feedback on stretch and stretch velocity. Golgi tendon 7500 AE Enschede, The Netherlands organs are located in the junction between muscle and 556 J Comput Neurosci (2011) 30:555–565 tendon, providing feedback on muscle force. Reflexes goal of this study was to investigate how neural and are involuntary responses to stimuli, and their strength sensory properties could map onto the (limited) set of is known to be modulated. Experiments have shown reflex gains as obtained from joint dynamics, using a that different task instructions (Doemges and Rack neuromusculoskeletal model. In other words: how do 1992a, b; Abbink et al. 2004), disturbance properties the underlying, physical neural and sensory properties (de Vlugt et al. 2002) and dynamic properties of the effectively link to the lumped reflex gains? We used environment (Van der Helm et al. 2002) all elicit adap- a neuromusculoskeletal model of a spinal neural net- tation of the reflexive contribution to joint dynamics. work controlling an antagonistic pair of muscles with Model simulations have indicated that these reflex set- afferent feedback (Stienen et al. 2007). In a sensitivity tings are close to optimal for suppressing the distur- analysis, sensory and neural properties in the model bances (Schouten et al. 2001). were systematically varied and reflex experiments were Additional to the many studies that quantify simulated to determine reflex gains. The results showed reflexive behavior by directly deriving metrics like in- that besides the expected mappings, there also exist tegrated EMG after the application of a mechanical intricate, counterintuitive relationships between neural or electrical stimulus (e.g. Dewald and Schmit 2003; and sensory properties and reflex gains that need to Schuurmans et al. 2009; Kurtzer et al. 2008; Nielsen be taken into account when interpreting experimental et al. 2005), other studies use a control engineering results. approach to parameterize joint dynamics. A combi- nation of system identification and modeling is then used to separate muscular and reflexive contributions 2 Methods (Kearney et al. 1997; Perreault et al. 2000;Vander Helm et al. 2002; Ludvig and Kearney 2007; Schouten 2.1 Approach et al. 2008a). The dynamic behavior of the joint is estimated by perturbing the joint with a random force Posture control experiments around the shoulder joint disturbance using a robotic manipulandum and record- (Van der Helm et al. 2002) were mimicked on a ing displacement and interaction force between the neuromusculoskeletal (NMS) model. In these types of subject and the manipulandum. The joint dynamics are experiments, a subject holds the handle of a linear expressed in terms of the mechanical admittance, de- manipulator and minimizes deviations while being per- scribing the amount of displacement per unit force. To turbed with a random force disturbance. This type of separate reflexive contributions from muscle viscoelas- posture control experiment was mimicked by perturb- ticity, lumped parameters (Schouten et al. 2008b)can ing a joint in a neuromusculoskeletal model with a be fitted to the mechanical admittance. The resulting continuous random force disturbance. The reflexive set of parameters quantify the joint inertia, the muscle contributions to the dynamic behavior of the joint viscoelasticity and the reflexive gains of the force, posi- were determined using system identification techniques tion and velocity feedback pathways. identical to experiments in vivo. The reflexive com- There are multiple mechanisms that affect the ponent was expressed in terms of reflex gains, which strength of the reflex pathway. Fusimotor drive (Crowe gave a quantitative measure for the amount of stretch, and Matthews 1964; Ribot-Ciscar et al. 2009) mod- stretch velocity and force related reflex action. In the ulates the ’s sensitivity to stretch and sensitivity analysis the model parameters were system- stretch velocity, therewith altering the gain of the reflex atically varied and the effects on task performance, pathway. With presynaptic inhibition (Rudomin and joint admittance and the reflex gains were determined. Schmidt 1999; Rudomin 2009; Baudry et al. 2010), the efficacy of transmission between primary afferent fibers 2.2 Model description and the receiving neurons can be centrally modulated. The strengths of interneuronal connections in the spinal The NMS model is briefly described here; for full cord affect the net amount of activation of the mo- details see Stienen et al. (2007). Posture control of toneuron pool due to the sensory feedback. All these the shoulder was modeled as a one degree of free- mechanisms contribute to the joint dynamics as deter- dom joint (inertia m = 0.18 kg m2), actuated by an an- mined experimentally. tagonistic pair of muscles (moment arm rm = 30 mm). Despite the known neural mechanisms of reflex Since the experiment involved only small deviations modulation and the available identification techniques, around a constant level of co-contraction (≈40% of there are still many unknowns on how exactly neural maximal force), a linearized muscle model was used. mechanisms affect the measured joint dynamics. The The viscoelasticity due to muscle co-contraction was J Comput Neurosci (2011) 30:555–565 557

IA IA Legend set, such that the rotational viscoelasticity of the model 196 196 was equivalent to the translational viscoelasticity taken Neurons: XX Population of n from experimental data (Van der Helm et al. 2002): RC RC n neurons of type XX 196 196 the endpoint stiffness and viscosity of the arm were : respectively 800 N/m and 40 Ns/m. Stiffness and vis- MN Eff. Eff. MN Excitatory cosity represented cross-bridge viscoelasticity, which is 169 169 Excitatory, double strength dominant in an experiment with co-contraction and Excitatory, long time constant IN EX EX IN Tonic descending excitation relatively small displacements. The muscle activation 196 196 196 196 dynamics were of first order with a time constant of Inhibitory 30 ms. IB IB 196 196 Afferent feedback to the spinal network was pro- vided by Golgi tendon organs (force, through Ib afferents) and muscle spindles (stretch velocity and stretch, through Ia and II afferents). Each of the 121 Ib II II Ib Golgi tendon organs per muscle fired a spike train with a spike rate rIb that was proportional (Crago et al. 1982) to the muscle force with a constant cIb in spikes/s Ia Ia (Eq. (1)). Flexor Extensor Fm(t) rIb = cIb (1) Fmax Perturbation force where Fm is the muscle force, Fmax is the maximal muscle force (800 N) and t denotes time. A Poisson Fig. 1 Neuromusculoskeletal model. A muscle pair actuated a process was used to convert spike rate rIb to the spike one degree of freedom joint while being controlled by a spinal trains of the individual fibers. The afferent time delay network with populations of motoneurons (MN), group Ia in- terneurons (IA), Renshaw cells (RC), inhibitory of the Ib fibers was 15 ms. (IN), excitatory interneurons (EX) and group Ib interneurons A model of the muscle spindle (Prochazka and (IB). Feedback is provided by Ia, Ib and II afferents Gorassini 1998) was used to determine the firing rates of the 121 Ia and II afferent fibers per muscle as a function of muscle stretch (xm) and stretch velocity potential, variable threshold, potassium conductance (x˙m). The Ia afferent firing rate rIa was the summation and synaptic conductance. Whenever the membrane of a background firing rate (constant aIa), a linear potential reached threshold, the fired a dis- length dependent part (cIa) and an exponential stretch crete spike which was transmitted to the connected velocity dependent part (constants dIa, eIa). For the II synapses. The synaptic connections between the neu- afferents (rII), which were assumed to only transmit rons were created according to the connection scheme length-dependent information, the same model was in Fig 1. Tonic, descending excitation (TDE) provided used without the velocity-dependent part. background activity to the motoneurons (resulting in co-contraction) and to some of the other neural pop- ( ) = + · ( ) + · ˙eIa ( ) rIa t aIa cIa xm t dIa xm t (2) ulations. Each neuron in a receiving population was connected to 34–232 neurons, afferent fibers or de- rII(t) = aII + cII · xm(t) (3) scending fibers. Generally, the afferent input fans out The afferent time delays of the group Ia and II fibers over the populations. The connections with the lower were respectively 15 ms and 30 ms. number of synapses are closer to the afferent input The spinal neural network, which integrated the than the connections with high number of synapses afferent input to generate the efferent control signals to (the interneuronal connections). A full overview of the muscles, was based on Bashor (1998) and presented count can be found in Stienen et al. (2007). before in Stienen et al. (2007). The model consisted of The individual projections were randomized. Five pre- six pairs of spinal neuron populations, i.e. motoneu- set types of synaptic connections were used: single, dou- rons, Renshaw cells, group Ia and Ib interneurons ble and triple strength excitatory synapses, excitatory and inhibitory and excitatory interneurons (see Fig. 1). synapses a with long time constant (to the Renshaw Each population consisted of either 169 or 196 indi- cells), and inhibitory synapses. No network training or vidual spiking neurons (MacGregor and Oliver 1974). any form of neural plasticity was implemented. Since These neurons have four state variables, i.e. membrane the many motoneurons in a population all activated a 558 J Comput Neurosci (2011) 30:555–565 single, lumped muscle (no individual muscle fibers and model onto the joint dynamics. The reflex gain model a single ), the input signal to the is illustrated in Fig. 2. The model input was disturbance muscle activation dynamics was obtained by taking a force d and output was joint position xˆ. The intrinsic 20 ms moving average of the summed spike output of dynamics were parameterized by the inertia of the arm the motoneuron populations. Efferent time delay was m and the muscle stiffness k and viscosity b. Reflexive 10 ms. feedback consisted of position feedback (with a gain kp), velocity feedback (gain kv) and force feedback 2.3 Simulation of the neuromuscular model (gain kf ). A single reflexive feedback neural time delay τdel is represented by Hdel. Like in the simulated NMS Similar to posture control experiments a crested multi- model, the muscle activation dynamics Hact were rep- sine disturbance signal (Pintelon and Schoukens 2001) resented by a first order system with time constant τact. with a flat power spectrum between 0.5 Hz and 20 Hz The reflex gain model transfer function Hmod was was applied to the joint of the NMS model. The mag- described by: nitude of the disturbance was chosen such that the root mean square (RMS) of the endpoint displacements was = 1 Hact τ + (4a) approximately 4 mm, similar to experiments on human acts 1 subjects (Van der Helm et al. 2002). A single model −τ H = e dels (4b) run simulated a 9 second perturbation experiment. The del model was run with a discrete time step of 1 ms. To ˜ Fnet 1 Hi = = (4c) prevent transient behavior from influencing the results Fint 1 + kf · Hact · Hdel the first 808 ms were rejected, leaving exactly 213 data · samples. To account for possible variability due to the ˜ = Fnet = Hact Hdel Hr + · · (4d) random processes involved in the generation of the Fref 1 Hact Hdel kf force disturbance and the Poisson spike trains, each ˆ simulation was repeated eight times with a different X 1 Hmod = = (4e) 2 ˜ ˜ initial seed of the random generators. D ms + (bs+ k)Hi + (kvs + kp)Hr ˜ ˜ 2.4 Lumped reflex gain model For convenience, two intermediate variables Hi and Hr describe the effect of the force feedback loop on the After simulation of a perturbation experiment, reflex afferent and reflexive velocity and position feedback gains were determined by fitting a lumped reflex gain loops (from intrinsic muscle force Fint and reflexive muscle force Fref to net force Fnet). Using these inter- mediate variables, the entire model transfer function Muscle viscoelasticity Hmod can be expressed in a form similar to that of a mass-spring-damper system (see also Schouten et al. k 2008b). b The eight model parameters m, b, k, k , kv, k , τ d p f del Fint τ

‹ and act (see also Table 1) were fitted onto the data Fnet 1 x with a criterion function J which minimized the error m between arm position x of the neuromusculoskeletal Fref kf

Hact Hdel kv Table 1 Parameters of the lumped reflex gain model kp Reflex gains Lumped Unit Description parameter Fig. 2 Lumped reflex gain model used to fit reflex gains onto mkg· m2 Joint inertia the output of the perturbation experiments of the neuromuscu- bNm· s/rad Muscle viscosity loskeletal model. In this lumped model, the force disturbance d is kNm/rad Muscle stiffness applied to a single inertia m. Muscle viscoelasticity is represented kp Nm/rad Position feedback gain by a stiffness k and viscosity b. Reflexive feedback is represented kv Nm · s/rad Velocity feedback gain by a positional feedback gain kp, a velocity feedback gain kv and kf Nm/Nm Force feedback gain a force feedback gain kf . A single reflexive feedback neural time τdel s Neural time delay delay τdel is represented by Hdel. The first order muscle activation τact s Muscle activation time constant dynamics are Hact. Output of this lumped model is joint position xˆ J Comput Neurosci (2011) 30:555–565 559

Table 2 Model parameters in Neural model parameter Description the sensitivity analysis and their description Sensory constants aIa Muscle spindle constant aIa (Eq. (2)) cIa Muscle spindle constant cIa (Eq. (2)) dIa Muscle spindle constant dIa (Eq. (2)) eIa Muscle spindle constant eIa (Eq. (2)) aII Muscle spindle constant aII (Eq. (3)) cII Muscle spindle constant cII (Eq. (3)) cIb constant cIb (Eq. (1)) Transport delays τIa Ia afferent transport delay τIb Ib afferent transport delay τII II afferent transport delay τEff Efferent transport delay Synaptic weights (between neurons) wRC−MN Synaptic weight → motoneuron wIA−MN Synaptic weight Ia → motoneuron wIN−MN Synaptic weight inhibitory interneuron → motoneuron wEX−MN Synaptic weight excitatory interneuron → motoneuron wMN−RC(L) Synaptic weight motoneuron → Renshaw cell (long time constant) wMN−RC(S) Synaptic weight motoneuron → Renshaw cell (short time constant) wRC−RC Synaptic weight Renshaw cell → Renshaw cell (reciprocal) wIA−RC Synaptic weight Ia interneuron → Renshaw cell wRC−IA Synaptic weight Renshaw cell → Ia interneuron wIA−IA Synaptic weight Ia interneuron → Ia interneuron (reciprocal) wIB−IN Synaptic weight Ib interneuron → inhibitory interneuron wIB−EX Synaptic weight Ib interneuron → excitatory interneuron Synaptic weights (afferents to neurons) wia−IB Synaptic weight Ia afferent → Ib interneuron wia−IN Synaptic weight Ia afferent → inhibitory interneuron wia−MN Synaptic weight Ia afferent → motoneuron wia−IA Synaptic weight Ia afferent → Ia interneuron wib−IN Synaptic weight Ib afferent → inhibitory interneuron wib−IB Synaptic weight Ib afferent → Ib interneuron wii−EX Synaptic weight II afferent → excitatory interneuron wii−IA Synaptic weight II afferent → Ia interneuron Tonic descending excitation wtd−MN Synaptic weight descending excitation → motoneuron wtd−RC Synaptic weight descending excitation → Renshaw cell wtd−IA Synaptic weight descending excitation → Ia interneuron wtd−EX Synaptic weight descending excitation → excitatory interneuron wtd−ALL Synaptic weight descending excitation → all receiving neurons model and the output of the lumped reflex gain mo- toolbox.1 The result of the fitting procedure was a del xˆ: set of 8 parameters that describe the joint dynamics,    2 including reflexive contribution, in the same way as J = x(t ) − xˆ(t ) (5) i i done in experiments. i After fitting, the goodness of fit was expressed in where i indexes the time vector t. For efficiency, lumped variance accounted for (VAF): reflex gain model output xˆ was first determined in    2 the frequency domain and then inverse Fourier trans- x(ti) − xˆ(ti) formed to obtain: VAF = 1 − i  (7) 2 −1 x(ti) xˆ = F (D · Hmod) (6) i where D is the Fourier transform of disturbance force d. The criterion J was minimized using the least squares algorithm lsqnonlin from the Matlab optimization 1The Mathworks, USA. 560 J Comput Neurosci (2011) 30:555–565

A VAF value of 1 indicates a perfect match between 4 the lumped reflex gain fit and the NMS model output. 2 Besides the parameters of the lumped reflex gain model 0 and the VAF, the RMS value of the joint deviation (in radians) was determined to get a measure of perfor- – 2 mance in counteracting the disturbance. – 4 3 3.5 4 4.5 5 5.5 6 6.5 7 Disturbance torque [Nm]

2.5 Data analysis 4 neural simulation reflex gain fit A sensitivity analysis was performed where the neural 2 and sensory parameters in the NMS model were sys- tematically varied. The effects on the lumped reflex 0 gains (Table 1) and the performance in terms of dis- Joint angle [deg] – 2 turbance suppression (RMS) were determined. The 3 3.5 4 4.5 5 5.5 6 6.5 7 36 parameters included in the analysis were: muscle Time [s] spindle constants (6), the Golgi tendon organ constant Fig. 3 Four-second segment of a perturbation experiment on the (1), synaptic weights between afferents and the neuron NMS model and the output of the lumped reflex gain fit for populations (8), synaptic weights between the neuron a single condition. Disturbance torque (top) and resulting arm populations (12), synaptic weights between descending motion (bottom). Simulation experiment with the NMS model excitation and each neuron population separately (4), (solid) and the fit of the lumped reflex gain model (dashed). In this case VAF of the fit was 0.95 synaptic weights between descending excitation and all neuron populations simultaneously (1), and afferent and efferent time delays (4). A complete overview of descending excitation (TDE) was minimal, causing the parameters including their description is listed in some of the neural populations to completely cease Table 2. activity. The average VAF of the remaining 215 fits was One by one each parameter was simulated at 0.5, 0.9, 0.95 with a standard deviation of 0.014 and a minimum 1.0, 1.1, 1.5 and 2.0 times its nominal value, with all of 0.84. other parameters kept to their nominal value. For each Table 3 lists the lumped reflex gains resulting from value, a single set of reflex gains was fitted onto the a simulation run with all neural and sensory parame- data of the eight simulation repetitions. A sensitivity ters at their nominal values. The simulation results measure was defined by taking the slope of a linear are generally close to the experimental results (within regression through the six resulting reflex gain values one SD of the experiments by Schouten et al. 2008a), (Fig. 4). To allow for comparisons between the different except for neural time delay τ , which seems to be sensitivities the sensitivity measure was normalized del underestimated in the model simulation result. with the reflex gain value when all neural parameters As an example, Fig. 4 shows how lumped reflex gains had their default, nominal value (relative sensitivity, were modulated by varying neural parameter d ;the see Frank 1978). So the sensitivity measure gave the Ia velocity component of the muscle spindles. The sensi- relative amount of change in a fitted lumped reflex gain tivity measure S is indicated in the figure. The example as the result of a changing neural or sensory parameter. ij Figure 4 illustrates this process for the three reflex gains kp, kv, kf and the RMS of joint deviation. Table 3 Estimated lumped reflex parameters of the neuromus- cular model with all neural parameters at their nominal values 3Results Lumped Value Unit reflex gain m 0.178 kg · m2 Figure 3 illustrates the results of a single model simu- b 2.99 Nm · s/rad lation. The top panel shows the multisine disturbance k 90.5 Nm/rad ( ) force d t acting on the joint. The resulting joint rota- kp 19.2 Nm/rad tion x(t) of the NMS model is illustrated in the bottom kv 3.39 Nm · s/rad panel, together with the reflex gain model fit xˆ(t).Of kf 0.384 Nm/Nm τ all reflex gain model fits, one fit with a VAF of 0.48 del 15.0 ms τ was rejected. This was the condition in which tonic act 47.5 ms J Comput Neurosci (2011) 30:555–565 561

25 5 S = –0.33 S = 0.38 tonic muscle activation. Further, viscosity was increased 4,3 5,3 20 4 by various excitatory pathways to the motoneurons and 15 3 decreased by inhibitory pathways via the inhibitory interneuron populations. Sensitivity of stiffness k is

[Nm/rad] 10 2 [Nm s/rad] p remarkably low (in the same order of magnitude as that v k 5 k 1 of inertia m). Because of the linearized muscle model,

0 0 one might expect similar sensitivity of stiffness and 0.5 1 1.5 2 0.5 1 1.5 2 viscosity. The difference is due to the relatively high d /d [–] d /d [–] Ia Ia,0 Ia Ia,0 value of stiffness k in the nominal condition, decreasing 0.5 the normalized sensitivity. S = –0.45 S = –0.12 6,3 0.015 9,3 The middle row in Fig. 5 illustrates the sensitivity 0.4 of reflex gains kp, kv and kf to the eight parameters 0.3 0.01 that they were most sensitive to. Velocity feedback 0.2 over the Ia afferent decreased kp. This is indicated [Nm/Nm] f RMS [rad] k 0.005 by the negative sensitivity to muscle spindle constant 0.1 dIa, and also by the positive sensitivity to inhibitory 0 0 pathways from Ia afferents to motoneurons. Group 0.5 1 1.5 2 0.5 1 1.5 2 d /d [–] d /d [–] II afferent feedback increased kp as expected. Ren- Ia Ia,0 Ia Ia,0 shaw cell activation increases position feedback gain Fig. 4 Sensitivity of reflex gain parameters kp, kv, kf and RMS kp, both through the long latency constant synapse of joint deviation to the velocity component d of the muscle Ia (wMN−RC(L)) and the synapse between the Renshaw spindle. Lines indicate the linear regression fit; the normalized cells and the IA interneuron (wRC−IA). Velocity reflex slope determined the sensitivity measure Sij gain kv increases with the stretch velocity components in the Ia afferent pathway (dIa, eIa) and with increas- ing strength of the synapse between Ia afferent and shows that when the velocity component dIa in the motoneuron. TDE to either the motoneurons alone or output of the muscle spindle increases, velocity feed- to all populations decreases kv. Further, kv decreases back gain kv increases as expected. Position and force with the stretch component of the Ia afferent, and the feedback gains kp and kf both decrease, demonstrating pathway over inhibitory interneurons. Activation by that position feedback gain kp and force feedback gain excitatory interneurons decreased kv, which might be kf also depend on velocity component dIa of the muscle caused by these interneurons receiving stretch feedback spindle. Performance increased with dIa, indicated by from the II afferent and no velocity feedback from the the decreasing RMS value of the joint deviation: as the Ia afferent. Force feedback gain kf strongly decreased amount of velocity feedback from the muscle spindles with TDE and Ia afferent feedback and increased increased, joint deviations decreased. In a position task, with Golgi tendon organ (GTO) feedback over the small deviations indicate good performance. Ib afferent pathway. The neural time delay of the Ib The sensitivity of the lumped reflex gains to varia- afferent increased kf as well. tions in the neural and sensory parameters of the NMS The bottom row in Fig. 5 illustrates the sensitivities model is illustrated in Fig. 5. For each of the parameters of neural time delay τdel, muscle activation time con- of the lumped reflex gain model and the RMS of the stant τact and the RMS of joint deviation. Neural time joint deviation, the eight neural and sensory parameters delay estimate τdel was highly sensitive to the Ia afferent with the largest effect (per parameter) are shown. The and efferent neural time delays as expected, and further height of the bars shows the magnitude of the sensitivity to a mixture of neural and sensory parameters. The metric Sij, plus and minus signs above the bars indicate high sensitivity of the estimated muscle activation time the sign of Sij. constant τact to a wide range of neural and sensory The top row in Fig. 5 shows the sensitivity of the parameters is surprising (note that muscle activation intrinsic parameters: inertia m, joint viscosity b and was not varied in the simulations). joint stiffness k. Sensitivity of the inertia estimate is low The bottom-right panel in Fig. 5 shows how the RMS for all parameters (sensitivity in the order of 0.05)as of the joint deviations changed with the neural para- can be expected, since mass was not varied. Joint viscos- meters. TDE had the strongest effect on performance, ity b is mainly affected by tonic descending excitation. by increasing motoneuron activity and therewith co- Increasing TDE to specifically the motoneurons or to contraction. Further, Ia afferent feedback (both synap- all populations increased viscosity by increasing the tic strength and muscle spindle properties) improved 562 J Comput Neurosci (2011) 30:555–565

1.5 1.5 1.5 m [kgm2] b [Nm s/rad] k [Nm/rad] + + 1 1 1 [–] ij S 0.5 0.5 0.5 + + + – – + – + + – – + – + + + + – – ––+ 0 0 0 τ Ia τ e Ia IN a Ia IN τ Ia d Ia c Ib c Ia Eff MN – MN MN –IN – EX MN –IA ia – –ALL –MN – ib ia – –ALL – td td– RC (S) td–RCw td ia w w w td td IN–MN w w td–ALL w – w w w td w w EX w td w w w MN 1.5 1.5 1.5 – k [Nm/rad] k [Nm s/rad] – k [Nm/Nm] p v – f 1 1 + 1 – [–] ij – – + – S 0.5 0.5 + 0.5 – + – – + + – + + + + – + – 0 0 0 d Ia c II IA τ Ia e Ia d Ia c Ia c Ia τ Ib τ Ia d Ia IB MN MN MN ALL MN ia–IN –EX ib– w RC– IB td–MN ia– td–MN ia– w w IN– w w w td–ALLw w IN– w w EX–MN w w td– w w MN–RC (L) 1.5 1.5 1.5 τ [s] τ [s] RMS [rad] del act + – 1 1 1 – – –

[–] + + ij – S + + + + + – 0.5 0.5 – 0.5 + – – – – – + – + 0 0 0 τ τ τ τ Eff Ia e Ia e Ia d Ia IB Ib e Ia d Ia Ia τ Eff MN MN MN MN MN ALL MN –ALL –MN ia– –MN – IN– td–MN ia– td ia– IN– w td ia– w td–EX w td–ALL w w w EX– w w td w w w w w td w

Fig. 5 Sensitivity measure Sij for the eight lumped reflex gain force disturbances result in small deviations. For each graph, only model parameters (m, b, k, kp, kv, kf , τdel, τact)andRMSof the eight parameters with the highest sensitivity values are shown the joint position. Low RMS indicates high task performance: the

performance while neural time delays decreased with velocity gain kv, with relatively low interaction performance. Force feedback over Ib afferents and with the other reflex gains. Position feedback gain kp position feedback over II afferents only had a small and force feedback gain kf however did not show such effect on performance. a distinct sensitivity. There was positive sensitivity of Figure 5 demonstrates that most lumped reflex gains kp to the stretch component of the muscle spindle cII, were sensitive to a mixture of neural and sensory pa- but kp decreased with velocity component dIa as well. rameters. To elucidate the relation between the prop- The GTO constant cIb ledtoanincreaseofkf , but the erties of the proprioceptors and the estimated reflex spindle parameters aIa, cIa and dIa have a far stronger gains, Fig. 6 shows a subset of the data: the sensitivity of negative (decreasing) effect on kf . Summarizing, Fig. 6 kp, kv and kf to only the sensory constants of the muscle demonstrates that kv and kf are mostly influenced by spindle and GTO. The velocity components of the mus- muscle spindle feedback, while kp is influenced by a cle spindle (constants dIa and eIa) positively correlated mixture of afferent feedback pathways. J Comput Neurosci (2011) 30:555–565 563

1 k in the example of presynaptic inhibition and fusimotor p drive, investigate how the net synaptic weights between k v afferents and motoneurons or the parameters of the k 0.5 f muscle spindle model affect the lumped reflex gains. We simulated posture control experiments with a neuromusculoskeletal model. In the NMS model, spinal

[–] cord circuitry controlled a joint, while receiving feed-

ij 0 S back from muscle spindles and Golgi tendon organs in the activated muscles. The joint was perturbed with –0.5 a random force disturbance. Like in experiments, the posture control behavior of the simulated NMS model was captured by fitting the parameters of a lumped reflex gain model onto the simulated data. The high –1 a Ia c Ia d Ia e Ia a II c II c Ib VAF of the reflex gain model fits onto the output of the NMS model indicated that the linear reflex gain model Fig. 6 Sensitivity measure Sij of the lumped reflex gains kp, kv, kf was able to adequately capture the behavior of the to the sensory parameters of the muscle spindle and Golgi tendon strongly nonlinear NMS model. This was greatly due to organs. The most closely related neural substrates of each reflex gain parameter are indicated with an asterisk (*), e.g: velocity the conditions in the simulated experiment (and com- feedback gain kv is expected to be closest related to the velocity monly used in in vivo experiments): during a posture components dIa and eIa. (See Eqs. 1–3 and Table 1 for a list of control task (minimize deviations), there is a constant these parameters) level of muscle co-contraction with small deviations around the equilibrium position of the joint. Therefore, the experimental conditions linearize the behavior of 4 Discussion the non-linear neural system. A comparison to actual experimental data showed that the reflex gains fitted Multiple mechanisms contribute to posture mainte- from the neuromuscular model closely resembled reflex nance. Perturbation experiments and system iden- gains identified in vivo. tification are widely used to assess posture maintenance The results showed a dominant role of tonic descend- in humans (e.g. Kearney et al. 1997; Perreault et al. ing excitation (TDE) on the parameters of the lumped 2000; Van der Helm et al. 2002; Ludvig and Kearney reflex gain model. TDE to the motoneurons (para- 2007). System identification and neuromuscular model- meter wtd−MN) increases tonic muscle activation (co- ing pose a powerful tool for quantifying posture main- contraction) and therewith the muscle viscoelasticity, tenance with a small set of control parameters. For in- expressed in parameters b and k. Parameter wtd−ALL stance, a position feedback gain expresses force buildup simultaneously increased the amount of TDE to both related to muscle stretch. However, many neural and motoneurons and all other populations receiving TDE. sensory properties contribute to these feedback gains. Besides viscoelasticity, also reflex gains kv and kf were When posture maintenance is captured in such a small influenced by these two TDE parameters; these reflex set of reflex parameters, interaction could be expected. gains decreased with TDE. In the simulations of the For instance, both decreasing presynaptic inhibition of NMS model, TDE increased activation of the receiving the Ia afferent and increasing fusimotor drive can in- populations. As a result, the neural network received crease the velocity feedback gain, since the net amount relatively less afferent input than tonic input, which of velocity related feedback in the control action to could explain the decreasing reflex gains with TDE. the muscle increases. These interactions complicate the The results have shown that each of the lumped interpretations of the results from reflex experiments. reflex gain parameters is positively sensitive to its most Neural and sensory mechanisms might affect multiple direct neural substrate: kv increases with parameters parameters of the lumped reflex model (see Figs. 5 related to Ia afferent stretch velocity feedback pathway, and 6). Since direct in vivo measurements of neural kp with the II afferent stretch feedback pathway, and and sensory properties are unfeasible during posture kf with Ib afferent force feedback pathway. For kv control experiments in humans, the goal of this study this most direct substrate was also the most dominant was to investigate how neural and sensory properties and interaction with other pathways was low. In con- could map onto the (limited) set of reflex gains. Or, trast, kp and kf showed stronger sensitivities to other 564 J Comput Neurosci (2011) 30:555–565

(and less directly related) neural and sensory mech- argue that under these conditions, the simple models anisms, which makes accurate identification difficult capture sufficient detail to describe the afferent feed- in the given experimental conditions. Both kp and kf back triggered by the perturbations, although we are decreased with velocity feedback over the Ia afferent aware of the intricate dynamics that muscle spindles pathway. Additionally, force feedback gain kf strongly and Golgi tendon organs can demonstrate (Mileusnic decreased with TDE. This is in concordance with exper- et al. 2006; Mileusnic and Loeb 2006). The sensory imental findings: force feedback modulation in humans models used had their output in spikes per second and was strongest in an experimental condition with low a Poisson process was used to convert output to spike co-contraction and low-frequent (slow) perturbations trains. Halliday and Rosenberg (1999) presented a (Mugge et al. 2010). Position feedback gain kp was re- point-process spectral estimate on a human Ia afferent markably less sensitive to neural and sensory property spike train, that closely matched the spectral properties changes than kv and kf . In the posture control task used of recorded spike trains, indicating that a point process in the experiments deviations were small, while high could describe the Ia afferent dataset. muscle stretch velocities resulted from the disturbance The spinal neural model was relatively straightfor- force. Therefore muscle length information played a ward. The spinal populations in the neural model were lesser role during the posture control task than velocity homogeneous, and connections between the individual feedback. These results suggest that to effectively iden- neurons in the populations were assigned using a ran- tify kp experimentally, stretch velocities need to be kept dom process. The neuron model was a simple point small to minimize the interactions between velocity and neuron, without spatial representations like a den- position feedback. drite structure. These are important simplifications, and Summarizing, the sensitivity analysis demonstrated more detailed models of the individual components of dominance of muscle spindle velocity feedback and our neural model do exist. Nevertheless, this simplicity tonic descending excitation. Velocity feedback gain kv serves a purpose. The main goal was to determine how was distinctly influenced by afferent velocity feedback changes in velocity-, position- and force-related neural and TDE, while position and force feedback gains kp feedback map onto the limited set of reflex gains that and kf depended on a wide variety of neural proper- are used in experiments. The main question was: which ties. The results of this modeling study contribute to neural pathways could be contributing when changes in determining experimental conditions suited for study- reflex gains are observed in an experiment? Unlike a ing reflex modulation in vivo. From reflex experi- lumped reflex gain model, where each feedback chan- ments (de Vlugt et al. 2002; Schouten et al. 2001), we nel has a distinct pathway, biological afferent feedback know that experimental conditions like level of co- fans out in the spinal populations before finally con- contraction, perturbation velocities and amplitudes, but verging at the motoneurons. The multitude of pathways also task instruction induce reflex gain modulation. and their interactions are the cause of most lumped When the relationship between neural properties and reflex gains being sensitive to multiple neural and sen- reflex gains is better understood, experimental condi- sory parameters. Each detail added to the neural model tions can be adapted to the type of reflex modulation adds one or more parameters and thus might add a new that is being studied. Furthermore, it is important to relationship with one of the reflex gains. However, the know if the experimentally estimated reflex gains are main phenomenon that was demonstrated here can be the result of the expected neural mechanism or may be captured with a relatively simple model that includes a combination of interacting mechanisms. For example, at least the afferent feedback pathways, fan-out and one must be aware that modulation of kv can, besides convergence in the spinal populations and output to a Ia afferent feedback, be caused by tonic descending musculoskeletal model. excitation to the motoneurons. It could therefore be We conclude that with postural control tasks and important to control the level of co-contraction in such wide-bandwidth force perturbations, parameters of the an experiment. Ia afferent feedback pathway largely contribute to the Like most models, the model presented here has task performance enabling accurate identification of many simplifications. Starting at the sensory level, kv. In these conditions of high stretch velocities, but a straightforward feline muscle spindle was chosen, low stretch and fairly constant muscle activation, pa- together with a linear Golgi tendon organ model. Be- rameters of the II and Ib afferent feedback pathways cause of the modeled task (a position task with co- hardly contribute to task performance. Because of this contraction and small, continuous perturbations with low contribution to task performance, the amount of fixed frequency content) both sensors operate with position and force related information in the measured small deviations around a relatively steady state. We joint deviations is low, resulting in poor identifiability J Comput Neurosci (2011) 30:555–565 565 of kp and kf . This is expressed by the high sensitivity to Kurtzer, I. L., Pruszynski, J. A., & Scott, S. H. (2008). Long- other, seemingly non-related feedback pathways. Iden- latency reflexes of the human arm reflect an internal model tification of k and k probably needs an experimental of limb dynamics. Current Biology, 18(6), 449–453. p f Ludvig, D., & Kearney, R. E. (2007). Real-time estimation of in- design that decreases the effects of interaction from trinsic and reflex stiffness. IEEE Transactions on Biomedical velocity feedback, like Mugge et al. (2010) suggests Engineering, 54(10), 1875–1884. MacGregor, R. J., & Oliver, R. M. (1974). A model for repetitive for kf . firing in neurons. Kybernetik, 16(1), 53–64. Mileusnic, M. P., Brown, I. E., Lan, N., & Loeb, G. E. (2006). Acknowledgement This research was funded by Dutch Gov- Mathematical models of proprioceptors. I. Control and ernment Grant BSIK03016. transduction in the muscle spindle. Journal of Neurophysi- ology, 96(4), 1772–1788. Open Access This article is distributed under the terms of the Mileusnic, M. P., & Loeb, G. E. (2006). Mathematical models of Creative Commons Attribution Noncommercial License which proprioceptors. II. Structure and function of the golgi tendon permits any noncommercial use, distribution, and reproduction organ. Journal of Neurophysiology, 96(4), 1789–1802. in any medium, provided the original author(s) and source are Mugge, W., Abbink, D. A., Schouten, A. C., Dewald, J. P., & credited. van der Helm, F. C. (2010). A rigorous model of reflex func- tion indicates that position and force feedback are flexibly tuned to position and force tasks. Experimental Brain Re- References search, 200(3–4), 325–340. Nielsen, J. B., Petersen, N. T., Crone, C., & Sinkjaer, T. (2005). Stretch reflex regulation in healthy subjects and patients Abbink, D. A., van der Helm, F. C. T., & Boer, E. R. (2004). with spasticity. Neuromodulation, 8(1), 49–57. Admittance measurements of the foot during ‘maintain po- Perreault, E. J., Crago, P. E., & Kirsch, R. F. (2000). Estimation sition’ and ‘relax’ tasks on a gas pedal. In Systems, man and of intrinsic and reflex contributions to muscle dynamics: A cybernetics, 2004 IEEE international conference on (Vol. 3, modeling study. IEEE Transactions on Biomedical Engi- pp. 2519–2524). neering, 47(11), 1413–1421. Bashor, D. P. (1998). A large-scale model of some spinal reflex Pintelon, R., & Schoukens, J. (2001). System identif ication; a circuits. Biological Cybernetics, 78(2), 147–157. frequency domain approach. New York, USA: IEEE Press. Baudry, S., Maerz, A. H., & Enoka, R. M. (2010). Presynaptic Prochazka, A., & Gorassini, M. (1998). Models of ensemble firing modulation of ia afferents in young and old adults when of muscle spindle afferents recorded during normal locomo- performing force and position control. Journal of Neuro- tion in cats. Journal of Physiology, 507(Pt 1), 277–291. physiology, 103(2), 623–631. Ribot-Ciscar, E., Hospod, V., Roll, J. P., & Aimonetti, J. M. Crago, P. E., Houk, J. C., & Rymer, W. Z. (1982). Sampling of (2009). Fusimotor drive may adjust muscle spindle feedback total muscle force by tendon organs. Journal of Neurophysi- to task requirements in humans. Journal of Neurophysiol- ology, 47(6), 1069–1083. ogy, 101(2), 633–640. Crowe, A., & Matthews, P. B. (1964). The effects of stimulation Rudomin, P. (2009). In search of lost presynaptic inhibition. of static and dynamic fusimotor fibres on the response to Experimental Brain Research, 196(1), 139–151. stretching of the primary endings of muscle spindles. Journal Rudomin, P., & Schmidt, R. F. (1999). Presynaptic inhibition of Physiology, 174, 109–131. in the vertebrate revisited. Experimental Brain de Vlugt, E., Schouten, A. C., & van der Helm, F. C. (2002). Research, 129(1), 1–37. Adaptation of reflexive feedback during arm posture to Schouten, A. C., de Vlugt, E., van der Helm, F. C., & Brouwn, different environments. Biological Cybernetics, 87(1), 10–26. G. G. (2001). Optimal posture control of a musculo-skeletal Dewald, J. P. A., & Schmit, B. D. (2003). Stretch reflex gain and arm model. Biological Cybernetics, 84(2), 143–152. threshold changes as a function of elbow stretch velocity in Schouten, A. C., de Vlugt, E., van Hilten, J. J., & van der Helm, hemiparetic stroke. In Engineering in medicine and biology F. C. (2008a). Quantifying proprioceptive reflexes during society, 2003. proceedings of the 25th annual international position control of the human arm. IEEE Transactions on conference of the IEEE (Vol. 2, pp. 1464–1467). Biomedical Engineering, 55(1), 311–321. Doemges, F., & Rack, P. M. (1992a). Task-dependent changes Schouten, A. C., Mugge, W., & van der Helm, F. C. (2008b). in the response of human wrist joints to mechanical distur- Nmclab, a model to assess the contributions of muscle visco- bance. The Journal of Physiology, 447(1), 575–585. elasticity and afferent feedback to joint dynamics. Journal of Doemges, F., & Rack, P. M. (1992b). Changes in the stretch Biomechanics, 41(8), 1659–1667. reflex of the human first dorsal interosseous muscle dur- Schuurmans, J., de Vlugt, E., Schouten, A. C., Meskers, C. G., ing different tasks. The Journal of Physiology, 447(1), de Groot, J. H., & van der Helm, F. C. (2009). The monosy- 563–573. naptic ia afferent pathway can largely explain the stretch du- Frank, P. M. (1978). Introduction to system sensitivity theory.New ration effect of the long latency m2 response. Experimental York: Academic. Brain Research, 193(4), 491–500. Halliday, D. M., & Rosenberg, J. R. (1999). Time and frequency Stienen, A. H., Schouten, A. C., Schuurmans, J., & van der domain analysis of spike train and time series data. In Helm, F. C. (2007). Analysis of reflex modulation with a bio- U. Windhorst, & H. Johansson (Eds.), Modern techniques logically realistic neural network. Journal of Computational in neuroscience research (pp. 503–543). Springer. Neuroscience, 23(3), 333–348. Kearney, R. E., Stein, R. B., & Parameswaran, L. (1997). Iden- Van der Helm, F. C., Schouten, A. C., de Vlugt, E., & Brouwn, tification of intrinsic and reflex contributions to human ankle G. G. (2002). Identification of intrinsic and reflexive com- stiffness dynamics. IEEE Transactions on Biomedical Engi- ponents of human arm dynamics during postural control. neering, 44(6), 493–504. Journal of Neuroscience Methods, 119(1), 1–14.