Biologically Inspired Neural Control Network for a Bipedal Walking Model

BIOLOGICALLY INSPIRED NEURAL CONTROL NETWORK

FOR A BIPEDAL WALKING MODEL

WEI LI

Submitted in partial fulfillment of the requirements

For the degree of Doctor of Philosophy

Dissertation Advisor:

Dr. Roger D. Quinn

Department of Mechanical and Aerospace Engineering

CASE WESTERN RESERVE UNIVERSITY

January, 2017

CASE WESTERN RESERVE UNIVERSITY

SCHOOL OF GRADUATE STUDIES

We hereby approve the dissertation of

Wei Li

Candidate for the degree of Doctor of Philosophy*

Committee Chair

Roger Quinn

Committee Member

Musa Audu

Committee Member

Kiju Lee

Committee Member

Richard Bachmann

Date of Defense:

11/28/2016

*We also certify that written approval has been obtained for any proprietary material contained therein.

Table of Contents

Table of Contents ...... i

List of Tables ...... v

List of Figures ...... vi

Abstract ...... xi

Chapter 1 Introduction ...... 1

1.1 Research Objectives ...... 4

1.2 Dissertation Structure ...... 4

Chapter 2 Background ...... 7

2.1 Introduction of CPG ...... 7

2.1.1 CPGs in Animals ...... 7

2.1.2 CPG Models ...... 8

2.2 Sensory Feedback in Walking ...... 12

2.2.1 Roles of Sensory Feedback ...... 12

2.2.2 Feedback of Leg Load and Hip Joint Movement ...... 14

2.3 Human Gait...... 15

2.4 Biologically Inspired Bipedal Walking Control ...... 18

2.4.1 Passive Dynamics ...... 18 i

2.4.2 CPG-based Control ...... 18

2.4.3 Reflex Control ...... 20

2.5 Features of This Dissertation ...... 21

Chapter 3 Neuromechanical Simulation ...... 23

3.1 Challenges in Researching Locomotion ...... 23

3.2 The Approach: Neuromechanical Simulation ...... 24

3.3 The Simulation Environment: Animatlab ...... 26

Chapter 4 Biomechanical model ...... 29

4.1 General Structure ...... 29

4.2 Foot Design...... 30

4.3 Muscle Model ...... 31

Chapter 5 Neural Control Network ...... 35

5.1 Neural Models ...... 35

5.1.1 Leaky Integrate-and-Fire Neuron Model ...... 35

5.1.2 Synapse Model ...... 38

5.1.3 Neural Facilitation Model ...... 39

5.2 Half-center Oscillator ...... 40

5.3 CPG Network Without Sensory Feedback ...... 44

5.4 Neural Network with Sensory Feedback ...... 50

5.4.1 Feedback of Ground Contact and Hip movement ...... 50

5.4.2 Architecture of the Neural Network ...... 53

5.5 Connecting Neural Network to Muscles ...... 56

Chapter 6 Simulation of Normal Walking ...... 60

6.1 Generation of Gait Cycle ...... 60

6.2 Kinematic Analysis ...... 68

6.2.1 Joint Angle...... 68

6.2.2 Displacement and Velocity ...... 72

6.3 Kinetic Analysis ...... 75

6.3.1 Ground Reaction Force ...... 75

6.3.2 Muscle Force ...... 77

6.3.3 Joint Moment...... 79

Chapter 7 Simulation of Walking with Perturbations ...... 82

7.1 Changing Neural Network Configuration ...... 82

7.2 Walking with Different Body Weights ...... 96

7.3 Walking with Horizontal Pelvis COM Displacements ...... 100

7.4 Walking Over Obstacle ...... 103

iii

7.5 Walking On Slopes ...... 106

Chapter 8 Conclusions and Future Works ...... 111

8.1 Contributions ...... 111

8.2 Future Works ...... 115

Appendix A: Neuron Properties ...... 119

Appendix B: Synapse Properties ...... 120

Appendix C: Synaptic Weights in the CPG Neural Network ...... 121

Bibliography ...... 122

List of Tables

Table 4.1: Mass and dimension of leg part...... 30

Table 4.2: Muscle parameters. EXT: extensor muscle, Flex: flexor muscle...... 34

Table 7.1: Walking performance after cutting off one synaptic connection...... 82

List of Figures

Figure 2.1: Classic half-center CPG model...... 9

Figure 2.2: Unit burst generator (UBG) model ...... 10

Figure 2.3: Rybak and McCrea’s two-level CPG model ...... 11

Figure 2.4: Two-level CPG model containing separated networks for rhythm generating and pattern formation ...... 12

Figure 2.5: Dynamic interaction between sensory afferents, spinal neuronal networks, and muscles in human walking ...... 13

Figure 2.6: Normal human gait and phase plot ...... 17

Figure 3.1: Neuromechanical simulations facilitating the interaction of robotics and biology...... 26

Figure 4.1: The biomechanical model...... 30

Figure 4.2: Human foot during stance phase...... 31

Figure 4.3: The structure of the two-part foot...... 31

Figure 4.4: Hill’s muscle model...... 32

Figure 4.5: Modeling muscle in Animatlab...... 33

Figure 4.6: The stimulus-tension curve (left) and the length-tension curve (right) of the hip extensor muscle...... 34

Figure 5.1: Simulation of a leaky integrate-and-fire neuron in Animatlab ...... 37

Figure 5.2: Neural facilitation and depression models...... 40

Figure 5.3: Simulation of a half-center oscillator in Animatlab...... 41

Figure 5.4: The output of a pattern generator entrained by IPSP ...... 43

Figure 5.5: The output of the pattern generator entrained by EPSP ...... 44

Figure 5.6: The architecture of a two-level CPG network...... 47

Figure 5.7: Membrane potentials of neurons in CPG network...... 48

Figure 5.8: Frames of the model “air stepping” without sensory feedback...... 49

Figure 5.9: Connecting sensors to the neural control network...... 52

Figure 5.10: Outputs of sensors and membrane potentials of sensory neurons ...... 53

Figure 5.11: The architecture of the neural network ...... 55

Figure 5.12: Connecting Hip PF neurons to hip extensor and flexor muscles ...... 57

Figure 5.13: Neuron and muscle membrane potentials, muscle tension force and muscle lengths...... 59

Figure 6.1: Model constrained in the sagittal plane by two vertical walls...... 60

Figure 6.2: Releasing the model at the beginning of walking...... 61

Figure 6.3: Comparison of the model and human walking at the same points in a gait cycle...... 64

Figure 6.4: Neuron membrane potentials in the walking cycle ...... 66

Figure 6.5: Phase plot of the model ...... 67

Figure 6.6: Comparison of the gait simulation with Klein and Lewis’ physical model...... 68

Figure 6.7: Definition of Joint angles and movements in the sagittal plane...... 69 vii

Figure 6.8: Hip, knee, ankle, toe joint angles in a gait cycle compared with human’s ...... 71

Figure 6.9: Hip-knee angle plots ...... 72

Figure 6.10: Vertical position and velocity of Pelvis COM in a gait cycle...... 74

Figure 6.11: The horizontal position of ankles in a gait cycle...... 75

Figure 6.12: Vertical ground reaction force compared with human’s in a gait cycle

...... 77

Figure 6.13: Muscle forces of the model in a gait cycle...... 78

Figure 6.14: Internal moments at hip, knee, and ankle in the sagittal plane compared with human’s in a gait cycle ...... 81

Figure 7.1: Neuron membrane potentials during walking without synaptic connection from Hip EXT neuron to Knee EXT neuron...... 85

Figure 7.2: Posture at the end of terminal stance ...... 86

Figure 7.3: Neuron membrane potentials during walking without synaptic connection from Heel GC neuron to Knee FLX inhibition neuron ...... 87

Figure 7.4: Neuron membrane potentials during walking without synaptic connection from Hip FLX neuron to Ankle DF neuron ...... 88

Figure 7.5: Comparison of ankle joint angles between with and without the synaptic connection from Hip FLX neuron to Ankle DF neuron ...... 89

Figure 7.6: Posture at the end of swing phase ...... 89

Figure 7.7: Neuron membrane potentials during walking without synaptic viii

connection from Heel GC sensory neuron to the Ankle PF inhibition integrator neuron...... 91

Figure 7.8: Posture following the initial contact ...... 91

Figure 7.9: Comparison of ankle joint angles between with and without the synaptic connection from Heel GC sensory neuron to the Ankle PF inhibition integrator neuron ...... 92

Figure 7.10: Neuron membrane potentials during walking without Toe GC feedback...... 93

Figure 7.11: Posture at the end of terminal stance ...... 94

Figure 7.12: Comparison of average walking speed of walking speed between models with and without Toe GC afferent ...... 94

Figure 7.13: Joint angles and horizontal velocity of pelvis COM...... 95

Figure 7.14: Posture in terminal swing ...... 96

Figure 7.15: Average walking speed of models with different pelvis masses ...... 97

Figure 7.16: Joint angles and pelvis COM velocities of models with 20 kg and 70 kg pelvises ...... 99

Figure 7.17: Average walking speed of models with different pelvis COM displacements ...... 101

Figure 7.18: Joint angles and pelvis COM velocities of models with 20 mm and 80 mm pelvis COM displacements...... 102

Figure 7.19: Walking over a 50 mm high obstacle ...... 106 ix

Figure 7.20: Walking up a 5° incline...... 108

Figure 7.21: Walking down a -5° decline...... 110

Figure 8.1: A nine-muscle human leg model ...... 116

Biologically Inspired Neural Control Network for A Bipedal Walking Model

ABSTRACT

WEI LI

This dissertation describes the development of a biologically inspired neural control network for planar human-like walking in the sagittal plane. The bipedal model is constructed as a simplified musculoskeletal system, with leg length about

0.84 m, to mimic the biomechanics of the human lower body. The leg model contains

3 active joints (hip, knee, ankle) driven by 6 muscles and a two-part foot with a passive joint. The neural network is composed of leaky integrate-and-fire neurons, which are organized as central pattern generators (CPGs) entrained by ground contact and hip joint movement sensory feedback to generate appropriate locomotor patterns for walking. The CPG model adopts a two-level architecture, which consists of separate rhythm generator (RG) and pattern formation (PF) networks. Bipedal walking is tested using neuromechanical simulation. Under the control of the dynamic neural network, the model walks stably with human-like gait in the sagittal plane without any inertial sensors or a central posture controller or a “baby walker” to help overcome gravity. The model’s walking speed varies from 0.61 m/s to 1.29 m/s, adapting to different horizontal COM displacements and pelvis mass settings. It walks over 50 mm high small obstacles, and up or down 5° slopes without any additional

higher level control actions. This model is flexible and expandable in further research.

Chapter 1 Introduction

The idea of creating machines that can walk like humans has fascinated people for generations. On one hand, various modern humanoid bipedal robots have been developed since the first fully articulated humanoid robot WABOT-1 (1973) [1]. Some well-known humanoid robots such as ASIMO (2000) [2], KHR-3 (2005) [3], LOLA

(2008) [4], HRP-4 (2010) [5], MABEL (2011) [6], and ATLAS (2013) demonstrate impressive walking performance in various environments but their control system is quite different from the human nervous system. They often rely on fast and powerful actuation and high-bandwidth, high-gain control approaches such as zero moment point [7], virtual model control [8], hybrid zero-dynamics [9], or optimal feedback control [10]. They require extensive computation and often consume more than 10 times the energy that a human does during walking [2].

Passive dynamic walkers provide another way of solving the problem. They are simple mechanical devices, composed of solid parts connected by joints, which are able to walk stably down a slope [11][12]. They have no controllers or actuators but their walking motion can be surprisingly human-like. Figure 1.1 illustrates that the simplest passive dynamic walker could be evolved from a rimless wheel rolling down a slope. With simple open loop control and a small amount energy input, robots based on passive dynamics can walk on level ground [13]. “…Passive walkers demonstrate

that the high-gain, dynamics-canceling feedback approach taken on ASIMO is not a necessary one…” [14]. More details of passive dynamics are given in section 2.4.1.

Figure 1.1: The evolution from a spoked wheel to a passive-dynamic walker. In a carefully designed rimless-wheel the potential energy could compensate the energy lost in every impact between ground and the spoke so it is able to roll down the slope with relatively constant velocity. The passive- dynamic walker walks down the slope with the similar mechanism (redrawn from T. Luksch [15]).

On the other hand, researchers strive to understand how the human nervous system controls walking. Researchers have known since the early 20th century that in vertebrate animals, it is the spinal cord rather than brain that plays the primary role in locomotion. In 1910 Sherrington [16] reported that decerebrated cats still perform stepping like movements. One year later his student Brown [17] made similar observations on decerebrated cats with the afferent nerves cut from hind-limb muscles. So Brown further concluded that “...mechanism confined to the lumbar part of the spinal cord is therefore sufficient to determine in the hind-limbs and act of progression...” Now it is commonly accepted that in vertebrate animals the basic locomotor pattern is generated centrally by spinal networks of neurons, called central pattern generators (CPGs) (reviewed by [18][19][20]). Sensory feedback, especially from afferents at the spinal level, adapts and modulates the activity of CPGs to the

environment (reviewed by [21]). The mechanisms that control bipedal human locomotion may be more complex than for quadruped animals but they share common systems including a spinal neuronal network modulated by sensory feedback [22].

So, can a controller be designed based on these findings in animal and human nervous systems to enable a bipedal model walk stably like a human? This dissertation strives to answer this question by developing a spinal-level neural network that can control the stable walking of a musculoskeletal bipedal model in the sagittal plane. Although there were some previous biologically inspired bipedal models, my work is novel in several ways (see section 2.4 and section 8.1).

Bipedal walking is a difficult task: the system is nonlinear, under-actuated, contains many degree of freedoms, and evolves hybrid dynamics. The organization and functional properties of neuronal networks in the spinal cord are still not well understood. So the research presented in this dissertation is limited to the current state of knowledge. My bipedal model is constrained in the sagittal plane. It lacks lateral movement, which is dynamically unstable and I believe needs supraspinal control as well as additional spinal circuits. Therefore, the neural system I developed should not be viewed as a complete model of the human spinal neuronal network.

The aim of this dissertation was to develop biomechanical and neural models that are fundamental and accessible without sacrificing key components of vertebrate locomotion systems. The model is flexible and expandable. It can be used for (1)

designing a physical robot and its control system, or (2) as a platform to test hypotheses about circuits in the spinal cord related to locomotion in animals and human.

1.1 Research Objectives

Objectives of this dissertation are:

 Construct a numerical musculoskeletal bipedal model mimicking the

human lower body;

 Design and implementation of a synthetic nervous system for the model

based upon what is known about the spinal cord in vertebrates neural

control network composed of CPGs and entrained by sensory feedback;

 Analyze interactions between neural network, biomechanics, and sensory

feedback in neuromechanical simulations of normal walking;

 Analyze dynamics Kinematic analysis and kinetic analysis of the bipedal

model in gait cycles;

 Investigate how different synaptic connections in the neural network affect

the model’s walking behaviors and stability;

 Investigate stability of the model’s walking behaviors when terrain and

body configuration are changed.

1.2 Dissertation Structure

Chapter 2 reviews the animal literature underlying this work and the current

research status of biologically inspired approaches to control of bipedal robots. It reviews the concept of the CPG and some CPG models. It then summarizes the roles of sensory feedback in locomotion and its relationship with CPGs, especially the sensory feedback of load and hip joint movement. It also includes a brief review of human gait. Finally, it describes the contributions of this research relative to other representative biologically inspired control methods and their applications.

Chapter 3 introduces the neuromechanical simulation and the simulation software used in this work. It summarizes challenges in researching locomotion as a result of complex interactions between controls, mechanics and the environment. It then introduces Animatlab, which is an efficient software tool for neuromechanical simulations.

Chapter 4 describes the construction of the biomechanical model. It includes the geometry and mass of different parts and the detailed design of the two-part foot model. It also introduces Hill’s muscle model used in this research.

Chapter 5 describes the design and implementation of the neural control network.

It begins by introducing the neuron model, synapse model, and neural facilitation model used in this neural network. It then shows how these basic blocks are used to form a neural oscillator and further constitute a two-level CPG network in Animatlab.

Finally, it describes how the CPG network is accomplished by adding sensory afferents and connecting CPG neurons to muscles through motoneurons.

Chapter 6 presents a neuromechanical simulation of a planar bipedal walking model in Animatlab. It describes how the gait is generated through interactions between the neural network, the biomechanical model, and the environment. It then separately analyzes the performance of the model in kinematics and kinetics. In kinematics it focuses on trajectories of joint movements, displacement, and velocity in a gait cycle. In kinematics it mainly focuses on trajectories of ground reaction force, muscle tension force, and joint torque in a gait cycle. All these tests of the model are compared with corresponding human data.

Chapter 7 presents simulations of the model walking in different conditions including with changes to its neural network and physical environment. It discusses the model’s performance reacting to different sensory afferents and lesioned synaptic connections. It then shows how the model adapts to walking over small obstacles, walking on inclines, and walking on declines.

The dissertation is concluded in Chapter 8. It summarizes the contributions of this work and also provides suggestions on further research.

Chapter 2 Background

2.1 Introduction of CPG

2.1.1 CPGs in Animals

The networks of neurons capable of producing coordinated patterns of rhythmic activity without rhythmic input from sensory feedback or from higher control centers are referred to as “central pattern generators” (CPGs) [23]. The CPG concept is defined by behaviors it produces more than by anatomical boundaries [24]. In the past

40 years, supporting evidence has been found in many animals that rhythmic locomotor patterns are generated centrally without any descending or afferent inputs

(reviewed by [18-20]). The existence of such a spinal network in humans has been in debate for a long time [25]. A research team claimed “the first well-defined example of a central rhythm generator for stepping in the adult human” [26] based on a study of a patient with chronic incomplete spinal cord injury. Involuntary rhythmic movements of his lower extremities were triggered when the patient was positioned supine with hips in extension. Those movements were abolished by flexing the hips, by standing and when sleeping in the supine position. It was also reported that a patient with a complete spinal cord injury expressed EMG signals for movement occurring spontaneously and occasionally coordinated between extensors and flexors in a rhythmic pattern resembling bipedal stepping [27]. However, conclusive evidence of CPGs in humans and functioning in a similar way as in other animals is still

lacking.

2.1.2 CPG Models

The organization of a neuronal network of locomotor pattern generators in the spinal cord is still not clear, but CPG models with different architectures have been developed based on findings in animals’ locomotor patterns.

1) Half-center CPG model

The classic half-center CPG model [28] (Figure 2.1), made of two populations of neurons reciprocally connected and reciprocally inhibiting each other, can produce a basic rhythm for locomotion. Each limb is controlled by a separate CPG. Each CPG contains two groups of excitatory neurons that directly project to and control the activity of flexor and extensor motoneurons, respectively. Each half-center has a fatigue process gradually reducing its excitation so phase is switched when the excitation of one half-center falls below a threshold and the other center is released from inhibition. Direct evidence supports that half-center structured pattern generators exist in the spinal cord. Direct evidence support that half-center structured neuronal networks do exist in the spinal cord [29][30].

This half-center model is a single-level CPG architecture, in which CPG neurons are connected to corresponding motoneurons and strictly divides the activity of motoneurons in one limb into two groups (extension and flexion). However, in experiments with decerebrated cat’s “fictive locomotion”, it is possible for individual 8

group of motoneurons in the same limb to show different onset and offset activities

[31].

Figure 2.1: Classic half-center CPG model.

2) One-level UBG model

In animal’s locomotion, the CPG can produce more delicate pattern that can control activities of different groups of motoneurons [32]. Grillner and Zangger developed a Unit Burst Generator (UBG) model [33]. It is composed of individual units of half-center oscillators as pattern generators for joints of each limb (Figure

2.2). Different units are coupled by excitatory synaptic connections. The UBG model generates specific patterns in each unit which project to corresponding motoneurons, but it is still a single-level architecture because the cycle period of the whole model depends on the excitability of individual units and the interaction between them.

Figure 2.2: Unit burst generator (UBG) model.

3) Rybak, McCrea’s two-level models

Another observation referred to as “non-resetting deletions” [34][35][36][37], further challenges single-level CPG models. In the case of “non-resetting deletions,” rhythmic bursts in the motoneurons are absent, seemingly “deleted” in records, for a few cycles while motoneurons in antagonists show sustained or rhythm activities. It is more likely that stimulation inhibits a neural network common to multiple populations of motoneurons rather than inhibiting motoneurons themselves. In a single-level model the stimulation would affect both the locomotor cycle and motoneuron activity and the step cycle would be “reset” following the “deletion”.

To explain these complex patterns, Rybak and McCrea [38][39] proposed a two- level CPG model (Figure 2.3), composed of a “half-center” rhythm generator (RG) and a pattern formation (PF) network. The RG and PF layers have a similar “half- center” organization. The RG layer is the higher level and the PF half-centers are

strongly modulated by it. PF half-centers project to corresponding motoneuron populations to control extensors and flexors. During locomotion, afferent input could access RG and PF levels separately with different synaptic weights so they could change the PF level output without affecting the locomotor pattern produced at the

RG level.

Figure 2.3: Rybak and McCrea’s two-level CPG model.

4) Two-level UBG model

Rybak and McCrea further proposed a two-level UBG model [36] [38] (Figure

2.4), in which a rhythm generator is added to the top of the one-level UBG architecture. Original UBG units form the PF level, which is under the influence of the RG, and control the activity of groups of motoneurons. Sensory afferents and supraspinal signals have separate access to the RG and individual PF modules. Small perturbations only affect the PF level without modifying the RG level output. It not only keeps the flexibility of the original UBG architecture, with its multiple

oscillators, but also is able to maintain the rhythm at the higher level.

Figure 2.4: Two-level CPG model containing separated networks for rhythm generating and pattern formation (redrawn from McCrea and Rybak [36] [38]).

2.2 Sensory Feedback in Walking

2.2.1 Roles of Sensory Feedback

Although the generation of basic locomotor pattern does not rely on any external input, sensory feedback from muscles and skin afferents as well as supraspinal commands play a crucial role in adapting locomotion to the natural environment, which may be complex and always changing. Peason [40] summarized three roles of sensory feedback in generating adaptive walking. Firstly, the sensory input could reinforce CPG activities, especially in the stance phase, exciting support limb extensor muscles. Secondly, sensory afferents provide signals to ensure motor output is correct for all body parts in terms of position, direction of movement, and force.

Thirdly, sensory inputs entrain the locomotor rhythm to facilitate phase transition to make sure that a certain phase is not triggered until a proper posture of the moving body is achieved. 12

Figure 2.5 shows the dynamic interaction between sensory afferents, neuronal circuits in spinal cord, and muscles during human walking. Various sensory inputs could be categorized into two levels, spinal and supraspinal. At the spinal level, sensory inputs from muscles or skin reach the spinal cord, could project to motoneurons directly, through integrator neurons, or through CPG circuits. At the supraspinal level, visual, auditory, vestibular, somesthesic and proprioceptive afferents reach supraspinal structures, projecting to neurons in the spinal cord. These sensory afferents are highly integrated into the central nervous system and the specific organization of the system is still not fully understood.

Figure 2.5: Dynamic interaction between sensory afferents, spinal neuronal networks, and muscles in human walking (Redrawn from S. Rossignol, R. Dubuc, and J. Gossard [21]).

2.2.2 Feedback of Leg Load and Hip Joint Movement

It is generally accepted that sensory afferents have direct access to the CPG network if stimulation of a given set of afferents leads to entrain and/or reset locomotor rhythm. It can either block or induce the switching between the alternating flexor and extensor locomotor bursts. There are three main sources of afferents satisfying this criteria, the proprioceptive afferents in extensor muscles, cutaneous afferents in foot, and hip joint movement afferents (capsular afferents in joint and muscle afferents around joint) signaling hip joint movement [41][42]. The first two sources both provide load feedback.

The loading condition of extensor muscles has a global effect on the continuation of the stance phase or the onset of the swing phase (reviewed by [43][44]). In decerebrate cats walking on a treadmill, loading the ankle flexor with hip position fixed increased the extensor bursts and diminished the flexor bursts. This suggests that the loading of the ankle extensor exerts an inhibitory effect on the flexor component of the CPG, reinforcing the stance phase, and the unloading of the ankle extensor induces the swing phase [45]. Similar results were obtained in another experiment. The activation of a decerebrate cat’s Ib afferents of fore-limb ankle extensor activates all extensor muscles through the limb and reinforces stance [46].

Hip joint movement feedback also influences animals’ locomotor pattern.

Decerebrate cat’s locomotor pattern could be entrained by applying sinusoidal hip

movements even its leg was extensively denervated [47]. In a similar experiment cat’s hind-leg locomotor pattern was entrained with low amplitude hip rotation. Hip extensor muscle was activated when hip was in flexion, while hip flexor muscles was activated when the hip was in extension [48].

Experiments in humans suggest that sensory feedback of leg loading and hip movement are both crucial factors, which combine to regulate the gait phase transitions. In experiments, a spinal injured patient has one leg stepping on a treadmill with assistance while the other leg was manipulated under different sensory conditions. When a subject’s one leg was manipulated to do air-stepping, and the other leg was minimally moving but was rhythmically loaded with 60% weight- bearing. The EMG activation is minimal in the air-stepping leg, but the rhythmic loading on the other leg induced clear rhythmic EMG bursts in three of four subjects.

This suggests that the stretching of the muscles without loading is not sufficient to induce stepping [49] [50]. In an infants’ treadmill stepping experiment, the stance phase was prolonged and the swing phase delayed when the hip was flexed and the load on the limb was high. In contrast, stance phase was shortened and swing phase advanced when the hip was extended and the load was low. These results were remarkably similar to observations in decerebrate cat experiments [22].

2.3 Human Gait

Normal human walking is composed of repeating cycles, which are referred to as

gait cycles. Figure 2. shows a gait cycle of normal human walking. It begins when the foot contacts the ground and ends when the same foot (heel) contacts the ground again. A gait cycle is composed of stance phase and swing phase. The stance phase is when the foot is on the ground and the swing phase is when the foot is off the ground.

The stance phase includes approximately 60% of the gait cycle and the swing phase takes the remaining 40% of the time. The gait cycles of both legs are slightly out of phase. When one leg contacts ground the other leg is just ending its stance phase. The gait cycle has two brief periods, each lasting approximately 10% of gait cycle, in which both legs are in contact with ground. These periods are called “double support” and in the remaining time the gait cycle is in “single support”.

The stance phase is typically divided into 4 periods (load response, mid-stance, terminal stance, and pre-swing) identified by specific events. Load response immediately follows initial contact and ends when the whole foot flattens on the ground (occurs at approximately 15% of gait cycle). During this period the leg absorbs the impact and becomes loaded. Mid-stance follows load response and ends when the heel leaves ground (heel off, at approximately 40% of gait cycle). During this period the trunk moves forward and over a point above the fixed foot. The terminal stance phase follows heel-off and ends when the other foot contacts the ground (occurs at approximately 50% of gait cycle.) The pre-swing is the final stage of stance phase. It begins with initial contact of the other leg and ends with toe-off

(occurs at approximately 60% of gait cycle).

The swing phase has been divided into early, middle, and terminal periods but it lacks distinct events to separate them. Early swing continues approximately from 60%

(toe off) to 75% of gait cycle. During early swing the leg quickly withdraws from the ground, providing foot clearance, and advances from its trailing position. Mid-swing continues from 75% to 85% of gait cycle. During this period the swing leg passes the stance leg. Terminal swing is the final period of the swing phase. It begins with a vertical tibia and ends when the knee fully extends and the foot strike the ground.

The stance phase and swing phase contribute to human walking in several aspects.

The stance phase provides support preventing falls, absorbs the impact between the leg and the ground, and provides propulsion force for forward progress. The swing phase provides ground clearance and leg displacement for the next stance phase [51].

Figure 2.6: Normal human gait and phase plot.

2.4 Biologically Inspired Bipedal Walking Control

2.4.1 Passive Dynamics

Bipedal walking can be self-stabilizing in the sagittal plane. Various passive dynamic walking models and robots [52] [53] [54] [55] demonstrate that human-like stable walking can be achieved in the sagittal plane with little control effort or energy input. The self-stability of passive walkers in the sagittal plane is a result of energy dissipation in the collision between the leg and ground. In these models when the swing leg reaches the maximal position, it passively goes back due to the gravity.

When a perturbation slightly increases energy in the system, the velocity of the COM increases, leading to an earlier heel strike, which results in a longer step, dissipating more energy. Likewise, when a perturbation slightly decreases the energy of the system, it takes a shorter step, dissipating less energy [56] [57]. Robots based on passive dynamics have the advantage of very high energy-efficiency but they rely much on carefully designed mass distribution of segments and often adopt curved feet

[58] to increase the basin of attraction. They emphasize the importance of the morphology but provide little information on the control mechanisms in the nervous system.

2.4.2 CPG-based Control

Taga’s simulation of linkage planar walking provides the early layout of CPG- based controllers for bipedal walking [59]. It contains multiple neural oscillators

proposed by Matsuoka [60], and each joint (hip, knee, and ankle) is controlled by one oscillator. Knee and ankle oscillators are under the control of the hip oscillator. The left and right hip oscillators are mutually inhibited. The walking movement emerges as a result of the interaction between neural oscillators and the passive dynamics of the pendulum-like body. However, its neural network is quite different from human nervous system and it needs feedback including precise joint angles and angular velocities, which are not biologically available. Some simulation models [61] [62]

[63] improve Taga’s work by using more realistic musculoskeletal structure and applying biological sensory feedback. But in these models sensory feedback only affects motoneurons through reflexes, having no effect on CPGs. This is contradictory to findings that sensory feedback could strongly affect CPG activities. In Luksch’s simulation model [64], the CPG is built as a finite-state machine driven by reflexes, which generates sets of joint torque signals corresponding to different gait phases.

Several physical bipedal robots have CPGs [65] [66] [67], but usually these CPGs are abstract mathematical oscillators tuned to generate desirable trajectories of joint angles tor torques, and the phases of the oscillators could simply be reset by sensory feedback. Klein and Lewis [68] claimed that their robot is the first that fully models human walking in a biologically accurate manner. It has a single-level CPG network entrained by ground contact, hip position, and feedback from muscle afferent

(simulated by motor torque) feedbacks. However, it relies on a baby walker-like frame to prevent it from falling down in the sagittal plane. Its knee of the swing leg remains 19

bent distinctly before the foot touches ground and its heels are always off the ground.

It looks like it is pushing a heavy object, which is fundamentally different from a human’s normal gait.

2.4.3 Reflex Control

A reflex action refers to involuntary and nearly instantaneous movement in response to a stimulus [69]. Some controllers have no pattern generators and only use reflex control, in which sensory feedback directly projects to motoneurons or drives muscle or actuators. Ground contact reflex was already applied in several energy efficient bipedal robots based on passive-dynamic walkers to trigger leg stance motion [13]. Geng, et al. [70] built a small planar bipedal robot with 4 active joints, controlled by a sensor-driven neuronal network. The switching between stance and swing phases is driven by a ground contact reflex. Its relatively fast walking speed owns much to its lightweight structure, curved feet, and joints driven by high-torque motors. Yin et al. [71] presented a 3D simulation of a biped able to walk with various gaits and directions. Its controller is a finite-state machine triggered by multiple reflexes, combined with precise control of swing hip angle as a linear function of center of mass position and velocity. Geyer and Herr [72] developed a 2D muscle- reflex model, which produces realistic human walking and muscle activities. Their model only relies on muscle reflexes without CPGs or any other neural networks. It is realized by two sets (swing phase and stance phase) of coupled equations for every

muscle to produce human walking and muscle activities. However, these equations could be viewed as abstract mathematical expressions of neural networks controlling muscle activities and the transitions between the two gait phases works like a finite- state machine. More recently, CPGs were added to a similar muscle-reflex model to control muscles actuating the hip joint [73]. These works rely on precisely tracking muscle and other biomechanical parameters and provide limited knowledge on how locomotion related neuronal circuits are organized in the human spinal cord.

2.5 Features of This Dissertation

Compared to other related works listed above, this work has several distinct features in both biomechanical aspects and neural control:

The model is a simplified musculoskeletal model, mimicking the human lower body in dimension, mass distribution, and motion ability in the sagittal plane. It is neither a copy of the whole musculoskeletal system of the human lower body, nor a deliberately designed passive dynamic walker. It has a human-like flexible foot instead of a one-piece rigid flat foot or curved foot. Its joint movements are driven by muscle force via Hill’s model instead of direct motor torque.

The model neural network contains some important features found in animal and/or human spinal neural circuits. It is a control network composed of CPGs entrained by sensory feedback, instead of pure reflex control. In this model, the CPG network is neither a finite-state machine nor coupled mathematical oscillators. It is

composed of spiking neurons and organized in a two-level architecture. The controller neither relies on tracking joint angular parameters or COM position, nor requires a model of the body. All of the model’s sensory feedback is known to be available in humans. The sensory input affects muscle activity by modulating the CPG network, in contrast to directly sending control signals to control muscle as most reflex control does.

Chapter 3 Neuromechanical Simulation

3.1 Challenges in Researching Locomotion

The locomotion of human and legged animals is a complex behavior not only highly coupled between the neural system and the musculoskeletal system but also having intensive dynamic interaction with the physical environment. During locomotion each movement changes the status of the musculoskeletal system and relation between the body and the physical world. Receptors are activated by those changes and send feedback to the central nervous system, which responds to these inputs and controls the musculoskeletal system to keep the locomotion stable and continuous. The dynamic interaction between the nervous system, the biomechanics, and the physical environment is essential to the control of locomotion and other behaviors [74].

However, the knowledge of those dynamical interactions is still limited. Elements in the dynamical interactions are highly coupled so it is difficult to investigate how certain elements contribute to the overall dynamics. Due to technical limitations it is difficult to record the activity of the central nervous system in an experimental subject that is intact and can move freely in the real environment. In experimental preparations, the subject could be anesthetized, restrained, and dissected so its muscle is impaired. Sometimes they can only perform “fictive locomotion” under stimulation.

Intracellular recording of motoneuron activity could indicate the pattern of muscle 23

action but it is very limited and could even be misleading. Muscle plays multiple roles in locomotion, serving as motors, brakes, springs, and struts [75], but most of those functions are often impaired or missing in experiments. Additionally, there are even more restrictions on experiments on humans. It is almost impossible to perform experiments which require no descending or peripheral inputs to the experimental subject’s central nervous system on healthy volunteers. So researchers have to rely on specific patients with chronic spinal cord injury, who have relatively isolated spinal cord, to conduct their observations and recordings.

3.2 The Approach: Neuromechanical Simulation

Neuromechanical simulation provides a promising approach to these challenges.

Neuromechanical simulations include the nervous system, sensory receptors and musculoskeletal system, and the real environment. These systems dynamically interact with each other [76][77][78]. Using neuromechanical simulation to study locomotion has several advantages: (1) It is easy to access and tune all parameters of the biological system and the environment to generate, test, analyze hypotheses of locomotion control. (2) It provides for clear and easily visualized interactions among the neural controller, biomechanics, and environment. (3) It allows for the generation of flexible simulation models of neural networks and biomechanics in different levels of details according to the focus of the research. (4) It is efficient both in time and cost. Advances in computing leading to decreasing of computational cost further

facilitates it.

Neuromechanical simulations can facilitate interactions between biology and robotics researchers. In the past, biological and robotic researchers had different interests in studying animal’s locomotion. Biologists focused on the biological control mechanism of locomotion but roboticists strived to develop robots that could walk stably and efficiently like animals. Since 1990s researchers have realized that biology and robotics share some common ground. Robots and animals have sensors and actuators and require control systems to carry out various tasks in a dynamic world

[79]. Biologists use robots to visualize biological data and demonstrate the control mechanism. Roboticists develop control mechanisms inspired from biology. However, there have been miscommunications between biologists and roboticists because they often use different “language” to describe control mechanisms, sensor and motor systems. Neuromechanical simulations bridge the gap by providing a unified

“language” to model those control or biomechanical systems in different levels of details (animal model, simulated robot model, or even an abstract “animat” model), allowing combinations of different controllers with different body models and testing them in different environments (figure 3.1) [80]. Such simulations increase the development and transferability of new hypotheses about control mechanisms in animals and new controllers for walking robots.

Figure 3.1: Neuromechanical simulations facilitating the interaction of robotics and biology. Simulations make it possible to combine different controllers with different body models in different level (animal, animat, robot) and test them in different environments (E1, E2, E3...) (redrawn from T. Buschmann, A. Ewald, A. Twickel, and A. Buschges [80]).

3.3 The Simulation Environment: Animatlab

The neuromechanical simulation environment we chose for this research is

Animatlab [81]. Animatlab is a powerful simulation software tool that allows users to build neural circuits and biomechanical models of animals in a virtual physical environment subject to laws of physics and to record and display time-series of variables while viewing 3D animations of the simulated behavior. It has three interactive components: the GUI that enables model building and data graphing, coupled solvers for the neural circuit and biomechanical simulations, and an interactive 3D animation of the model’s behavior in a virtual world. Compared with other simulators such as NEURON [82], GENESIS [83] for neural simulation and

SIMM, OpenSim [84] for biomechanical simulation Animatlab is the only commercial simulator that combines the “brain” and the “body” together in a physical environment.

In Animatlab a model’s biomechanical and nervous systems are built in separate editors. In its “Body Plan Editor” a simulated animal’s body model could be constructed from simple blocks with different material properties in Lego®-like fashion in a 3D GUI. Different body parts are connected by generalized “joints”.

Those joints are modeled with kinematic constraints including planar hinge, ball and socket, one-dimensional sliding, and fixed. Animatlab also provides actuators including muscles, motors and springs to drive body segments. Muscle is represented by Hill’s muscle model, consisting of a spring in series with the parallel combination of a spring, a viscous damper, and a contract force generator. Users can add a muscle to the body by adding attachment points to body segments on either side of a joint, and then spanning the muscle between these points. Neural control of muscle tension is mediated by the motoneuron, which depolarizes the muscle. A stimulus-tension function relates the muscle membrane potential to the tension force. It is further scaled by the length-tension curve and applied to the muscle model to cause contraction.

In the “Behavior Editor” the neural circuit model, composed of single or multi- compartment neurons and neural circuits, is constructed by dragging elements from a toolbox onto a page and linking them by electrical or chemical synapses. Those elements include integrate-and-fire neurons [85] [86], non-spiking neurons and firing rate neurons. Their parameters can be displayed and changed in the “Property Table”.

Neurons are connected by conductance-based synapses, which are presented by cursor-drawn lines.

Animatlab also provides biological sensory receptors for touch, muscle stretch, and odor. Those sensory receptors could be created in both the body model and the neural circuit model to map the field of sensory receptors onto the population of sensory neurons. In the body model, single or an array of receptors can be placed on the body model surface where stimulus would activate it. In the neural circuit model the corresponding representation of a body receptor is linked through a transduction adapter to a neuron compartment, which represents the sensory neuron excited by the receptor. The current that goes into the sensory neuron is scaled from the intensity of the stimulus. Receptors for joint rotation, extension of muscle and other physical changes all use similar mechanism to produce the current that is applied on the sensory neuron.

Many neuromechanical simulations of human and animal behaviors have been implemented in Animatlab, including human arm muscle stretch reflex [81]; the kicking and jumping of the locust [87]; the walking of crayfish [88]; the walking and turning of the cockroach [89]; and leg-coordination of rat in locomotion [90].

Chapter 4 Biomechanical model

4.1 General Structure

I based the biomechanical structure of the model on the human lower body, which has been shaped and selected by millions of years’ evolution, and is energetically close to optimal [91]. The planar skeleton is made up of rigid bodies, and is actuated by antagonistic pairs of Hill’s muscles. The morphology employed significantly reduces the control effort and provides a basis for comparison to real human gait [56]

[92]. The simulated biped is modeled as a pelvis supported by two legs in Animatlab

(figure 4.1). Each leg has 3 segments (thigh, shank and foot), which are connected by

3 hinge joints (hip, knee and ankle) and actuated by 6 muscles. The foot is composed of two parts, connect by one hinge joint and a passive spring with damping. All those joints have one degree of freedom (DOF) only rotating in the sagittal plane. So the model has a total of 8 DOFs in the sagittal plane. The mass and length ratios of segments follow human segments data [93]. The leg length of the robot is approximately 1 m. The 2D musculoskeletal model is a simplification of the human lower body containing more complex and redundant muscles in the sagittal plane. The purpose was to build a functional non-redundant biomechanical model, and then develop the neural controller to verify whether stable walking could be achieved by our control strategy.

Figure 4.1: The biomechanical model.

Table 4.1: Mass and dimension of leg part

Leg Part Mass (kg) Length (m)

thigh 8.8 0.41

shank 2.8 0.43

foot 0.6 0.20

toe 0.08 0.04

4.2 Foot Design

Figure 4.2 shows that the human foot is flexible during walking to facilitate the body load roll over the foot from heel to toes. During the terminal stance phase the metatarsophalangeal (MP) joints are in hyperextension while the ankle is in plantarflexion. The plantar aponeurosis (PA), which spans the arches of the foot, bears the load of the body moving over the stance foot toward the other foot. As shown in figure 4.3, the foot of the model is composed of two parts connected by one hinge joint, spanned by a passive spring with damping (spring constant = 16 kN/m, damping 30

coefficient = 20 N·m/s), which models the PA in the human foot. The dimension of the foot is about 240 mm × 120 mm × 40 mm. The foot model has a compliance of 10

µm/N to mimic the soft tissue in the human foot. Two contact sensors are attached to the bottom of the toe and heel.

Figure 4.2: Human foot during stance phase.

Figure 4.3: The structure of the two-part foot.

4.3 Muscle Model

The muscle force is active and passive components. Muscle’s contractile elements provide the active force through muscle fibers. Muscle’s non-contractile elements

contribute to its passive force. When muscle is stretched beyond a certain length in the absence of a contraction, the muscle exerts a resistance force against the stretch.

Passive structures in the muscle producing resistance force, such as investing connective tissue, are modeled as parallel elastic components. Tendons at either end of the muscle also provide force against the stretch and these components are modeled as series elastic elements [94]. Hill found that the faster a muscle shorten; the less total force it generates. So in Hill’s muscle model, he added a viscous component parallel with the contractile component [95].

Hill’s muscle model is illustrated in Figure 4.4. The activation of the muscle engages the contractile component and produces active force, which is scaled by muscle, interacts with a viscosity and two elastic elements to generate muscle force.

Figure 4.4: Hill’s muscle model.

The muscle tension force T is:

dT k   k pe    se k x  bx 1 T  A  pe    (Eq. 4.1) dt b   kse  

where A is the active force, kse is the serial spring constant, k pe is the parallel spring constant, b is the damping coefficient, and x is the muscle length.

In Animatlab, Hill’s muscle model properties are determined by the resting muscle length, serial and parallel spring constants, damping coefficient, the stimulus-tension curve, and the length-tension curve. In figure 4.5 the muscle membrane potential is depolarized by inputs from one or more motoneurons and generate the contractile tension force through a sigmoid function. The contractile tension force is further scaled by the current muscle length and implemented in Hill’s muscle model as the active force A . The output force of Hill’s muscle model drives the movement of joint. The muscle generates less active force when it is stretched or in contraction from its resting length.

Figure 4.6 shows that the model’s hip extensor muscle generates 800 N contractile tension force when its muscle membrane potential is completely depolarized. It produces the maximal active force at its resting length of 21 cm and remains approximately 80 % of the maximal value when it is stretched to 40 cm. Table 2 presents passive parameters, resting length, and maximal contractile tension force of 6 muscles used in this model.

Figure 4.5: Modeling muscle in Animatlab. 33

Figure 4.6: The stimulus-tension curve (left) and the length-tension curve (right) of the hip extensor muscle.

Table 4.2: Muscle parameters. EXT: extensor muscle, Flex: flexor muscle.

Max Resting k Contractile Muscle pe kse Length b ( N·s/m ) Tension ( mm ) ( kN/m ) ( kN/m ) (N)

Hip EXT 4 6.2 800 210 800

Hip FLX 4 6.3 800 170 800

Knee EXT 4.6 7.7 600 340 800

Knee FLX 3.8 6.3 600 345 500

Ankle EXT 4.5 7.5 400 240 800

Ankle FLX 4 3.2 400 250 500

Chapter 5 Neural Control Network

5.1 Neural Models

Neurons are elementary processing units in the central nervous system. They connect to each other and form an intricate network. A synapse is a junction between two neurons. If a neuron sends a signal to another neuron cross a synapse it is common to refer to the neuron sending the signal as the presynaptic neuron and the neuron receiving the signal as the postsynaptic neuron. The input produces electrical transmembrane current and changes the membrane potential of the postsynaptic neuron, which is called the postsynaptic potential. If the synaptic current exceeds the threshold value it could produce abrupt and transient changes of membrane potential, which are called spikes. Spikes propagate in the neural network and are the main means of communication between neurons. Before building a neural network the first thing is to model the neurons and synapses.

5.1.1 Leaky Integrate-and-Fire Neuron Model

The leaky integrate-and-fire (IF) neuron model is one of the simplest but often used models for simulating and analyzing the behavior of neuron systems. It describes the membrane potential of a neuron as a “leaky integrator” of currents it receives, including synaptic input and injected current:

d vt C  I t I t I t (Eq. 5.1) m dt leak syn inj

where vt is the membrane potential, Cm is the membrane capacitance, Ileak t is

negative current due to the passive leak of the membrane, I syn t is the synaptic

current from other neurons, Iinj t is the current externally injected into the neuron.

The leak current is

Cm Ileak (t)   Erest  vt (Eq. 5.2)  m where  is the time constant; E is the resting potential. m rest

When the neuron membrane potential vt reaches the threshold value Eth the

neuron is said to fire a spike, and is reset to reset potential Er . Eq. 5.2 describes the membrane potential below the threshold value [95]. To generate spiking the total external current must exceed the threshold value:

Cm Ith  Eth  Er  (Eq. 5.3)  m

Otherwise its membrane potential increases but with no spiking.

The leaky IF neuron can be simulated in Animatlab (figure 5.1). Because the spike duration is too short in the original IF neuron model, Animatlab also provides afterspike hyperpolarising potential (AHP) parameters (figure 5.1) to increase the delay between spikes.

Figure 5.1: Simulation of a leaky integrate-and-fire neuron in Animatlab. (a) Neuron

property. Erest = -60 mV, Eth = -55 mV,  m = 5 ms. AHP parameters are related to the delay time between spikes. (b) Neuron spiking. The

simulated neuron has Ith =5 nA, so it spikes when it is injected 10 nA current from 100 ms to 150 ms but keeps silent when it is injected 4 nA current from 20 ms to 70 ms.

The leaky IF model is one-dimensional so it does miss some important features of biological neurons such as phasic spiking, bursting, and other nonlinear dynamics.

This research mainly focuses on the structure of the neural network and the interaction between the neural network and the musculoskeletal system, instead of the spiking behavior of a single neuron. The biologically based IF model is enough to meet the current need with less free parameters. It is reasonable to replace it by other more advanced neuron models after the fundamental architecture work of the neural network is done. The one dimensional IF model is also expandable to higher dimensions by adding potassium current, calcium current and other ionic current terms on the right side of Eq.5.1.

5.1.2 Synapse Model

In computational neuron modeling transmitter-activated ion channels could be

modeled as time-dependent postsynaptic conductance gsyn t. The current that passes

channels I syn t depends on the reversal potential Esyn of the synapse and the postsynaptic neuron membrane potential vt

I syn t  gsyn tEsyn  vt (Eq. 5.4)

Esyn and gsyn t can describe different types of synapse. Typically the reversal

potential of inhibitory synapse Einh equals the reversal potential of potassium ions and the reversal potential of excitatory synapses Eexc equals about 0 mV and usually

Einh  Erest  Eth  Eexc (Eq. 5.5)

In a spiking chemical synapse model the postsynaptic conductance suddenly increases to a value with a time delay to the arrival of the presynaptic spike and then decays exponentially to zero:

 t t0 tdelay  g t  g exp ,t  t t (Eq. 5.6) syn   max   0 delay   syn 

where gmax is the initial amplitude of the postsynaptic conductance increase; t0

denotes the arrival time of a presynaptic spike; tdelay is the time delay between the

presynaptic spike and the postsynaptic response;  syn is the time constant of the decay rate.

5.1.3 Neural Facilitation Model

Some chemical synapses show an increase in synaptic efficacy or a growth in the amplitude of individual postsynaptic potentials or postsynaptic currents on repetitive activation. This enhancement of transmission is referred to as “facilitation”. Similarly, at some other chemical synapses repeated activation leads to a decrease in synaptic strength and this declining of transmission is referred to as “depression”. Both effects are forms of short-term synaptic plasticity [97].

In a simple analytic model (figure 5.2), if two presynaptic spikes occur, the

facilitated (or depressed) postsynaptic conductance g f can be described according to:

 t  g t  g  g  k 1 exp  (Eq. 5.7) f   unf unf  f      f 

t is the time between the two presynaptic spikes;  f is a time constant; g f t is

the facilitated postsynaptic conductance, gunf is the unfacilitated postsynaptic conductance, k is the relative facilitation factor. equals 1 means no f facilitation or depression. k f 1 means facilitation while k f 1 means depression.

Both of facilitation and depression effects exponentially decay with time. In this model, if there is a serial of presynaptic spikes, the facilitation or depression effects from several presynaptic spikes add to each postsynaptic spikes.

Figure 5.2: Neural facilitation and depression models. The postsynaptic conductance increase at time B is affected by the facilitation or depression initiated by previous postsynaptic conductance increase at time A.

5.2 Half-center Oscillator

Figure 5.3 presents a half-center neural oscillator composed of two reciprocally inhibited neurons in Animatlab. The neuron adopts the integrate-and-fire model and the synapse is modeled as time dependent postsynaptic conductance with depression.

Both neurons receive stimulus tonic current just beyond the threshold value to make them spike. Because the reversal potential of the inhibitory synapse is lower than the resting potential of the neuron model, this induces inhibitory postsynaptic current

(IPSC). The postsynaptic neuron stops spiking if the total current input it received drops below the threshold value. Due to the depression effect the repeated spiking of the presynaptic neuron gradually hampers the synaptic transmission and reduces the postsynaptic conductance so the amplitude of the IPSC declines gradually (figure

5.3(d)). The postsynaptic neuron spikes again as soon as the total current input recovers to the threshold value. And when it starts to spike it would inhibit the other 40

neuron in the same way. Potential noise is added to both neurons to unbalance them at the starting point and trigger the oscillation. The short-term synaptic plasticity

(depression) makes the two neurons alternatively spike and rest and serve as a pattern generator.

Figure 5.3: Simulation a half-center oscillator in Animatlab. (a) a half-center oscillator

in Animatlab’s “behavior editor”. (b) Neuron properties: Erest = -60 mV,

Eth = -55 mV,  m =5 ms. Both neurons are excited by 6 nA injected tonic current. 0.5 mV noise is added to unbalance them from the starting point.

(c) Synapse properties: Einh =-70 mV, g max = 1 µS,  syn = 10 ms, k f =0.6. (d) Membrane potentials of the two neurons, synaptic current and neural 41

depression of neuron 2. When neuron 1 starts to spike neuron 2 receives inhibitory postsynaptic current (IPSC) and its membrane potential drops below the threshold value. The IPSC declines due to the accumulated depression effects. When the IPSC diminishes, neuron 1 starts to spike and inhibit the spiking of neuron 1 in the same way.

The output of the half-center oscillator could be entrained by IPSC input. In figure

5.4 (a) the half-center oscillator has inhibitory synaptic connection with two integrator neurons. In figure 5.4 (b) when integrator neuron 1 is activated by injected tonic current it generates 15 nA IPSC, making the membrane potential of neuron 1 drop quickly and stop spiking. As soon as neuron 1 is inhibited by integrator neuron 1, neuron 2 receives no IPSC from neuron 1 and starts to spike. There is no depression effect in the synapses between integrator neurons and half-center neurons so neuron 1 stays inhibited and neuron 2 keeps spiking as long as integrator neuron 1 is activated.

Neuron 2 could be inhibited in the same way. If both integrator neurons are activated, neuron 1 and neuron 2 are inhibited simultaneously and the oscillator stays silent.

Figure 5.4: The output of a pattern generator entrained by IPSP. (a) Half-center oscillator having inhibitory synaptic connections with integrator neurons. The integrator neuron adopts the same neuron model and neural properties as half-center neurons. For inhibitory synaptic connections from integrator

neurons, Einh =-70 mV, gmax = 1 uS,  syn = 5 ms, k f =1. (b) The membrane potentials of neurons. When integrator neuron 1 is activated, neuron 1 is inhibited and neuron 2 spikes. Similarly, when integrator neuron 2 is activated neurons 2 is inhibited and neuron 1 spikes. When both integrator neurons are activated, both half-centers are inhibited.

The output of the half-center oscillator could also be entrained by EPSC input. In figure 5.5(a) the half-center oscillator has excitatory synaptic connection with two integrator neurons. In figure 5.5(b) when integrator neuron 1 is activated by injected tonic current it generates 10 nA EPSC, making the membrane potential of neuron 1 quickly recover to the threshold value, and spikes as long as integrator neuron 1 is 43

activated. Neuron 2 could be activated in the same way.

Figure 5.5: The output of the pattern generator entrained by EPSP. (a) Half-center oscillator having excitatory synaptic connections with integrator neurons. The integrator neuron adopts the same neuron model and neural properties

as half-center neurons. For excitatory synaptic connections, Eexc =-50 mV,

gmax = 1 µS,  syn = 5 ms, k f =1. (b) The membrane potentials of neurons. When integrator neuron 1 is activated, neuron 1spikes and neuron 2 is inhibited. Similarly, when integrator neuron 2 is activated neurons 2 spikes and neuron 1 is inhibited.

5.3 CPG Network Without Sensory Feedback

Figure 5.6 presents a CPG neural network composed of half-center oscillators. It adopts a two-level architecture based on Rybak and McCrea’s two-level UBG model

[36] [38], containing rhythm generator (RG) and pattern formation (PF) networks.

The RG level has a single half-center oscillator, which generates the basic periodic stance signal for both legs. The left stance (LS) neuron and right stance (RS) neuron correspond to left leg stance and right leg stance. The PF network contains unit pattern formation (UPF) modules. Each joint of each leg (hip, knee, and ankle) has one UPF module. The UPF adopts the same half-center oscillators as the RG. Each neuron in the UPF module is connected to corresponding extensor and flexor motoneurons to drive extensor and flexor muscles of each joint. Hip extensor (Hip

EXT) neuron, knee extensor (Knee EXT) neuron, and ankle plantar flexor (Ankle PF) neuron correspond to hip extension, knee extension, and ankle plantarflexion movements. Hip flexor (Hip FLX) neuron, knee flexor (Knee FLX) neuron, and ankle dorsiflexor (Ankle DF) neuron correspond to hip flexion, knee flexion, and ankle dorsiflexion movements.

The RS neuron has excitatory synaptic connections with right Hip EXT neuron and left Hip FLX neuron. Similarly, the LS neuron has excitatory synaptic connection with left Hip EXT neuron and right Hip FLX neuron. When the RS neuron spikes, right Hip EXT and left Hip FLX neurons spike, which encourages the model’s right leg in stance phase and left leg in swing phase. Alternatively, when the LS neuron spikes, the left leg is in stance phase and right leg is in swing phase. As a result of reciprocal inhibition, the RS and LS neurons spike alternatively, which makes the model’s right leg and left leg always move in antiphase. In the initial design it is

supposed that all joints are in extension when the leg is in stance phase and all joints are in flexion when the leg is in swing phase. So Hip EXT neuron has an excitatory synaptic connection with the Knee EXT and Ankle PF neurons; also, Hip FLX neuron has excitatory synaptic connection with the Knee FLX and Ankle DF neurons.

Figure 5.7 presents the output of the CPG neural network. In the RG level the RS and LS neurons spike alternatively. In the PF level, Hip EXT, Knee EXT, and Ankle

PF neurons spike synchronously with the half-center neuron of the RG on the same side. The outputs of left leg and right leg UPF modules are always antiphase.

However, due to the noise added to neurons, the output of the RG is not stable. The duration of RS and LS neurons spiking varies, which makes the durations of stance and swing phases vary.

Figure 5.6: The architecture of a two-level CPG network. In the PF level each joint contains one half-center oscillator referred to as unit pattern formation (UPF) module. The output of the UPF module is the extension or flexion signals for each joint. Hip UPF module has excitatory synaptic connections with knee and ankle UPF modules in the same leg. UPF modules in one leg have no synaptic connection with UPF modules in the other leg. In the higher level a rhythm generator (RG) produces periodic “left leg stance” or “right leg stance” signals, which are transmitted to both legs by excitatory synaptic connection to corresponding hip UPF neurons.

All neurons has the same neuron properties, Erest = -60 mV, Erest = -55

mV,  m = 5 ms. For inhibitory synapse Einh =-70 mV, gmax = 1 µS,  syn =

10 ms, k f = 0.8. For excitatory synapse Eexc = -50 mV, gmax = 1 uS,  syn

= 3 ms, k f =1.0. All neurons have 5 nA injected tonic current and 0.5 mV noise.

Figure 5.7: Membrane potentials of neurons in CPG network.

Figure 5.8 shows the model “walking in the air”, controlled by the CPG neural network. The pelvis of the model is fixed in space. Hip, knee, and ankle joints rotating ranges are preset to -15° to 27°, 90° to 0°, and -15° to 0°, respectively. Simulation shows that the two leg move as designed. When one leg is in the stance phase the other leg is in the swing phase. All joints are extending when the leg is in stance; and all joints are flexing when the leg is in swing. Comparing figure 5.8 with figure 2.6, the model’s “walking” has some similarity with human gait, especially in the stance phase but there are differences. The most noticeable issue is that the knee begins to extend in mid-swing when the hip joint is still in flexion in normal human gait. The knee is almost straight, preparing for loading, when the hip joint is in maximal flexion at the end of the swing phase. But the model’s knee is in flexion during the total swing phase, which makes it difficult for the leg to contact ground and take load at the beginning of the stance phase. When this model was released on the ground it fell quickly. So, this initial CPG neural network is able to produce rhythmic and coordinated patterns for joints but is not sufficient to control stable human-like walking.

Figure 5.8: Frames of the model “air stepping” without sensory feedback.

5.4 Neural Network with Sensory Feedback

5.4.1 Feedback of Ground Contact and Hip movement

Locomotion is the result of dynamic interactions between the central nervous system, body biomechanics, and the external environment. Sensory feedback provides information about the status of the biomechanics and the relationship between the body and the external environment to make locomotion adaptive to the environment.

Sensory feedback can access the neural network and strongly affect CPG activities.

So, in this model sensory feedback is added, including ground contact feedback and hip movement sensory feedback. It entrains the CPG neural network by suppressing corresponding RG and PF neurons. Figure 5.9 shows how sensors are connected to corresponding sensory neurons through adapters, which transfer sensory input to current output. Sensory neurons have excitatory synaptic connections with one or several integrator neurons, which have inhibitory synaptic connections with corresponding RG or PF neurons.

1) Ground contact afferent

The support leg loading in walking mainly comes from the contact between foot and ground. In this model heel ground contact (Heel GC) sensor and toe ground contact (Toe GC) sensor are separately attached to the bottom surface of the heel and toe of each foot to detect ground contact, which indicates the loading status in each leg. When they contact ground in the simulation their output is 1, otherwise it is 0.

The adapter between ground contact sensor and sensory neuron is a sigmoid function, which outputs 10 nA excitatory current when ground contacts happens (figure 5.9).

Heel GC sensory neuron and Toe GC sensory neuron spike when corresponding sensors detect ground contact (figure 5.10).

The heel ground contact is designed to initiate the stance phase and keep the knee straight. The sensory input of toe ground contact is designed to trigger the ankle plantarflexion in the terminal stance period. So, the Heel GC sensory neuron has excitatory synaptic connections with integrator neurons connected to the Knee FLX and Stance (of the other leg) neurons in the CPG network. It also has an excitatory synaptic connection with the integrator neuron connected to the Ankle Plantarflexion neuron to prevent ankle plantar flexing too early in the load response period. Toe GC sensory neuron has excitatory synaptic connection with the integrator neuron connected to the Ankle DF neuron in the CPG network.

2) Hip movement afferent

During human normal walking the knee extends before the foot touches the ground at the initial contact. The knee reaches its peak flexion between 25% and 40% of the swing phase and then it extends [98]. The hip movement (Hip Mov) afferent is designed that the knee extension begins when the hip angle passes a predetermined angle value (15°) during the swing phase. The adapter between the hip angle sensor and Hip Mov sensory neuron is a sigmoid function, which outputs 10 nA current

when the hip angle passes 15° (figure 5.9). Hip Mov sensory neuron spikes when sensor detects that the hip angle is greater than 15° and stops spiking when the hip angle is below 15° (figure 5.10). The Hip Mov sensory neuron has an excitatory connection with the integrator neuron connected with the Knee FLX neuron in the

CPG network.

Figure 5.9: Connecting sensors to the neural control network.

Figure 5.10: Outputs of sensors and membrane potentials of sensory neurons. Heel GC or Toe GC sensory neuron spikes when heel or toe contacts ground respectively. Hip Mov sensory neuron spikes when the hip angle is over 15°.

5.4.2 Architecture of the Neural Network

Figure 5.11 shows the final architecture of the CPG network combined with sensory feedback. Black lines are synaptic connections between CPG neurons; Blue lines are synaptic connections are synaptic connections between sensory neurons and

CPG neurons. Compared with the initially designed network shown in figure 5.6, not only sensory afferents are added but alsoand some synaptic connections between with

CPG neurons are adjusted. The excitatory synaptic connection between the Hip EXT neuron and Ankle PF neuron is deleted. The excitatory synaptic connection between

Hip FLX neuron and Knee FLX neuron is changed to an excitatory synaptic connection between the Hip FLX neuron and the inhibitory integrator neuron connected to the Knee EXT neuron. Synaptic weights of connection between CPG neurons and sensory afferents are listed in Appendix C.

Figure

The

connections from sensory neurons to CPG neurons neuronsconnections to CPG from sensory

architecture of the ne

ural

network. Blue lines are Blue connections synaptic network. neurons; between CPG synaptic black lines are

5.5 Connecting Neural Network to Muscles

In human nerves, impulses generated by the central nervous system cause muscles to contract. Somatic motoneurons have cell bodies located in the spinal cord which project axons to skeletal muscles. They carry signals from the central nervous system and control activities of muscle fibers. When Motoneurons spike they release acetylcholine (ACh) at the neuromuscular junctions and depolarize muscle fibers. The flood of depolarization travels through the tissue and triggers the contraction of sarcomeres. A motor neuron usually controls several adjacent muscle fibers and forms a motor unit [99].

In figure 5.12 each muscle is driven by a corresponding motoneuron, which also adopts the leaky IF model. The muscle membrane potential is modeled as a non- spiking neuron, which adopts the leaky IF model but does not “spike” when its potential rises to the threshold potential, which is higher than in the spiking neuron.

The excitatory nicotinic ACh synapse between the motoneuron and the muscle potential neuron adopts the similar spiking synapse model with higher reversal potential compared with the excitatory synapse between sensory neurons and integrator neurons. The muscle membrane potential is converted to contractile tension force by the stimulus-tension curve. The contractile tension force is further scaled by the length-tension curve and applied to Hill’s muscle model as the active force.

In human motor control, antagonist motoneurons are inhibited when agonist

muscle contracts during joint movement [100], which is referred to as reciprocal inhibition. The neural control network also has another important feature. In one UPF module a PF neuron not only has excitatory nicotinic ACh synaptic connection with the corresponding agonist motoneuron but also has an inhibitory synaptic connection with the antagonist motoneuron. Due to mutual inhibition when one PF neuron spikes, the other one is inhibited. The spiking PF neuron will trigger the agonist motoneuron and inhibit the other one. The UPF module controlling joint movement will encourage one muscle to contract while the other muscle is relaxing causing reciprocal inhibition.

Figure 5.12: Connecting Hip PF neurons to hip extensor and flexor muscles.

Motoneuron: Erest =-60 mV, Eth =-59 mV,  m = 5 ms; muscle membrane

potential neuron: Erest =-100 mV, Eth =50 mV,  m = 5 ms; nicotinic ACh

synapse: Eexc = -10 mV, gmax = 1 uS,  syn = 3 ms, k f =1.5. The adapter is a linear function passing origin with slope equaling 1. Knee UPF module and Ankle UPF module control corresponding muscles with similar architecture.

Fig 5.13 shows that the motoneurons drive muscle membrane potential under the control of PF neurons. The muscle membrane potential, which is mimicked by a non-

spiking neuron with high threshold potential, does not spike as do the PF neuron and motoneuron. The muscle tension applied on the skeleton is not simply determined by the muscle membrane potential but also affected by its passive properties and changing length. So, the muscle tension is not constant during joint movement.

Mutually inhibited PF neurons reciprocally inhibit hip extensor and flexor muscles.

Figure 5.13: Neuron and muscle membrane potentials, muscle tension force and muscle lengths.

Chapter 6 Simulation of Normal Walking

6.1 Generation of Gait Cycle

In the initial experiments, the model walks on a horizontal surface. It is constrained to slide without friction along two vertical walls in the sagittal plane (figure 6.1). The model is a 2D walker in the sagittal plane without any lateral movement. The model can fall down forward or backward freely without any friction between it and the walls. The model’s joint angle ranges for hip, knee, and ankle are preset to -10° to

27°, 0° to 90°, and -15° to 0°, respectively. The center of mass of the pelvis is 30 mm forward of its geometric center, which is directly over the middle of the feet.

Figure 6.1: Model constrained in the sagittal plane by two vertical walls.

Rather than setting initial conditions for an experiment to those typical in a walking gait, it is dropped in a neutral standing posture. When the simulation starts, the model is released (figure 6.2) approximately 10 cm above the ground. During the short time of the fall, random joints movement cause one leg to touch the ground first.

The neural network causes that leg to enter the stance phase instantaneous and initiating walking. If the initial status of the model has both heels contacting the ground, it typically will not walk properly and falls down. This configuration does not happen in a normal walking gait. Also, when the model drops 10 cm initially, a

20 N horizontal force is applied on the pelvis for 1 second to push it forward, helping the model initiate forward walking.

Figure 6.2: Releasing the model at the beginning of walking.

After the release, the model walks under the control of the neural network with a moderate speed (0.9 m/s). Figure 6.3 shows a frame by frame comparison with that of a human. It is interesting to observe that they appear similar at corresponding phases of the gait cycle.

1) Load Response

In frame A in figure 6.3, when the right leg enters the stance phase the right Heel

GC sensor causes the Heel GC sensory neuron to spike (time A in figure 6.4(a)). This inhibits the left leg stance RG neuron and leads to the right leg stance RG neuron spiking, causing right leg Hip EXT neuron and Knee EXT neuron to spike and initiate right leg stance (time A in figure 6.4(b)). The spiking of the Heel GC sensory neuron also inhibits the Ankle PF neuron and encourages ankle joint dorsiflexion.

2) Mid-stance

In frame B, the right heel is still on the ground so the right hip joint is extending and the knee is straight. The toe part of the right foot contacts the ground so the right

Toe GC sensor triggers the Toe GC sensory neuron to spike (time B in figure 6.4(a).

This inhibits the Ankle DF neuron (time B in figure 6.4(b)). Because the Ankle PF neuron is still inhibited by the Heel GC sensory neuron, the ankle joint keeps at maximal dorsiflexion (ankle joint is 0°).

3) Terminal stance

In frame C the right heel just leaves ground so the right Heel GC sensory neuron stops spiking (time C in figure 6.4(a)). The right Ankle PF neuron is released from inhibition and begins to spike, causing the right ankle plantarflexion (time C in figure

6.4(b)). From time C to time D, neither heels are contacting the ground and both Heel

GC sensory neurons are quiet. The right leg stance RG neuron is not inhibited and keeps spiking, so the right leg Hip EXT neuron and Knee EXT neuron are still spiking, keeping right hip and knee in extension. 62

4) Pre-swing

In frame D the heel of the left foot just contacts the ground. Most part of the right foot are departing the ground except the toe part. The right foot is in flexion with the spring in compression. At this point, both the right toe and left heel are in contact with the ground so the model is in double support phase. The left Heel GC sensory neuron begins to spike (time D in figure 6.4 (a)) and inhibit the right leg stance RG neuron.

At time D in figure 6.4(b) right Hip FLX neuron and Knee FLX neuron are released from inhibition and begin to spike, encouraging right hip and knee to flex. The left leg enters stance phase and begins to take load from the right leg.

5) Early swing

In frame E the toe part of the right foot is just leaving the ground and the double support phase ends. The right knee joint is in apparent flexion before the right foot completely leaves the ground. The right Toe GC sensory neuron stops spiking (time E in figure 6.4(a)), releasing the right Ankle DF neuron from inhibition (time E in figure

6.4(b)). From time E to time F the right hip and right knee further flex and the right ankle goes into dorsiflexion.

6) Mid-swing

In frame F the right leg just passes the preset position. The right Hip Mov sensory neuron spikes (time F in figure 6.4(a)), inhibiting the right Knee FLX neuron from

spiking (time F in figure 6.4(b)). This causes the flexing knee to begin to extend passively due to momentum.

7) Terminal swing

In frame G towards the end of the right leg’s swing phase the right hip is in maximal flexion and the right ankle is in dorsiflexion. Both Knee EXT neuron and

Knee FLX neuron are quiet (time E in figure 6.4(b)) right knee is extended and prepared for ground contact.

Figure 6.3: Comparison of the model and human walking at the same points in a gait cycle.

Figure 6.4: Neuron membrane potentials in the walking cycle. (a) sensory neuron membrane potentials (b) Right leg CPG neuron membrane potentials.

Figure 6.5 shows a phase plot of the model. The duration of one gait cycle is approximately 1.2 to 1.3 sec. The stance phase takes approximately 60% of the gait cycle and swing phase takes the remaining 40%, which is similar to human walking at this speed. The double support period takes approximately 15% of the gait cycle, a little shorter than human’s 10%. The double support period coincides mostly with the toe support period of the leg in stance phase.

Figure 6.5: Phase plot of the model.

In figure 6.6, frames of the model’s gait are compared with Klein and Lewis’ robot

[68]. Although they also adopt neural networks composed of CPGs entrained by sensory feedbacks, they don’t use CPGs in a PF layer and the gait of our model is appears more humanlike. In Klein and Lewis’ robot, the knee of the swing leg remains bent during terminal swing phase. It also inclines overly forward and might fall down without the support of the baby walker-like frame. This may be an unfair comparison considering we are comparing a numerical model with a robot.

Figure 6.6: Comparison of the gait simulation with Klein and Lewis’ physical model [68].

6.2 Kinematic Analysis

Kinematic analysis of gait quantitatively describes the position and orientation of body segments, angles of joints, and corresponding velocity and acceleration. Here the kinematic data of the sagittal motion of the model is recorded and compared with human data.

6.2.1 Joint Angle

Figure 6.7 shows how hip, knee, and ankle motions are defined in this work and how these joint angles are measured in the sagittal plane. Positive values of joint angles correspond to hip in flexion, knee in flexion, and ankle in dorsiflexion, while negative values of joint angles correspond to hip in extension, knee in extension, and ankle in plantarflexion. The foot of the model also has a joint connecting the major foot part with the toe part. This joint is similar to the metatarsophalangeal (MP) joint in human.

Figure 6.7: Definition of Joint angles and movements in the sagittal plane.

Figure 6.8 presents the model’s angular displacement of three major active joints in one gait cycle compared with a human’s. The model’s hip joint angle trajectory shows that at the beginning of the gait cycle the hip is in maximum flexion and gradually extends to its maximum extension at approximate 50% of the gait cycle. It then begins to flex, reaching maximum flexion again at approximately 85% of gait cycle, and keeps maximum flexion at rest time of the cycle. The limits of maximum flexion and extension are present to 32° and -13° of joint motion. The shape, motion range, and position of inflection point are similar to a human’s. The major difference is that the model’s hip joint flexion takes a larger percent of the gait cycle (the hip joint angle less than 0°).

The model’s knee angle joint trajectory is close to full extension at initial foot contact, flexes approximately 10°, and then extends before the foot lays flat on the 69

ground. Between 15% to 60% of the gait cycle, the majority of the stance phase, the knee is in full extension supporting load. At 60% of the gait cycle the knee begins to flex and reaches maximum flexion, close to 60°, at approximately 75% of the gait cycle during mid-swing. There is no preset angular limit of the knee flexion. The knee extends again and reaches full extension before the end of the gait cycle. It shows a similar double-hump shape and maximum flexion angle compared with a human’s.

The major differences are the human’s knee angle trajectory has a larger flexion

(approximately 10° to 20°) at load response and flexes earlier at the beginning of the pre-swing period (50% of gait cycle).

The motion range of the model’s ankle joint is preset to 0° to 15°. The model’s ankle joint trajectory shows that the ankle joint remains at 0° from the beginning to approximately 55% of the gait cycle and then extends into plantarflexion to the maximum of -15°. It begins to flex at approximately 70% of the gait cycle, reaches 0° at 75% of the gait cycle, and remains at 0° for the remainder of the gait cycle.

Dorsiflexion from initial contact to terminal stance is less as compared with human’s ankle joint trajectory but it has similar strong plantarflexion after the beginning of the pre-swing period (55% of gait cycle).

In figure 6.8, angular displacement of the model’s toe joint is also compared with a human’s Metatarsophalangeal (MP) joint in one gait cycle. The model’s toe joint begins to flex at 40% of the gait cycle, reaching approximately 25°, and then recovers

to the neutral position at 70% of gait cycle. It has similar strong flexion from the terminal stance to pre-swing compared with a human’s MP joint trajectory.

Figure 6.8: Hip, knee, ankle, toe joint angles in a gait cycle compared with human’s [101][102].

Figure 6.9 plots the hip vs. knee angle trajectories for 20 gait cycles. Trajectories of continuous gait cycles repeat closely. They appear similar in the swing phase, from pre-swing to mid-swing (3 to 4) and mid-swing to terminal swing (4 to 1) as compared with a human’s hip vs. knee joint motion plot. The major difference is in the stance phase, from 1 to 2, and 2 to 3. The model’s knee bends less during load phase so its knee joint is mostly straight from mid-stance to terminal stance. The result is that the section 1-2 is shorter and section 2-3 rotates to vertical.

Figure 6.9: Hip-knee angle plots. 1: initial contact 2: knee in flexion at end of load response; 3: hip and knee in maximum extension at end of terminal stance; 4: knee in maximum flexion during mid-swing.

6.2.2 Displacement and Velocity

The purpose of walking is to change one’s physical position in the world. The model advances steadily in the sagittal plane by manipulating joint angles between segments and reacting with the ground. Figure 6.10 plots trajectories of velocity and position of the pelvis COM in the sagittal plane.

Figure 6.10 (a) shows vertical velocity is positive (upward) or negative

(downward), varying with phases of the gait cycle. The pelvis goes up and down 72

correspondingly in a gait cycle (figure 6.10(b)). It goes up following initial contact, reaching the maximum height in mid-stance (approximately 30% of gait cycle) and goes down to the minimum when the other leg contacts the ground (approximately

50% of gait cycle). It rises, reaching the maximum again at approximately 80% of gait cycle, and goes down to the minimum at the end of the gait cycle. The difference between the maximum and minimum height is approximately 70 mm, 8% of its leg length.

Figure 6.10(c) shows the horizontal velocity also decreases and increases. The horizontal velocity is at its maximum following initial foot contact and decreases, reaching the minimum in mid-stance (approximately 30% of gait cycle). It then increases, reaching the maximum around 50% of the gait cycle when the other leg contacts the ground. The horizontal velocity repeats the same “U-shape” trajectory in the remaining 50% of gait cycle. Its horizontal displacement is approximately 1.25 m in one gait cycle (figure 6.10(d)). Although its horizontal velocity is not constant, it advances smoothly. Figure 6.10(e) shows it walks approximately 30 m in 30 seconds and average walking speed is 1.0 m/s.

Figure 6.10: Vertical position and velocity of Pelvis COM in a gait cycle.

The model’s walking is realized by its two feet alternately contacting the ground.

As a fixed point on the foot, the ankle’s displacement trajectory reveals some kinetic characteristics of the model’s walking. In figure 6.11 the ankle’s horizontal position changes little in the stance phase and advances in the swing phase until the foot contacts the ground again. The ankle travels approximately 1.25 m (about 149% leg 74

length) from the beginning to the end of the gait cycle, which is the stride length. The step length of the model is the distance between left and right ankles in the double support phase, which is approximately 0.5 m (about 60% of leg length).

Figure 6.11: The horizontal position of ankles in a gait cycle.

6.3 Kinetic Analysis

Kinetics relates forces and the resulting accelerations. In locomotion, kinetic analysis includes internal forces applied across joints, external forces applied by the ground, joint moments generated by muscle forces, and motion of joints. It is an important diagnostic of pathological gait and can assess the efficiency of gait. For this bipedal model we mainly focus on trajectories of vertical ground reaction force, muscle force, and joint moment in a gait cycle.

6.3.1 Ground Reaction Force

During walking, each stance foot applies force to the ground and the ground 75

applies ground reaction forces back on the foot. The magnitude and direction of the ground reaction force changes in the stance phase of each foot. In the sagittal plane the ground reaction force could be decomposed into a vertical component and a shear component.

Vertical ground reaction force provides the supporting force and is the major component in magnitude. Figure 6.12 presents the vertical ground reaction force of the model in a gait cycle. It shows a peak immediately following the ground contact and a hump between 10% to 60% of the gait cycle, when the foot is in stance phase.

Except for the first large peak and fluctuations, the maximal value of vertical ground reaction force is approximately 125% of the body weight. Comparing it with a human’s trajectory we can see they both have two humps in a stance phase, but the model’s first hump is much higher and narrower than a human’s. We expect that the soft and ball-shaped human heel helps reduce the impact force when it contacts the ground. In contrast the brick like foot of the model contacts the rigid ground with its edge, resulting in a large impact force. In addition, human gait has more knee flexion following heel contact, which also reduces the impact force.

Figure 6.12: Vertical ground reaction force compared with human’s [103] in a gait cycle. Force value is normalized by body weight.

6.3.2 Muscle Force

In the simulation, muscle forces of the model’s 6 muscles are recorded and plotted.

Figure 6.13 presents the model’s trajectories of muscle force in a gait cycle. The hip extensor muscle force jumps to approximately 700 N at the beginning and remains high in the stance phase. It drops at 50% of gait cycle before the leg leaves the ground. The hip flexor muscle force keeps low in the stance phase and jumps to approximately 600 N at 50% of gait cycle. The knee extensor muscle force is approximately 700 N before 50% of gait cycle and then drops to zero. The knee flexor muscle force is approximately 600 N between 50% and 70% of gait cycle, and remains almost zero the rest of the time. The ankle plantarflexor muscle force remains approximately 800 N between 40% and 75% of the gait cycle, and stays almost zero the rest of the time. The ankle dorsiflexor muscle force is approximately 450 N before

20% and after 75% of gait cycle, and stays low the rest of the time.

In this model, all muscle activities are driven by motoneurons, which are directly

under the control of PF neurons. So comparing figure 6.13 with figure 6.4 we could see that muscle activities are synchronized with the status of corresponding PF neurons. Generally, muscles corresponding to each joint are in reciprocal inhibition since corresponding PF neurons in each UPF module and motoneurons are mutually inhibited. Due to the passive properties of the muscle and the length change of the antagonist muscle there are some fluctuations in muscle forces and muscle force of the antagonist muscle is not always zero. The musculoskeletal model is a simplification of a human lower body. The 6 muscles in this model are not modeled one to one with human muscles as there are many more muscles in a human. So it is difficult to compare muscle activities of the model with experimental data of human leg muscles.

Figure 6.13: Muscle forces of the model in a gait cycle.

6.3.3 Joint Moment

The moments applied on a leg can be put into two categories: external moments, which are applied on a leg by external forces such as weight and ground reaction forces, and internal moments, which are applied by muscles and soft tissue. Joint moments displayed in figure 6.14 are internal moments at hip, knee, and ankle in the sagittal plane. They present the net effect of all agonist and antagonist activity, and could be considered as the final motor pattern at joints [104].

The hip moment trajectory shows extension moment in the first half of the stance phase and flexion moment in the second half of the stance phase and early swing. It has a small extension moment in the rest of the swing phase. The maximum hip extension moment is in the load response (approximately 10% of gait cycle) and the maximum hip flexion moment is in the pre-swing phase (approximately 50% of gait cycle). The knee moment trajectory has very similar shape as the hip moment trajectory. The difference is that maximum knee flexion moment is in the early swing

(approximately 60% of the gait cycle). The ankle moment trajectory shows a short period of dorsiflexion moment in load response and plantarflexion moment in pre- swing and early swing. It maintains dorsiflexion in the rest of the swing phase.

In figure 6.14, the model’s joint moment trajectories are compared with a human’s joint moment in the sagittal plane. All joint moments are normalized by total body mass (55 kg) to make them comparable. Generally, except fluctuations, the major

difference is that the model needs more moment to drive joint movement than a human with similar walking speed (1.0 m/s), especially at the hip and knee. For example, the maximum hip extension moment at the model’s hip is almost 2 times higher than the human’s value. It indicates the high efficiency of human walking. The shape of the model’s hip joint moment trajectory matches with the human’s. They all show extension moment before 25% and after 75% of gait cycle, and a peak of flexion moment at 50% of gait cycle. The knee joint moment trajectory resembles the human’s curve before 30% of gait cycle. But the model shows a short period of zero moment after the knee extension moment. Additionally, the moment at the model’s knee is an extension moment, not flexion moment in terminal swing (75% - 100% of gait cycle). This is due to the model’s controller being designed to keep the knee straight to prepare for load after terminal swing. The model’s ankle moment trajectory shows notable similarity with the human’s. They all have one peak of plantarflexion moment at 50% of gait cycle. But the model has a period of zero moment at the ankle between 25% and 40% of the gait cycle instead of plantarflexion moment. Because in the model’s CPG network Ankle DF and PF neurons are all inhibited when heel and toe both contact ground. The model also shows ankle dorsiflexion moment after 75% of gait cycle instead of zero moment at the human’s ankle. This is because the model’s controller is designed to leave the ankle in dorsiflexion to prevent the toe contacting ground before the heel.

Figure 6.14: Internal moments at hip, knee, and ankle in the sagittal plane compared with human’s [105] in a gait cycle. Joint moment is defined as positive in flexion and negative in extension. Joint moment values are normalized by body mass. EXT: extension, FLX: flexion, PF: plantarflexion, DF: dorsiflexion.

Chapter 7 Simulation of Walking with Perturbations

7.1 Changing Neural Network Configuration

In a series of simulations, we eliminated various synaptic connections in the network to investigate their contributions to walking. We let the model walk for 30 seconds and repeated 20 times for each case. The results are summarized in Table 7.1.

We also tried to increase the synaptic weights of these connections but the pattern of the neural network is not affected expect that the increased synaptic weight (over

5µS) of the excitatory connection from Hip FLX neuron to Ankle DF neuron diminishing ankle plantarflexion motion in the terminal stance. We expect this is because these synaptic weights are already large enough to induce synaptic currents which let corresponding CPG neurons or integrator neurons spike.

Table 7.1: Walking performance after cutting off one synaptic connection

From To Walking Performance Stance Inhibit (The Other Heel GC Fails to walk Leg) Hip EXT Knee EXT Walk but fail after a distance

Heel GC Knee FLX Inhibit No obvious effect

Still walk but falls down in 6 Hip FLX Ankle DF trials (30%)

Heel GC Ankle PF Inhibit No obvious effect

Toe GC Ankle DF Inhibit Walking speed decreases

Hip Mov Knee FLX Inhibit Fails to walk

More details are analyzed as follows:

1) Heel GC sensory neuron to Stance inhibition integrator neuron (the other leg)

The model fails to walk when the connection between Heel GC afferent and the

Stance inhibition integrator neuron corresponding to the other leg is severed. This synaptic connection carries the key signal that modulates the pattern of the RG oscillator according to whether the swing leg’s foot contacts ground.

2) Hip EXT neuron to Knee EXT neuron

The model could walk a distance and then fall when the excitatory synaptic connection from Hip EXT neuron to Knee EXT neuron is cut. This connection encourages that when the hip joint is in extension the knee joint remains in extension also. When it is cut, the Knee EXT neuron spikes as long as the Heel GC sensory neuron inhibits the Knee FLX neuron during mid-stance. When the heel leaves ground in the terminal stance the Heel GC sensory neuron stops spiking and Knee

FLX neuron is not inhibited by it. At the same time, the Knee UPF module does not receive any synaptic input from other neurons (Hip Mov sensory neuron and Hip FLX neuron are also quiet) so which neuron spikes depends on post synaptic currents between the two neurons.

Figure 7.1 provides more details. At time A, the right Heel GC stops spiking but the right Knee EXT neuron still keeps spiking for a short period until right Hip FLX

neuron spikes, which is triggered by the left leg Heel GC contact sensor neuron spiking. The left leg begins to take load and the right leg is in pre-swing (figure

7.1(a)). At time B, the right Knee FLX neuron begins to spike immediately when right

Heel GC sensory neuron stops spiking. At the same time, the left leg is still in the air and the left Heel GC neuron is still quiet. This means that the right knee begins to flex when the right leg still supports the total weight. Normal walking collapses and the model falls backward (figure 7.1(b)).

Figure 7.1: Neuron membrane potentials during walking without synaptic connection from Hip EXT neuron to Knee EXT neuron.

Figure 7.2: Posture at the end of terminal stance. (a) normal (b) failure.

2）Heel GC sensory neuron to Knee FLX inhibition integrator neuron

The walking performance of the model is not obviously affected when the synaptic connection from Heel GC sensory neuron to the integrator neuron connecting Knee

FLX neuron is severed. The role of the connection is to inhibit Knee FLX neuron from spiking when the heel is on the ground.

Figure 7.3 shows Knee FLX neuron is inhibited most of the time except some sporadic spiking in the stance phase. The duration of this sporadic spiking is not long enough to change the movement of the legs. As discussed in section 7.2 the Hip EXT neuron keeps spiking until the other leg contacts ground. The Knee EXT neuron has an excitatory synaptic connection from Hip EXT neuron. So even without the synaptic connection from Heel GC sensory neuron, the Knee EXT neuron spikes when the Hip EXT neuron spikes. It is still useful to keep the synaptic connection to

inhibit Knee EXT neuron from sporadic spiking.

Figure 7.3: Neuron membrane potentials during walking without synaptic connection from Heel GC neuron to Knee FLX inhibition neuron.

3）Hip FLX neuron to Ankle DF neuron

The model is still able to walk but its stability is affected when the synaptic connection from Hip FLX neuron to Ankle DF neuron is severed. The role of this connection is to keep Ankle DF neuron spiking and make the ankle dorsiflex during swing phase. Without that synaptic connection, Ankle PF neuron may keep spiking and leave the ankle in plantarflexion during swing phase, which causes the model to overly lean forward and fall.

When the foot is in the air the Ankle UPF module does not receive any synaptic currents from other neurons (Toe GC and Hip Mov sensory neurons are both in 87

silence) so the output is uncertain, depending on postsynaptic currents between the two neurons. In figure 7.5, comparing to the model with this intact connection, the model without the connection shows more ankle plantarflexion motion. In figure 7.4 at time A, Ankle DF neuron is spiking before the Hip EXT neuron begins to spike.

The ankle is in dorsiflexion (time A in figure 7.5(b)) and heel contacts ground first at the initial contact (figure 7.6(a)). The model walks normally. At time B, Ankle PF neuron is spiking until the Hip EXT neuron begins to spike. So the ankle is in plantarflexion (time B in figure 7.5(b)) and toe contacts ground first (figure 7.6(b)) instead of heel, resulting in a failure to trigger the stance phase.

Figure 7.4: Neuron membrane potentials during walking without synaptic connection from Hip FLX neuron to Ankle DF neuron.

Figure 7.5: Comparison of ankle joint angles between with and without the synaptic connection from Hip FLX neuron to Ankle DF neuron.

Figure 7.6: Posture at the end of swing phase (a) heel firstly contacting ground; (b) toe firstly contacting ground.

4) Heel GC sensory neuron to Ankle PF inhibition integrator neuron

The walking performance of the model is not obviously affected when the synaptic connection from the Heel GC sensory neuron to the Ankle DF inhibition integrator neuron is cut off. The role of the connection is to inhibit Ankle PF neuron from spiking when the Heel GC sensory neuron spikes. So the ankle maintains dorsiflexion following the heel contacting ground at the beginning of stance phase.

Figure 7.7 shows that if this synaptic connection is removed, the Ankle PF neuron keeps spiking during the whole stance phase. Following the initial contact, Hip FLX neuron stops spiking and Toe GC sensory neuron does not begin to spike yet so the

Ankle UPF module does not receive any synaptic current from other neurons, resulting in its output being uncertain. If the Ankle PF neuron is spiking after the initial contact, the ankle joint quickly rotates into plantarflexion (figure 7.8(b)) instead of keeping dorsiflexion (figure 7.8(a)). The early plantarflexion impedes the stance leg rotating along the edge of the heel and reduces the model’s horizontal velocity.

Figure 7.9(a) shows with the synaptic connection the ankle plantarflexion only happens at the terminal stance. The trajectories of the pelvis COM horizontal velocity are relatively constant in each gait cycle. Figure 7.9(b) shows early plantarflexion happens without the synaptic connection and the velocity drops when it happens (11 seconds to 14 seconds). The velocity trajectories also show more fluctuations between each gait cycle.

Figure 7.7: Neuron membrane potentials during walking without synaptic connection from Heel GC sensory neuron to the Ankle PF inhibition integrator neuron.

Figure 7.8: Posture following the initial contact (a) ankle in dorsiflexion; (b) ankle in plantarflexion.

Figure 7.9: Comparison of ankle joint angles between with and without the synaptic connection from Heel GC sensory neuron to the Ankle PF inhibition integrator neuron.

5) Toe GC afferent

Without the Toe GC afferent, the model still walks stably but its ankle plantarflexion movement disappears and its walking speed reduces. The role of the

Toe GC feedback is to inhibit Ankle DF neuron in the Ankle UPF module when the toe contacts ground. The Ankle PF neuron is inhibited by Heel GC afferent so both

Ankle DF and Ankle PF neuron are inhibited when the whole foot contacts ground in

mid-stance. The Ankle PF neuron is released from inhibition and spikes as soon as the heel leaves the ground while the Ankle DF neuron is still inhibited by the Toe GC afferent. As a result, the ankle extends and pushes the model forward (figure 7.11(a)) during terminal stance. Figure 7.10 shows that the Ankle DF neuron keeps spiking almost constantly during the whole gait cycle when the Toe GC afferent is cut off. The ankle joint is approximately 0° at the end of the terminal stance (figure 7.11(b)).

In 20 trials of 30 seconds walking with and without the Toe GC afferent, the average walking speed of the model with the Toe GC afferent is 0.959 m/s compared with 0.6499 m/s without the Toe GC afferent (figure 7.12). Figure 7.13 shows removing the Toe GC afferent mainly eliminates ankle joint movement. As a result, the maximum value of the horizontal velocity of the pelvis COM drops from approximately 1.1 m/s to 0.9m/s; the minimum value drops from approximately 0.8 m/s to 0.6 m/s. Walking cycles reduce from 9 cycles to 8 cycles in 10 seconds.

Figure 7.10: Neuron membrane potentials during walking without Toe GC feedback.

Figure 7.11: Posture at the end of terminal stance (a) ankle in plantarflexion with Toe GC afferent; (b) ankle in dorsiflexion without Toe GC afferent.

Figure 7.12: Comparison of average walking speed of walking speed between models with and without Toe GC afferent.

Figure 7.13: Joint angles and horizontal velocity of pelvis COM (a) with Toe GC afferent (b) without Toe GC.

6) Hip Mov afferent

The model fails to walk when Hip Mov afferent is removed. As shown in figure

7.14 the swing leg’s knee still keeps flexion in the terminal swing so it is not able to properly contact ground and take load. The role of Hip Mov afferent is to initiate the swing leg’s knee extension by inhibiting Knee FLX neuron in mid-swing and terminal swing when hip joint passes a predetermined position (here it is 5° hip angle). When it is removed, the swing leg’s Knee FLX neuron keeps spiking during the whole swing phase since the Knee EXT neuron is always inhibited by Hip FLX neuron during swing phase.

Figure 7.14: Posture in terminal swing (a) Knee extending with Hip Mov afferent (b) Knee keeping flexion without Hip Mov afferent.

7.2 Walking with Different Body Weights

In a series of experiments, we let the model walk on level ground with different pelvis mass to investigate how different body weight affects walking. Movement

limits of hip, knee, and ankle joints are preset to -10° to 27°, 0° to 90°, and -15° to 0°.

Pelvis COM displacements are all preset to 30 mm in the forward direction. Models with heavier pelvis mass need more forward force to help them initiate walking.

Models with 20 kg and 70 kg pelvises need 10 N and 60 N push force, respectively.

We let each model walk 30 seconds for 20 times, recorded its walking distance each time, and calculated its average walking speed.

Results show that models could keep stable walking with pelvis mass up to 70 kg, and autonomously adjust walking speed to adapt varying carrying loads. Models with heavier pelvises walk more slowly. Figure 7.15 shows its average walking speed drops approximately 43%, from 1.072 m/s to 0.607 m/s when the mass of pelvis increases from 20 kg to 70 kg (total mass 45 kg to 95 kg).

Figure 7.15: Average walking speed of models with different pelvis masses.

Figure 7.16 shows the maximum and minimum horizontal velocities decrease from

approximately 1.3 m/s to 0.9 m/s, and 0.9 m/s to 0.55 m/s respectively, when the mass of the pelvis increases from 20 kg to 70 kg. Ranges of joint movements change little but the period of the gait cycle increases from approximately 1 sec to 1.6 sec. So there are less gait cycles in a fixed time period when pelvis COM displacement increases as shown in figure 7.16. The model almost maintains the same step length but reduces its step frequency, reducing walking speed, when the load it carries increases. Increasing load also causes more knee flexion during the stance period to absorb more kinetic energy. It emphasizes that the passive characteristics of the muscle help the model to adapt to increasing loads.

Figure 7.16: Joint angles and pelvis COM velocities of models with 20 kg and 70 kg pelvises.

7.3 Walking with Horizontal Pelvis COM Displacements

In a series of experiments, we set the model’s pelvis COM displacement to different values and let them walk 30 seconds for 20 times on level ground. We recorded their walking distance each time and calculated their average walking speed.

The mass of pelvis was set to 30 kg. Movement limits of hip, knee, and ankle joints are preset to -15° to 27°, 0° to 90°, and -15° to 0°.

Results show that the model can adapt to varying pelvis COM position up to 8 cm by autonomously changing its walking speed. The farther forward the pelvis COM is displaced, the faster it walks. Figure 7.17 shows that its average walking speed increases from 0.850 m/s to 1.289 m/s when its pelvis COM displacement increases from 2 cm to 8 cm. It is interesting to note that the small displacement of the pelvis

COM in the forward direction provides a moment that helps the model roll forward on the foot at mid-stance. The model falls back occasionally when its pelvis COM displacement is less than 20 mm.

100

Figure 7.17: Average walking speed of models with different pelvis COM displacements

Figure 7.18 shows the maximum and minimum horizontal velocity increase from approximately 1.2 m/s to 1.6 m/s, and 0.7 m/s to 1.1 m/s respectively, when the pelvis

COM horizontal displacement is increased from 2 cm to 8 cm. Ranges of joint movements change little but the period of the gait cycle decrease from approximately

1.3 second to 1.0 second. So there are more gait cycles in a fixed time period when the pelvis COM displacement increases as shown in figure 7.18. It means the model approximately maintains the same step length but increases step frequency to increase walking speed, adapting to pelvis COM displacement.

101

Figure 7.18: Joint angles and pelvis COM velocities of models with 20 mm and 80 mm pelvis COM displacements.

102

7.4 Walking Over Obstacle

In series of experiments bricks of different heights are placed on the model’s route.

The model (pelvis mass 30 kg, pelvis displacement 30 mm is able to pass obstacle as high as 50 mm, which is about 6% its leg length without reconfiguring its neural control network. Figure 7.19(a) shows continuous a series of frames of the model walking over a 50 mm high obstacle and continuing its normal walking. At frame A and frame B its right leg steps on and down the obstacle respectively. At frame C and frame D its left leg step on and down the obstacle following its right leg. Figure

7.19(b) shows between time A and time B the model is on the obstacle according to its pelvis COM rising. The trajectory of the pelvis COM’s horizontal velocity drops following time A because a part of the kinetic energy transfers to potential energy.

When the model steps down the obstacle following time B the trajectory of velocity rises but soon recovers to normal level before time A.

Figure 7.19(b) also shows the stance phase of the right leg following stepping on the obstacle (from time A to time B) is longer than the stance phase following stepping off the obstacle (from time C to time D). It is because that the spiking period of the right leg stance neuron in the RG (figure 7.19(d)) is entrained by the spiking periods of right heel and left heel contact sensory neurons (figure 7.19(c)), which depends on the contacting between feet and the terrain. Right hip extension neuron and flexion neurons in the PF are under the control of the RG and generate signals to

103

control the right hip joint movement adapting the terrain (figure 7.19(d)). The passive dynamics of the model also makes its movements have small adjustments to adapt to the sudden changes in terrain when it steps on and down the obstacle.

104

105

Figure 7.19: Walking over a 50 mm high obstacle. (a) Frames of the model walking up. (b) Joint angles, pelvis COM height, and horizontal velocity. (c) Sensory neuron membrane potentials. (d) RG and PF neuron membrane potentials.

7.5 Walking On Slopes

Humans lean forward on an inclined treadmill and lean backward on a declined treadmill [106]. To help the model walk on slopes we manually increase the horizontal displacement of its pelvis COM when it walks up inclines and decrease it

106

when the model walks down declines.

1) Walking up an incline

We released the model on horizontal ground and let it walk up inclines. The mass of the pelvis is 25 kg. The displacement of the pelvis COM is 5 cm. The Movement limits of hip, knee, and ankle joints are preset to -15° to 27°, 0° to 90°, and -15° to 0°.

Results show that the model can walk up inclines up to 5° and continue to walk on the horizontal surface connected to the top of incline (figure 7.20(a)). In figure 7.20(b) between time A and time B the model is walking up the incline. The pelvis COM horizontal velocity decreases when the COM rises, because kinetic energy is transferring to potential energy. When the model walks on a horizontal surface again after time B the pelvis horizontal COM velocity gradually recovers to similar trajectory before time A. When the model is waking on the incline, its hip stays at maximum flexion position for a longer time than when it is walking on level ground.

Its gait cycle recovers to normal when the model walks on the horizontal surface.

107

Figure 7.20: Walking up a 5° incline. (a) Frames of the model walking up. (b) Joint angles of right leg, pelvis COM height, and horizontal velocity.

2) Walking down a decline

We release the model on horizontal ground and let it walk down declines. The mass of the pelvis is 25 kg. The displacement of pelvis COM is 2 cm. The movement limits of hip, knee, and ankle joints are preset to -15° to 27°, 0° to 90°, and -15° to 0°.

108

Results shows the model can walk down declines of -5° and continue to walk on the horizontal ground (figure 7.21(a)). In figure 7.21(b) between time A and time B, the model is walking down the decline. The pelvis COM horizontal velocity increases when COM descends, because potential energy is transferring to kinetic energy. When the model walks on horizontal ground after time B the pelvis horizontal COM velocity gradually recovers to a trajectory similar to before time A. The duration of the gait cycle decreases when the model is on the decline. The hip stays at maximum flexion position for a shorter time. Its gait cycle recovers when the model walks on the horizontal ground connected to the bottom of the decline.

In summary, results of a series of experiments show that the model can adapt to walking on slightly inclined or declined surfaces with corresponding walking speed. It is realized by adapting its step frequency (gait cycle period) without changing its neural control network configuration.

109

Figure 7.21: Walking down a -5° decline. (a) Frames of the model walking down (b) Joint angles of right leg, pelvis COM height, and horizontal velocity.

110

Chapter 8 Conclusions and Future Works

8.1 Contributions

Contributions of this work to the biologically inspired control methodology for bipedal walking are summarized as follows:

1) A plantar musculoskeletal model mimicking human lower body.

The musculoskeletal model we developed in this dissertation is a 2D simplification of a human’s lower body. Each leg has 4 DOFs and contains six muscles modeled with Hill’s model and one passive spring component. This reduces the redundancy of the human leg muscle system to the minimal but still keeps the same motion ability in the sagittal plane. The similar geometry and mass distribution of leg segments makes its morphology similar to a human’s, which combined with the passive property of muscle system reduces the control effort during walking. The more human-like two- part foot structure not only enhances the stance foot’s ground contact at terminal stance, but also makes it possible to provide more accurate dynamic sensory information of the relation between the foot and ground during stance by using individual sensors on the toe and heel.

2) A neural network with CPGs entrained by sensory feedback

The neural network we developed has several important features: Firstly, the neural network is not a static network. It is composed of biological neuron models. Secondly,

111

it is a two-level structure. Hip, knee, and ankle movements in each leg are coordinated in the pattern formation level. The gait phases of the two legs are coordinated in the higher-level rhythm generator level. Compared with most one-level organizations in which hip pattern generators are directly coupled, it is not only biologically plausible but also more easily expanded into a more complex structure required for mimicking more complicated neural behaviors. Thirdly, in this model, all sensory feedback is biologically realistic. Load and hip joint movement feedback entrains the output of

CPG network instead of directly modifying motoneuron activity, following what is currently known [45-47]. The interaction between sensory neurons and CPG neurons is easy to view during walking. Compared with using coupled abstract mathematical equations to control muscle activities, it provides a clear, tangible, and bio-inspired picture of how sensory afferents are integrated with the central neural network controlling bipedal walking. Additionally, this neural control network does not depend on specific body dynamics and can be easily transplanted to other bipedal models.

3) A neuromechanical simulation of human-like walking in the sagittal plane

In simulations the model walks stably in the sagittal plane without any “baby- walker” devices to prevent it falling down. It has no vestibular system. It is solely controlled by a dynamic neural network mimicking the spinal cord without any additional higher level control actions. Its controller does not need to track joint angles or the position of the COM. A dynamic model of the biomechanical system is

112

not required. The simulation model achieved walking speeds up to 1.29 m/s (about

4.64 km/h) on level ground with 55 kg body mass. Its gait appears human-like, with has posture similar to a human’s in different phases of the gait cycle. It also has similar trajectories of joint movements (hip, knee, ankle, and metacarpophalangeal joint) in the sagittal plane. The trajectories of joint moment at hip, knee and ankle joints are also comparable to human’s.

4) Investigating how different synaptic connections affect walking

In simulations we lesioned various synaptic connections between sensory neurons and CPG neurons and between CPG neurons themselves to investigates how they affect walking. Some connections are indispensable and stable walking is impossible without them. In this model, the transition from swing phase to stance phase and maintaining hip extending movement in stance phase relies on the heel ground contact sensory feedback. The swing leg’s knee extension before ground contact relies on the hip movement feedback. The connections between hip UPF neurons and knee UPF neurons are crucial to keep the knee straight when the hip is extending and initiates knee flexion when the hip begins to flex

Some synaptic connections are not crucial but still affects the model’s walking.

Connections from Heel GC sensory neuron and Hip FLX neuron to Ankle UPF neurons cause ankle dorsiflexion during the gait cycle except the terminal stance. The walking stability decreases without them. The connection from Toe GC sensory

113

neuron to Ankle DF neurons reinforces the ankle plantarflexion motion at terminal stance. The walking speed decreases without it. The connection from the Heel GC sensory neuron to the Knee FLX neuron reinforces the knee extension when the leg is loaded.

5) Investigating how the model adapts to different body setting and terrains

In a series of simulations, the model is walked over a 50 mm high obstacle (6% leg length), walked up a 5° incline, and walked down a -5° decline. It is able to walk with pelvis mass changed from 20 kg to 70 kg (total mass from 45 kg to 95 kg) and also is capable of walking with its pelvis COM displaced from 20 mm to 80 mm in the forward direction. No adjustment of its neural control network is needed. These simulations explore the robustness of this bipedal model and further demonstrate that a neural controller composed of CPGs entrained by ground contact and joint movement sensory feedback combined with the inherent dynamics of human morphology in the sagittal plane is not only enough to generate human-like walking but is also adaptive to uneven terrains, slopes, varying loads, COM displacements to some extent in the sagittal plane. This model achieves a balance between robustness and simplicity. It generates more adaptive walking than powered passive dynamic walkers but requires less control effort than those relying on high-gain feedback control methods and powerful actuators.

114

8.2 Future Works

The bipedal model, including the neural control network and the biomechanical model, is flexible and expandable so further works could be continued based on it as follows:

1) Leg model

Muscle in our leg model lacks the corresponding relations with human leg muscle in anatomy. So it is difficult to compare their activities with human leg muscle activities during walking. It is missing biarticular muscles existing in human legs, which cross two joints instead of one and drive the movement of two joints instead of one. For example, human’s hamstrings, which cross the hip and the knee, make the hip and knee extend at the same time when they contract. So the original leg model could be improved with a more realistic muscle configuration in future work. One of the choices is the leg model illustrated in figure 8.1 [107]. It includes 9 muscles, which provides reasonably accurate biomechanical basis of investigating activities of individual muscles.

Furthermore, torso, arms, and head could be added to this model and expand it from lower body to the whole body. More complex muscle systems, high position of

COM, and moving arms give more challenges to the neural control network but it could be interesting to investigate interactions between the neural control network and whole body dynamics.

115

Figure 8.1: A nine-muscle human leg model (Redrawn from Prilutsky and Zatsiorsky [106]).

2) Muscle afferents

This model uses sensory input from ground contact sensors as indirect input of leg load during walking and hip angle as input of hip position. However, in animals and humans major load receptors are Ib afferents from Golgi tendon organs and the stretch receptors are type II muscle-spindle afferents. Muscle afferents not only modulate the

CPG network but also project to specific motoneurons to directly affect muscle activities. Sensory feedback from Ib and II afferents are provided in Animatlab. So it is necessary to replace ground contact sensors and hip angle sensors in this model with type Ib and II afferents. It will not only provide more precise and realistic sensory feedback but also make the neuromechanical simulation more comparable with experimental data. The major role of sensory feedback in this controller is to modulate the CPG but muscle afferents also directly project to specific motoneurons

116

to affect muscle activities. So in future work, connections between muscle afferent input and corresponding motoneurons could be added to facilitate generating more complex and flexible muscle activity.

3) Neuron model

The one-dimensional integrate-and-fire neuron model is good enough for this model but it does lack some important neuron features such as phasic bursting, which is a period of rapid spiking followed by a resting period. In Animatlab, a calcium channel could be added to the integrate-and-fire model and make it an integrate-and- fire-or-bust model, which is capable of bursting and other properties. Other neuron models [108] and neuron population models could also be considered in future work.

4) Supraspinal control

Human bipedal walking has greater dependency on supraspinal control and the control of muscle activity by monosynaptic projections directly from the brain motor cortex to the spinal motoneurons and significant involvement of transcortical pathways in reactions to disturbance during walking compared with animals’ locomotion [109]. The neural control network of this model mimics the spinal neural circuits without any supraspinal input, so its ability to overcome large disturbances during walking is still very limited. Adding supraspinal control to the original neural network is necessary to increase the model’s ability to adapt to irregular terrain and large disturbances. Although a feedback control model was developed, which 117

reproduces human muscle activities responding to support-surface translations in standing [110], a supraspinal control model is yet to be developed, which could modify spinal neural network output and adjust muscle activities responding to disturbances during walking. In addition, human walking is passively unstable in the coronal plane, also requiring active control from the supraspinal level against dynamic instability [111]. So if we want to upgrade this 2D model into a 3D model, more

DOFs must be added to the mechanical model and higher level feedback control must be integrated into the original controller, achieving lateral stability.

118

Appendix A: Neuron Properties

Resting Threshold Time AHP Time Relative Neuron Potential Potential Constant Constant Accommodation (mV) (mV) (ms) (ms) Sensory Neuron -60 -55 5 0.3 3

Integrator Neuron -60 -55 5 0.3 3

CPG Neuron -60 -55 5 0.3 5

Motoneuron -60 -59 5 0.3 90 Muscle -100 50 5 0.3 3 Membrane

AHP: after-hyperpolarization

119

Appendix B: Synapse Properties

Reversal Time From Neural To Neuron Potential Constant Neuron Facilitation (mV) (ms) -50 Sensory Integrator 3 1 (Excitatory) -70 Integrator CPG 10 1 (Inhibitory) -70 CPG CPG 10 0.8 (Inhibitory) -50 CPG CPG/Integrator 3 1 (Excitatory) -10 CPG Motoneuron 3 1.5 (Excitatory) -70 CPG Motoneuron 10 1 (Excitatory) Muscle -10 Motoneuron 3 1.5 Membrane (Excitatory)

120

Appendix C: Synaptic Weights in the CPG Neural Network

Min Value Max Value From Neuron To Neuron (µS) (µS)

Heel GC Stance (the other leg) 3.5 No limit

Heel GC Knee FLX Inhibition 4 No limit

Heel GC Ankle PF Inhibition 3 No limit

Toe GC Ankle DF Inhibition 3.5 No limit

Hip Mov Knee FLX inhibition 3.5 No limit

Stance Hip EXT 1 No limit

Stance Hip FLX (the other leg) 1 No limit

Hip FLX Knee EXT Inhibition 3.5 No Limit

Hip FLX Ankle DF 0.5 5

Hip EXT Knee EXT 0.5 No Limit

121

Bibliography

[1] Kato, “The WABOT-1,” Bulletin of Science and Engineering Research Laboratory, Waseda University, no. 62, 1973. [2] Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, K. Fujimura, “The intelligent ASIMO: system overview and integration,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and System (IROS), pp. 2478-2483, IEEE, 2002. [3] W. Park, J. Y. Kim, J. Lee, and J. H. Oh, “Online free walking trajectory generation for biped humanoid robot KHR-3(HUBO),” in Humanoid Robots, 2005, 5th IEEE-RAS International Conference on, pp. 321-326, IEEE, 2005. [4] T. Buschmann, S. Lohmeier, H. Ulbrich, “Biped walking control based on hybrid position/force control,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and System (IROS), pp. 2478-2483, IEEE, 2009. [5] K. Kaneko, et al., “Humanoid robot hrp-4-humanoid robotics platform with lightweight and slim body,” in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, pp. 4400-4407, IEEE, 2011. [6] K. Sreenath, H. W. Park, I. Poulakakis, and J. W. Grizzle, “A compliant hybrid zero dynamics controller for stable, efficient and fast bipedal walking on MABEL,” The International Journal of Robotics Research, vol. 30(9), pp. 1170- 93, 2011. [7] M. Vukobratovic, and B. Borovac, “Zero-moment point thirty-five years of its life,” International Journal of Humanoid Robotics, vol. 1(1), pp. 157-173, 2004. [8] J. Pratt, P. Dilworth, G. Pratt, “Virtual model control of a bipedal walking robot,” in Robotics and Automation, IEEE International Conference on, vol. 1, pp. 1993- 198, IEEE, 1997. [9] J. W. Grizze, W. Jessy, and E. R. Westervelt, “Hybrid zero dynamcis of planar bipedal walking,” Analysis and design of nonlinear control systems, Springer Berlin heidelberg, 223-237, 2008. [10] R. Tedrake, “LQR-Trees: Feedback motion planning on sparse randomized trees,” in Proceedings of Robotics Science and Systems (RSS), Zaragoza, Spain, 2009. [11] T. McGeer, “Passive dynamic walking,” International Journal of Robotics Research, vol. 9, 62-82, 1990. [12] S. H. Collins, M. Wisse, and A. Ruina, “A three-dimensional passive-dynamic walking robot with two legs and knees,” The International Journal of Robotics Reseach, vol. 20(7), pp. 607, 2001. [13] S. Collins, A. Ruina, R. Tedrake, and M. Wisse, “Efficient bipedal robots based on passive-dynamic walkers”, Science, vol. 307, pp. 1082-1085.

122

[14] Russ Tedrake, Underactuated robotics: Algorithms for walking, running, swimming, flying, and manipulation (Course Notes for MIT 6.832), 2014. [15] T. Luksch, Human-like control of dynamically walking bidpeal robots. Verlag Dr. Hut, 2010. [16] C.S. Sherrington, “Flexion-reflex of the limb, crossed extension reflex and the stepping reflex and standing,” J. Physiol., vol. 40, pp. 28-121, 1910. [17] T. G. Brown, “The intrinsic factors in the act of progressing in the mammal,” Proc. R. Soc. Lond. Ser. B, vol. 84, pp. 308-319, 1911. [18] V. Dietz, “Spinal cord pattern generations for locomotion,” Clinical Neurophysiology, vol. 114, pp. 1379-1389, 2003 [19] H. Hultborn, J. Nielsen, “Spinal control of locomotion—from cat to man,” Acta. Physiol., vol. 189, pp. 111-121, 2007. [20] P. A. Guertin, “Central pattern generator for locomotion: anatomical, physiological, and pathophysiological considerations,” Frontiers in Neurology, vol. 3, pp. 4-15, 2013. [21] S. Rossignol, R. Dubuc, J. P. Gossard, “Dynamic sensorimotor interaction in locomotion,” Physiol. Rev., vol. 86, pp. 89-154, 2006. [22] M. Y. C. Pang, and J. F. Yang, “The initiation of the swing phase in human infant stepping: importance of hip position and leg load, ” Journal of Physiology, vol. 528(2), pp. 389-404, 2010. [23] Auke Jan Ijspeert, “Central pattern generators for locomotion control in animals and robots: a review,” Neural Networks, vol. 21, pp. 642-653, 2008. [24] P. S. Dickinson, “Interactions among neural networks for behavior,” Curr. Opin. Neurobiol., vol. 5, pp.792-798, 1995. [25] K. Minassian, I. Persy, F. Rattay, M. M. Pinter, H. Kern, M. R. Dimitrijevic, “Human lumbar cord circuitries can be activated by extrinsic tonic input to generate locomotor-like activity,” Hum. Mov. Sci., vol. 26, pp. 275-295, 2007. [26] B. Calancie, B. Needham-Shropshire, et al., “Involuntary stepping after chronic spinal cord injury: evidence for a central rhythm generator for locomotion in man,” Brain, 1994, vol. 117, pp. 1143-1159, 1994. [27] B. Calancie, “Spinal myoclonus after spinal cord injury,” J. Spinal Cord Med., vol. 29, pp. 413-424, 2006. [28] T. G. Brown, “On the nature of the fundamental activity of the nervous centres; together with an analysis of the conditioning of rhythmic activity in progression, and a theory of the evolution of function in the nervous system,” J. Physio., vol. 48, pp. 18-46, 1914. [29] E. Jankowska, M. G. Jukes, S. Lund, and A. Lundberg, “The effect of DOPA on the spinal cord. 6. Half-centre organization of interneurones transmitting effects form the flexor reflex afferents,” Acta Physiol. Scand., vol. 70(3), pp. 389-402, 1967.

123

[30] E. Jankowska, M. G. Jukes, S. Lund, and A. Lundberg, “The effect of DOPA on the spinal cord. 5. Reciprocal organization of pathways transmitting excitatory action to alpha motoneurones of flexors and extensors,” Acta Physiol. Scand., vol. 70(3), pp. 369-388, 1967. [31] S. Rossignol, “Neural control of stereotypic limb movements,” in: L. B. Rowell, J. T. Sheperd, editors. Exercise: Regulation and Integration of Multiple Systems. Handbook of Physiology. Sect. 12., Bethesda, MD: Am. Physiol. Soc. 1996: 173- 216. [32] S. Grillner, “Control of locomotion in bipeds, tetrapods, and fish,” Comprehensive Physiology, pp.1179–1236, 2011. [33] S. Grillner, P. Zangger, “On the central generation of locomotion in the low spinal cat,” Exp. Brain Res., vol. 34(2), pp. 241-161, 1979. [34] L. M. Jordan, “Brainstem and spinal cord mechanisms for the initiation,” in Neurobiological Basis of Human Locomotion, eds. M. Shimamura, S. Grillner, V. R. Edgerton, Tokyo: Japan Scientific Societies Press, p. 3-20, 1991. [35] V. V. Tukin, T. M. Hamm, “Changes in locomotor drive potentials and cycle characteristics associated with deletions during fictive locomotion,” Soc. Neurosci. Abst., vol. 34, 883.14, 2004. [36] M. Lafreniere-Roula, D. A. McCrea, “Deletion of rhythic motoneuron activity during fictive locomotion and scratch provide clues to the organization of the mammalian central pattern generator,” J. Neurophysiol., vol. 94, pp. 1120-1132, 2005. [37] J. Duysens, “Reflex control of locomotion as revealed by stimulation of cutaneous afferents in spontaneously walking premammillary cats,” J. Neurophysiol., vol. 40, pp. 737-751, 1977. [38] A. Rybak, N. A. Shevtsova, M. Lafreniere-Roula, D. A. McCrea, “Modelling spinal circuitry involved in locomotor pattern generation: insights from deletions during fictive locomotion,” J. Physiol., vol. 577, pp 617-639, 2006. [39] A. Rybak, K. Stecina, N. A. Shevtsova, D. A. McCrea, “Modelling spinal circuitry involved in locomotor pattern generation: insights from afferent stimulation” J. Physiol., vol. 577, pp 641-658, 2006. [40] K. G. Pearson, “Generating the walking gait: role of sensory feedback,” Prog. Brain Res., vol. 143, pp. 123-129, 2004. [41] K. G. Pearson, “Proprioceptive regulation of locomotion in vertebrates and invertebrates,” Ann. Rev. Neurosci., vol. 16, pp. 265-297, 1993. [42] P. Whelan, “Control of locomotion in the decerebrate cat,” Prog. Neurobiol., vol. 49, pp. 481-515, 1996. [43] V. Dietz, J. Duysens, “Significance of load receptor input during locomotion: a review,” Gait and Posture, vol. 11, pp. 102-110, 2000. [44] D. A. McCrea, “Spinal circuitry of sensorimotor control of locomotion,” The Journal of Physiology, vol. 533(1), pp. 41-50, 2001.

124

[45] J. Duysens, K. G. Pearson, “Inhibition of flexor burst generation by loading ankle extensor muscles in walking cats,” Brain Res., vol. 187, pp. 321-332, 1980. [46] P. Guertin, M. J. Angel, M. C. Perreault, D. A. McCrea, “Ankle extensor group I afferents excite extensors throughout the hindlimb during MLR-evoked fictive locomotion in the cat,” J. Physiol., vol. 487, pp. 197-209, 1995. [47] Andersson, S. Grillner, “Peripheral control of the cat’s step cycle. II. Entrainment of the central pattern generator for locomotion by sinusoidal hip movements during “fictive locomotion”,” Acta. Physiol. Scand., vol. 118(3), pp. 229-239, 1983. [48] D. J. Kriellaars, R. M. Brownstone, B. R. Noga, L. M. Jordan, “Mechanical entrainment of fictive locomotion in the decerebrate cat,” Journal of Neurophysiology, vol. 71, no. 6, pp. 2074-2086, 1994. [49] D. P. Ferris, P. Aagaard, E. B. Simonsen, C. T. Farley, and P. DyhrePoulsen, “Soleus H-reflex gain in humans walking and running under simulated reduced gravity,” J. Physiol., vol. 530, pp. 167-180, 2001. [50] V. Dietz, “Do human bipeds use quadrupedal coordination?” Trends Neurosci., vol. 25, pp. 462-467, 2002. [51] D. A. Winter, Biomechanics and motor control of human gait: normal, elderly and pathological. 1991. [52] T. McGeer, “Dynamics and control of bipedal locomotion”, Journal of Theoretical Biology, vol. 163, pp. 277-314, 1993. [53] M. Garcia, A. Chatterjee, A. Ruina, M. Coleman, “The simplest walking model: stability, complexity, and scaling,” J. Biomech. Eng., vol. 120, pp. 281-288, 1998. [54] M. Wisse, J. Van Frankenhuyzen, “Design and construction of MIKE; a 2-D autonomous biped based on passive dynamic walking,” In Adaptive Motion of Animals and Machines Springer Tokyo, pp.143-154, 2006 [55] D. Kuo, “A simple model of bipedal walking predicts the preferred speed-step length relationship,” J. Biomech. Eng., vol. 123(3), pp. 264-269, 2001. [56] S. H. Collins, Dynamic walking principles applied to human gait. ProQuest, 2008. [57] J. M. Donelan, R. Kram, and A. D. Kuo, “Mechanical work for step-to-step transitions is a major determinant of the metabolic cost of human walking,” Journal of Experimental Biology, vol. 205(23), pp. 3717-3727, 2002. [58] P. G. Adamczyk, S. H. Collins, and A. D. Kuo. “The advantages of a rolling foot in human walking,” Journal of Experimental Biology, vol. 209(20) pp. 3953, 2006. [59] G. Taga, Y. Yamaguchi, H. Shimizu, “Self-Organized control of bipedal locomotion by neural oscillators in unpredictable Environment”, Biological Cybernetics, vol. 65, pp. 147–159, 1991. [60] K. Matsuoka, “Mechanisms of frequency and pattern control in the neural rhythm generators,” Biological Cybernetics, vol. 56, pp. 345-353, 1987.

125

[61] K. Hase, N. Yamazaki, “Computational evolution of human bipedal walking by a neruo-musculo-skeletal model”, Artificial Life and Robotics, vol. 3, no. 3, pp. 133-138, 1999. [62] N. Ogihara, N. Yamazaki, “Generation of human bipedal locomotion by a bio- mimetic neuro-musculo-skeletal model,” Biological Cybernetics, vol. 84, no. 1, pp. 1-11, 2001. [63] C. Paul, M. Bellotti, S. Jezernik, and A. Curt, “Development of a human neuromusculo-skeletal model for investigation of spinal cord injury,” Bio. Cybern., vol. 93, pp. 153-170, 2005. [64] T. Luksch, K. Berns, “Modeling and control of dynamically walking bipedal robots”, Cognitive Systems Monographs, vol. 18, pp. 131-143, 2013. [65] L. Righetti, and A. J. Ijspeert. “Programmable central pattern generators: an application to biped locomotion control,” in ICRA 2006 Proceedings 2006 IEEE International Conference on, pp. 1585-1590, 2006. [66] J. Morimoto, G. Endo, J. Nakanishi, and G. Cheng, “A biologically inspired bipded locomotion strategy for humanoid robots: modulation of sinusoidal patterns by a coupled oscillator model,” IEEE Trans. on Robotics, vol. 24, no.1, pp. 185-191, 2008. [67] S. Aoi, and K. Tsuchiya, “Locomotion control of a biped robot using nonlinear oscillators,” Autonomous Robots, vol. 19, pp. 219-232, 2005. [68] Theresa J. Klein, and M. Anthony Lewis, “A physical model of sensorimotor interactions during locomotion,” J. of Neu. Eng., vol. 9(4), pp. 1-14, 2012. [69] Purves, Dale, et al. Neuroscience. Sunderland, MA: Sinauer Associates 3, 2001. [70] T. Geng, B. Porr, and F. Worgotter, “Fast bipedal walking with a sensor-driven neuronal controller and real-time online learning,” The Inter. J. of Robotics Res., vol. 25, No. 3, pp. 243-259, 2006. [71] K. Yin, K. Loken, M. van de. Panne, “Simbicon: simple bipedal locomotion control,” ACM Transaction on graphics (TOG), vol. 26, no. 26, pp. 105, 2207 [72] H. Geyer, and H. Herr, “A muscle-reflex model that encodes principles of legged mechanics produces human walking dynamics and muscle activities,” IEEE Trans. on Neural Systems and Rehabil. Engineering, vol. 18, no. 3, 2010. [73] N. Van der Noot, A. J. Ijspeert, and R. Ronsse, "Biped gait controller for large speed variations, combining reflexes and a central pattern generator in a neuromuscular model." In ICRA 2015 IEEE International Conference on, pp. 6267-6274, 2015. [74] H. J. Chiel, R. D. Beer, “The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment,” Trends Neurosci., vol. 20, pp. 553-557, 1997. [75] M. H. Dickinson, C. T. Farley, R. J. Full, M. A. Koehl, R. Kram, and S. Lehman, “How animals move: an integrative view,” Science, vol. 288, pp. 100-106, 2008.

126

[76] Ö. Ekeberg, M. Blumel, and A. Büschges, “A combined neuronal and mechanical model of fish swimming,” Biol. Cybern., vol. 69, pp. 273-374, 1993. [77] G. E. Loeb, I. E. Brown, E. J. Cheng, “A hierarchical foundation for models of sensorimotor control,” Exp. Brain Res., vol. 126, pp. 1-18, 1999. [78] K. Pearson, Ö. Ekeberg, A. Büschges, “Assessing sensory function in locomotor systems using neuro-mechanical simulations,” Trends in Neurosciences, vol. 29, pp. 625-630, 2006. [79] B. Webb, “Can robots make good models of biological behavior?” Behavioral and Brain Sciences, vol. 24, pp. 1033-1050, 2001. [80] T. Buschmann, A. Ewald, A. Twickel, and A. Buschges, “Controlling legs for locomotion-insights from robotics and neurobilogy”, Bioinspir. Biomim., vol. 10, 041001, 2015 [81] D. Cofer, G. Cymbalyuk, J. Reid, Y. Zhu, and et al., “Animatlab: A 3D graphics environment for neuromechanical simulations,” Journal of Neuroscience Methods, vol. 187, pp. 280-288, 2010. [82] M. L. Hines, N. T. Carnevale, “NEURON: a tool for neuroscientists,” Neuroscientist, vol. 7, pp. 123-135, 2001 [83] J. Bower, D. Beerman, “Constructing realistic neural simulations with GENESIS,” Methods Mol. Bio., vol. 401, pp. 103-125, 2007. [84] S. L. Delp, F.C. Anderson and A.S. Arnold, et al. “OpenSim: Open-source software to create and analyze dynamic simulations of movement,” IEEE Trans. Biomed. Eng., vol. 54, pp. 1940-50, 2007. [85] Destexhe, “Conductance-based integrate-and-fire models,” Neural Computation, vol. 9, pp. 503-514, 1997. [86] R. MacGregor, E. Lewis, Neural modeling: electrical signal processing in the nervous system. New York: Plenum Press; 1977. [87] D. Cofer, G. Cymbalyuk, W. J. Heitler, and D. H. Edwards, “Neuromechanical simulation of the locust jump,” J. Exp. Biol, vol. 213(7), pp. 1060-1068, 2010. [88] M. D. Rinehart, J. H. Belanger, “Biologically realistic limb coordination during multi-legged walking in the absence of central connections between legs,” in Society for Neuroscience Annual Meeting, 2009. [89] N. S. Szczecinsk, A. E. Brown, J. A. Bender, R. Q. Quinn, R. E. Ritzmann, “A neuromechanical simulation of insect walking and transition to turning of the cockroach Blaberus discoidalis,” Biological Cybernetics, vol. 108(1), pp. 1-21, 2014. [90] Hunt, M. Schmidt, M. Fischer, and R. Quinn, “A biologically based neural system coordinates the joints and legs of a tetrapod,” Bioinspir. Biomim., vol. 10, pp. 1-10, 2015. [91] M. Fischer, and H. Witte, “Legs evolved only at the end,” Phil. Trans. R. Soc., vol. A365. pp. 185-198, 2006.

127

[92] M. Fischer, and H. Witte, “Legs evolved only at the end,” Phil. Trans. R. Soc., vol. A365, pp. 185-198, 2006. [93] S. Plagenhoef, F. G. Evans, and T. Abdelnour, “Anatomical data for analyzing human motion,” Research Quarterly for Exercise and Sport, vol. 54, pp. 169-178, 1983. [94] Shadmehr, R., and S. Wise, A mathematical muscle model. Core course on Neuroscience, JHU School of Medicine, Supplementary materials for muscle models, 2003. [95] V. Hill, First and last experiments in muscle mechanics. Cambridge: Cambridge University Press, 1970. [96] N. Burkitt, “A review of the integrate and fire neuron model: I. Homogeneous synaptic input”, Biol. Cybern., vol. 95, pp. 1-19, 2006. [97] R. S. Zucker, W. G. Regehr, “Short-term synaptic plasticity,” Annual Review of Physiology, vol. 64(1), pp. 355-405, 2002. [98] S. Arnold, D. G. Thelen, M. H. Schwartz, F. C. Anderson and S. L. Delp, “Muscular coordination of knee motion during the terminal-swing phase of normal gait,” Journal of Biomechanics, vol. 40, pp. 3314-3324, 2007. [99] P.H. Raven, et al., Biology (10th edition). New York: McGraw-Hill Education Press, 2014. [100] F. Baldissera, H. Hultborn, and M. Illert. “Integration in spinal neuronal systems,” Comprehensive Physiology, 1981. [101] M. P. Kadaba, H. K. Ramakrishnan, and M. E. Wootten, “Measurement of lower extremity kinematics during level walking,” J. of Orthopedic Res., vol. 8, 383-392, 1990. [102] S. Kim, J. Son, B. Jung, and Y. Kim, “Metatarsophalangeal Joint Kinetics in Normal Walking,” in World Congress on Medical Physics and Biomedical Engineering 2012, pp. 136-138, 2013. [103] R. Umberger, P. E. Martin, “Mechanical power and efficiency of level walking with different stride rates,” Journal of Experimental Biology, vol. 210, pp. 3255- 3265, 2007. [104] A. Winter, “Kinematic and kinetic patterns in human gait: variability and compensating effects,” Human Movement Science, vol. 3, pp. 51-76, 1984. [105] K. E. Zelik, and A. D. Kuo. “Human walking isn't all hard work: evidence of soft tissue contributions to energy dissipation and return,” Journal of Experimental Biology, vol. 213, no. 24, pp. 4257-4264, 2010. [106] J. Sun, M. Walters, N. Svensson, and D. Lloyd, “The influence of surface slope on human gait characteristics: a study of urban pedestrians walking on an inclined surface,” Ergonomics, vol. 39(4), pp. 677-692, 1996 [107] B. I. Prilutsky, and V. M. Zatsiorsky, “Optimization-based models of muscle coordination,” Exerc. Sport Sci. Rev., vol. 30(1), pp. 32-38, 2002.

128

[108] M. Izhikevich, “Which model to use for cortical spiking neurons?” IEEE Trans. on Neural Networks, vol. 15, no. 5, pp. 1063-1069, 2004. [109] J. B. Nielsen, “How we walk: central control of muscle activity during human walking,” The Neuroscientist, vol. 9, No. 3, pp. 195-204, 2003. [110] T. D. Welch, and L. H. Ting, “A feedback model reproduces muscle activity during human postural responses to support-surface translations”, J. Neurophysiol, vol. 99, pp. 1032-1038, 2008. [111] C. E. Bauby, A. D. Kuo, “Active control of lateral balance in human walking,” J. of Biomechanics, vol. 33, pp. 1433-1440, 2000.

129