<<

The critical phase for visual control of human walking over complex terrain

Jonathan Samir Matthisa,1, Sean L. Bartonb, and Brett R. Fajenb

aCenter for Perceptual Systems, University of Texas at Austin, Austin, TX 78712; and bCognitive Science Department, Rensselaer Polytechnic Institute, Troy, NY 12180

Edited by Michael E. Goldberg, Columbia University College of Physicians, New York, NY, and approved June 21, 2017 (received for review July 20, 2016) To walk efficiently over complex terrain, humans must use vision to down again until the swinging foot contacts the ground, which is tailor their gait to the upcoming ground surface without interfering called “heel strike” (Fig. 1A). As the COM increases its height during with the exploitation of passive mechanical . We propose that the first half of the single support phase, some of the of walkers use visual information to initialize the mechanical state of the COM is transferred into , which is then trans- the body before the beginning of each step so the resulting ballistic ferred back into kinetic energy as the COM drops down to its original trajectory of the walker’s center-of- will facilitate stepping on height in the latter half of the step. In the ideal case, the exchange target footholds. Using a precision stepping task and synchronizing between kinetic and potential energy is perfectly lossless and sym- target visibility to the gait cycle, we empirically validated two predic- metric, and human walking closely approximates an idealized inver- tions derived from this strategy: (1) Walkers must have information ted acting conservatively. As such, humans can harness about upcoming footholds during the second half of the preceding and inertia and exploit the dynamics of step, and (2) foot placement is guided by information about the po- their bodies to traverse the distance traveled during the single support sition of the target foothold relative to the preceding base of support. phase at a minimal cost in terms of and muscle (6, 7). We conclude that active and passive modes of control work synergis- Of course, in reality walking does incur costs. A de- tically to allow walkers to negotiate complex terrain with efficiency, terminant of the metabolic cost of walking is restoration of the stability, and precision. energy lost when the swinging foot contacts the ground (refs. 8 and 9, but see ref. 10). Ground contact by the swinging foot human locomotion | visual control | foot placement | biomechanics | marks the end of the single-support phase and the beginning of inverted pendulum the double-support phase, during which the COM must be redirected from a downward trajectory to the upward trajectory umans and other animals are remarkable in their ability to it will need for the coming step (11). Upon ground contact, the Htake advantage of what is freely available in the environment to force applied by the leading leg has a horizontal component the benefit of efficiency, stability, and coordination in movement. opposite to the direction of motion of the COM; that is, the This opportunism can take on at least two forms, both of which are leading leg performs negative work on the COM. This collision is evident in human locomotion over complex terrain: (i)harnessing external forces to minimize the need for self-generated (i.e., mus- Significance cular) forces (1), and (ii) taking advantage of passive stability to simplify the control of a complex movement (e.g., ref. 2). In the The physical dynamics of the body are central to the generation ensuing section, we explain how walkers exploit external forces and and maintenance of the human gait cycle. The ability to exploit passive stability while walking over flat, obstacle-free terrain.* We the force of gravity and bodily inertia increases the energetic ef- then generalize this account to walking over irregular surfaces by ficiency of locomotion by minimizing the need for internally explaining how walkers can adapt gait to terrain variations while still generated muscular forces and simplifies control by obviating the reaping the benefits of the available mechanical forces and inherent need to actively guide each body segment. Here we explore how stability. This account leads to hypotheses about how and when these principles generalize to situations in which foot placement is walkers use visual information about the upcoming terrain and constrained, as when walking over a rocky trail. Walkers can ex- where that information is found. We derive several predictions from ploit external forces to efficiently traverse extended stretches of these hypotheses and then put them to the test in three experiments. complex terrain provided that visual information about the up- coming ground surface is available during a particular (critical) Passive Control in Human Walking phase of the gait cycle between midstance of the preceding step The basic movement pattern of the human gait cycle arises pri- and toe-off. marily from the phasic activation of flexor and extensor muscle groups by spinal-level central pattern generators, regulated by sen- Author contributions: J.S.M., S.L.B., and B.R.F. designed research; J.S.M. and S.L.B. per- sory signals from lower limb proprioceptors and cutaneous feedback formed research; J.S.M., S.L.B., and B.R.F. analyzed data; and J.S.M., S.L.B., and B.R.F. from the plantar surface of the foot. This low-level neuromuscular wrote the paper. circuitry serves to maintain the rhythmic physical oscillations that The authors declare no conflict of interest. define locomotor behavior (see ref. 3 for review). This section will This article is a PNAS Direct Submission. provide an overview of the basic biomechanics of the bipedal gait Freely available online through the PNAS open access option. cycle to show how these inherent physical dynamics contribute to 1To whom correspondence should be addressed. Email: [email protected]. the passive stability and energetic efficiency of human locomotion. This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. During the single support phase of the bipedal gait cycle, when 1073/pnas.1611699114/-/DCSupplemental. only one foot is in contact with the ground, a walker shares the *The forthcoming description of the human gait cycle is an adaptation of the “simplest ” physical dynamics of an inverted pendulum. The body’s center of walking model, which is an attempt to develop the most parsimonious description of bipedal walking that still captures key characteristics of human locomotion (13, 71, 72). mass (COM) acts as the bob of the pendulum and is supported Although more complete musculoskeletal models can be invaluable for determining the above a single point of in the planted foot (4, 5). At the contributions of individual muscles to locomotor behavior (63), such models are compu- onset of the single support phase—that is, at “toe off,” when the tationally expensive and conceptually difficult to analyze. By abstracting the highly — complex action of human locomotion down to the simplest possible biomechanical nonsupporting leg breaks contact with ground car- model, it is possible to explore how the control of human locomotion is organized ries the COM up to a maximum height at midstance and then around the underlying physical dynamics of bipedal gait (16, 73–75).

E6720–E6729 | PNAS | Published online July 24, 2017 www.pnas.org/cgi/doi/10.1073/pnas.1611699114 Downloaded by guest on September 27, 2021 AB Cthat is injected into the system by the will be dis- PNAS PLUS Midstance sipated at collision (15). Although the basin of attraction for bipedal gait may be relatively shallow, spinal-level reflex path-

t t r r ways (3) may serve to facilitate gait-restorative responses that o o † p p 4 – p p take advantage of this passive stability (18 20).

u u 4 2

3 S

S

e

e

l

l

g

b

n

u Active Control of Walking

i o

S 3 D Swing These insights imply that certain desirable characteristics of move- ment, such as coordination, efficiency, and stability, can emerge Le without active (i.e., perceptually guided) control. Nevertheless, hu- g 2

Stance Leg Stance mans are not locked into rigid patterns of movement. The spinal and

Heel Strike + Off 1 subcortical activity underlying the basic rhythmic pattern of the pre- Toe ferred gait cycle that humans adopt during steady-state walking over flat terrain can be modulated by descending signals from the cortex to 1 step length 1 step length accommodate variations in task demands and variations in the en- D Control Control Control Control vironment (3, 21, 22). Such descending signals are essential when Phase for Phase for Phase for Phase for walking over complex terrain, where it may not be an option to place Target 3 Target 4 Target 5 Target 6 the feet in the that is ideal for the exploitation of the body’s inverted pendulum dynamics. Instead, walkers must identify potential footholds in the upcoming terrain and regulate the movement of their body to ensure that the foot lands on a suitable location. Recent research on the neural underpinnings of visually guided walking in cats suggests that area 5 of the posterior parietal cortex (PPC) may play an essential role in integrating visual signals into the gait cycle to facilitate the control of foot placement during lo- 1 2 3 4 5 6 comotion over complex terrain (21, 23). This area has been impli- cated in the control of reaching to remembered locations in Fig. 1. (A) A conceptual diagram of the steady-state gait cycle. In the step- to-step transition, positive and negative work from the trailing and leading primates (24, 25) and may play an analogous role in stepping be- legs (red and blue arrows, respectively) redirect the COM and establish the havior during locomotion. In vivo recordings from cats walking on a conditions leading into the next step. (B) The ballistic trajectory of the COM treadmill revealed that neurons from this region increase their firing during a step is defined by two factors—the location of the planted foot and rates in the 2–3 steps before the step over an obstacle (22), sug- the magnitude of the push-off force from the trailing limb. (C) By adjusting gesting that this area could be involved in encoding the position of these factors, a walker can tailor the passive trajectory of the COM in an upcoming obstacles relative to the moving observer. This estimate oncoming step to facilitate a step onto a target foothold. (D) The critical may then be used by the motor cortex to generate the descending – control phase for targets 3 6 (see also Movie S1). signals that modulate the activity of synergistic muscle groups in the COGNITIVE SCIENCES

limbs to enact the gait adjustment necessary to account for the PSYCHOLOGICAL AND dissipative, which means that some mechanical energy is lost and upcoming obstacle (or target foothold) (23). that positive work must be performed to maintain forward progress When walking over complex terrain, visual information about (12). To counteract this loss of energy, the trailing leg applies a push- the upcoming path must be integrated into the ongoing gait cycle off force beginning toward the end of the single support phase and to adjust the basic locomotor pattern to fit the available footholds. continuing through the double-support phase (13, 14). The vertical Although visual information about the world is often noisy, hu- component of the push-off force from the trailing leg sums with the mans appear to use state estimation to derive accurate estimates vertical component of the force from the leading leg to direct the of target footholds to support precise foot placement (26). This COM into the upward trajectory that it will need for the next step visual information does not need to be continuously available; (Fig. 1A). If the forward horizontal component of the push-off force human walkers are able to walk successfully over constrained is equal to the backward horizontal component of the force from the terrain with only intermittent visual information about the up- – leading foot, the two forces cancel out and the mechanical state of coming path (27 29). The ability to step successfully on a target the body will be the same at the onset in the upcoming step as it that is not continuously visible is a hallmark of the feedforward was in the previous one. Both the negative work performed by the control that underlies visually guided locomotion. leading leg and the positive work performed by the trailing leg The feedforward plans necessary to traverse an obstacle or involve muscle activation and therefore incur some metabolic cost. other locomotor impediment must be enacted based on visual However, healthy human walkers adopt strategies that reduce the information gathered during uninterrupted locomotion. When cost of the step-to-step transition to the degree possible (8, 15). subjects’ vision was occluded several steps before traversing an Thus, by exploiting the inverted pendulum dynamics of the single- obstacle, they performed successfully provided that the occlusion support phase and using the push-off force to restore the energy occurred during uninterrupted locomotion (30). However, if sub- lost during the step-to-step transition, humans are able to generate jects stopped walking after their vision was occluded, performance a steady-state gait cycle with minimal energetic cost (9, 12, 16). degraded and collisions with the obstacle became more frequent. The second form of opportunism by the motor system during The visual information necessary to cross complex terrain may be walking involves reliance on control strategies that take advan- tage of passive stability. When such strategies are adopted, small perturbations are automatically dissipated over subsequent steps, † making it possible for stable walking behavior to emerge with no Passive stability has its limits. Larger perturbations will result in shorter steps, which active, sensory-driven control (17). In human walking, passive dissipate less energy and require active compensation to avoid falling (15). Furthermore, stability arises because the loss of energy at collision increases walkers are passively stable along the axis of forward progress but not along the medial- with walking speed (13). As such, a small forward perturbation lateral axis (76). Active control is needed to stabilize the body in response to lateral perturbations. Such responses rely on sensory (e.g., visual, vestibular) information, in- applied to the COM at midstep increases the speed of the COM, volve adjustments to foot placement that affect step width, and exact some metabolic which in turn increases the collision loss so the additional energy cost (77, 78).

Matthis et al. PNAS | Published online July 24, 2017 | E6721 Downloaded by guest on September 27, 2021 sampled by briefly fixating obstacles and targets approximately two also takes full advantage of the passive dynamics inherent to step lengths in advance (31–33) or by relying on peripheral vision bipedal walking. (34, 35). When terrain variations can be detected far in advance, visually guided adjustments to gait parameters are observed as early The Critical Control Phase Hypothesis as three to four steps ahead and spread out over the remaining steps The idea that the control of foot placement over complex terrain (36). Occasionally, a more rapid response is needed—when the lo- serves to maintain an energetically efficient exploitation of the cation of an intended target changes midstep or an obstacle suddenly body’s physical dynamics motivates a hypothesis about when vi- appears or changes in size, humans are capable of redirecting the sual information about the upcoming terrain is most critical. This trajectory of their feet with short latency (∼120 ms) (37–40). These hypothesis has two components and will be referred to as the studies highlight some of the various roles of active, vision-based critical control phase hypothesis. control in the guidance of walking over complex terrain. The first component concerns how far in advance walkers must be able to sample visual information about the upcoming terrain. A How Are Passive and Active Modes of Control Used During visual control strategy based on manipulation of the ballistic tra- Walking over Complex Terrain? jectory of the COM during the single support phase would require The imperative to adapt gait to the layout of obstacles and safe walkers to see the terrain far enough in advance to be able to make footholds entails neuromuscular intervention that interferes with the appropriate adjustments to the two determinants of that tra- the natural passive trajectory of the body. During locomotion over jectory. That is, tailoring the mechanical state of the body appro- uneven terrain, steps become more variable in both length and width, priately to allow a walker to step on a target foothold (say, with the mechanical work increases at the hips and knees, and muscle acti- right foot) requires the walker to see that target before the me- vation increases in several proximal leg muscles, all of which lead to chanics of the relevant step have been defined. Given the inverted an increase in energy expenditure (41, 42). Does this mean that pendulum dynamics of the single support phase, this means the walkers must give up entirely on exploiting external forces and passive walker must see the right foothold before the left foot, which de- stability when walking over complex terrain, or is it possible to si- fines the base of the upcoming inverted pendulum, has contacted multaneously use information about the upcoming terrain to modu- the ground and before the push-off force from the supporting leg late gait while still taking advantage of the benefits of passive control? has been enacted. As such, the walker must see the terrain relevant We propose that the latter could be achieved by using in- to a right foothold while the right foot is still on the ground (and formation about upcoming terrain to initialize the mechanical vice versa for the left foot). Put another way, to tailor the state of the body before the beginning of each step, such that the of the body to enable stepping to a specific foothold requires a resulting ballistic trajectory of the body will facilitate stepping on walker to see the terrain relevant to that foothold by around mid- the upcoming target footholds. In this way, a walker can make small stance of the previous single-support phase—roughly two step ‡ adaptations to their energetically optimized preferred gait cycle to lengths of visual look ahead (Fig. 1D). accommodate upcoming environmental impediments. To explain Indeed, restricting a walker’s ability to see upcoming terrain to how such a strategy would work, we first identify the mechanical less than two step lengths results in decreases in stepping accuracy determinants of the passive trajectory of the body during the single and walking speed (43). Furthermore, the trajectory of the COM support phase that could serve as control variables for the walker. is less like that of a passively moving inverted pendulum compared If we assume that the single support phase is adequately modeled with when walkers can see two or more step lengths ahead, sug- by a simple inverted pendulum, the ballistic trajectory of the walker’s gesting that walkers need at least two step lengths of visual look- COM during the single support phase depends on two factors—the ahead to exploit their inverted pendulum dynamics when walking position of the planted foot and the initial conditions (position and over complex terrain (44). Interestingly, these findings converge velocity) of the COM at toe off (Fig. 1B). During steady-state walking with studies of gaze behavior that demonstrate that walkers tend over flat, obstacle-free terrain, the push-off force restores the energy that is lost during step-to-step transition, resulting in a regular, to look about two steps ahead when walking over complex terrain roughly sinusoidal path of the COM between each foothold (Fig. 1C, (32). In addition, Zaytsev et al. (45) recently showed that a dy- black line). However, a walker seekingtostepontoaparticular namic bipedal walking model with realistic actuators could reach foothold could also purposefully alter the direction and magnitude of almost any desired target velocity or foothold location by using a that push off force to cause the COM to follow a different trajectory two-step-ahead planning window. These results suggest that the tendency of human walkers to look approximately two step lengths during the subsequent single support phase. The walker could – achieve the same effect by maintaining a normal push-off force and ahead during locomotion over complex terrain (31 33) may be altering the location of the planted foot that defines the base of the rooted in the physical dynamics of the bipedal gait cycle. pendulum in the coming step (Fig. 1C, green and orange lines). The second part of the critical control phase hypothesis con- To summarize, the two determinants of the passive COM tra- cerns when visual information about upcoming terrain is no jectory (the position of the planted foot and the push-off force longer needed. Do walkers require visual information about the from the trailing foot) are potential control variables that walkers location of an upcoming target foothold throughout the entire may use to initialize the mechanical state of the body before toe- step, or can they achieve comparable levels of stepping accuracy off to tailor the ballistic trajectory of the COM to the upcoming if they stop sampling information partway through the step or terrain. By making small adjustments to the placement of a step or even before the step is initiated? If, in fact, walkers attempt to by adjusting the restorative push-off force to be either more or less initialize each step so that the body can follow a ballistic trajectory than the magnitude required to exactly recoup the energy loss of to the target, then the role of visual information about a target heel strike, the walker can alter the parameters of the steady-state, is primarily to allow the walker to tailor the two determinants of preferred gait cycle to accommodate upcoming terrain. These adjustments to the basic pattern of muscle activation underlying the gait cycle may arise from descending cortical signals that ‡ support visually guided locomotion (21). A step length is defined as the distance between two subsequent footfalls (see Fig. 1A) Once the step is initiated, the body can simply be allowed to and is distinct from a step, which refers to the action of lifting the foot and moving it to a follow its natural pendular trajectory to the next target with no new location. The term visual look ahead refers to a linear distance from the vertical ’ need for energetically costly midflight corrections that interfere projection of the walker s head onto the ground plane. As shown in Fig. 1A, a visual look ahead of two step lengths would allow a walker to see a target foothold when he or she with that trajectory. In this regard, this strategy relies on active is halfway through the step to the preceding target (i.e., 1 1/2 steps in advance). See the control based on visual information about upcoming terrain but Materials and Methods for additional discussion.

E6722 | www.pnas.org/cgi/doi/10.1073/pnas.1611699114 Matthis et al. Downloaded by guest on September 27, 2021 the ballistic trajectory. Importantly, this strategy relies on feed- of Fig. 2), each target first appeared when the subject was in PNAS PLUS forward control—walkers use information about the upcoming midstance of the step to the preceding target and disappeared at path to establish the mechanical state of the body before the toe off of the step to the target (that is, it was visible from 1 1/2 to upcoming step so that the natural dynamics of their body will 1 steps before the target, see also Materials and Methods). The carry them in the desired direction (Fig. 1D). Any active control colored horizontal bars in Fig. 2 indicate the location of the of the trajectory of the swinging leg during the single-support walker’s COM when the target with the corresponding color was phase would require the walker to use muscular forces to in- visible (e.g., the third target, which is depicted in red, was visible terfere with the ballistic trajectory of the body. As such, we should when the subject was within the range of positions indicated by the expect that visual information about the target should not be red horizontal bar). To help visualize the timing of target visibility necessary once the step to that target is initiated. relative to the critical control phase for each target, Fig. 2 also Indeed, subjects walking across a path of small target foot- includes colored vertical bands showing the critical control phase holds showed a negligible decrease in stepping accuracy when for each target. Notice that, in this condition, the horizontal bars the targets became invisible at toe-off of the step to the target corresponding to target visibility are aligned with the vertical (46). However, stepping accuracy decreased dramatically if the bands indicating the critical control phase; that is, each target was target became invisible at any point before toe-off. In short, it visible during the critical control phase and only at that time. As appears that once the foot has left the ground toward a target such, we call this the “Just in Time” condition. As shown in Fig. 2, foothold, no further visual information about that target is re- Right, the mean absolute stepping error in this condition was quired for accurate foot placement. This result entails that any 43.4 mm (95% CI [40.5, 46.3]). vision-based control of the trajectory of the foot during a step Next, let us compare performance in the Just in Time condi- must take place during the preceding step. tion to performance in the three other conditions in which tar- Taken together, these results suggest that there is a critical gets were visible for twice as long (one full step) during, before, or phase within the bipedal gait cycle for the visual control of foot after the critical control phase. If the last half of the preceding step placement when walking over complex terrain. When guiding the is truly the critical phase of the gait cycle during which visual in- placement of a particular step, visual information about the formation must be available, then extending the duration of visi- terrain relevant to that foothold becomes maximally important bility to include the previous 1/2 step amounts to making targets following midstance of the preceding single-support phase but is visible needlessly early and should not improve performance. In- “ ” no longer necessary once the foot leaves the ground on its way to deed, stepping accuracy in the Needlessly Early condition was = – = = the selected foothold (Fig. 1D and Movie S1). not significantly different (t11 0.24, P 0.81, d 0.07; mean difference relative to Just in Time condition = –0.25, 95% CI Results [–2.51, 2.01]). When targets were visible for a full step before the “ ” Testing the Critical Phase Hypothesis (Experiment 1). Although the critical control phase ( Too Early condition), stepping error was = < = findings of Matthis and Fajen (43, 44) and Matthis, Barton, and significantly worse (t11 3.42, P 0.01, d 0.99; mean difference relative to Just in Time condition = 8.78, 95% CI [3.13, 14.43]). In Fajen (46) are consistent with the critical control phase hypothesis, “ ” they are not sufficient to conclude that visual information is primarily the Too Late condition, each target first appeared at toe-off of used during the latter part of the preceding step because the two the step to that target and remained visible. Stepping error was not significantly worse in this condition (t11 = 0.92, P = 0.38, d = 0.27; COGNITIVE SCIENCES

components of the critical phase hypothesis were tested in separate PSYCHOLOGICAL AND mean difference relative to Just in Time condition = 2.92, 95% CI experiments. The first aim of this study was to provide a direct test by – combining the manipulations used in the previous studies in a single [ 4.03, 9.87]). However, subjects walked slower in the Too Late condition than they did in each of the other conditions (see Fig. experimental task. Subjects were instructed to walk over a series of ∼ small targets that did not appear until they fell within a threshold S1). Although the decrease in walking speed is small ( 0.1 m/s), paired-sample t tests comparing each of the three other conditions distance of the subject and then disappeared when they fell within a to the Too Late condition confirmed that these differences were second threshold distance. This manipulation was applied to each statistically significant (t = 5.33, P < 0.01, d = 1.54 for Just in target individually, with the exception of the first two, which were 11 Time vs. Too Late; t = 5.20, P < 0.01, d = 1.50 for Needlessly visible throughout the entire trial (stepping accuracy to the first two 11 Early vs. Too Late; t = 6.04, P < 0.01, d = 1.74 for Too Early vs. targets was not included in later analysis). 11 Too Late). Thus, subjects were able to maintain stepping accuracy We tested four conditions with different triggers for target vis- in this condition but only by trading off on walking speed. This ibility and invisibility (Fig. 2; see Materials and Methods for details trade-off is consistent with behavior observed while attempting to and Movie S2). In one of the conditions (depicted in the first row avoid stepping on obstacles under analogous visibility constraints (43, 44). Taken together, it is apparent that the best overall performance CP-3 CP-4 CP-5 CP-6 Experiment 1 was found in the two conditions for which targets were visible during the critical control phase. In one of those conditions (i.e., the Just in Time condition), targets were visible for only half a Mean Absolute Stepping Error ± 95% CI (mm) step, supporting the idea that the usefulness of visual information 40 45 50 55 123 4 5 6 is determined more by the phasic timing of its availability within Just In Time the gait cycle than by the duration of target visibility. (1.5 - 1.0) Needlessly Early (2.0 - 1.0) The Two Targets Hypothesis. The findings of experiment 1 demon- Too Early ’ (2.5 - 1.5) strate that the objective to exploit one s biomechanical structure Too Late during walking constrains when visual information about the up- (1.0 - 0) coming terrain is needed. Next, we consider whether biomechani- cal factors similarly constrain where the information that is needed Fig. 2. (Left) Depiction of four target visibility manipulations used in ex- to guide foot placement is found; that is, which regions of the periment 1 (see Movie S2). Horizontal bars indicate the location of the walker’s COM when the target of the corresponding color was visible. Ver- upcoming terrain must be visible to the walker. Again, the pro- tical bands indicate hypothesized critical control phase for each target. posal that walkers attempt to exploit their biomechanical structure (Right) Mean absolute stepping error and 95% CI for each condition in ex- by initializing the upcoming step leads to a testable hypothesis. To periment 1. avoid having to make midflight adjustments, the push-off force

Matthis et al. PNAS | Published online July 24, 2017 | E6723 Downloaded by guest on September 27, 2021 from the trailing foot should be adjusted to the position of the target turned off; that is, target visibility was “contiguous” in time target at the end of the upcoming step relative to the position of (second row of Fig. 3, second animation of Movie S3). the previous target (8). For example, when the distance between The analyses in this experiment focused on stepping error relative the two upcoming targets is short, a weak push-off force would be to a full-vision control condition in which all six targets remained sufficient; a longer separation between upcoming targets would re- visible for the entire trial. If the two targets hypothesis is correct, quire a stronger push-off force. The point is that a walker attempting stepping error should be at full-vision level in the Overlap condition to initialize the upcoming step with an appropriate push-off force because the two upcoming targets are simultaneously visible and needs to know the position (both distance and direction) of higher in the Gap condition in which the targets were not simulta- target N relative to the preceding foothold (target N–1). That is, neously visible. The results were consistent with these predictions target N–1 serves as a visual indication of the origin of the (Fig. 3, Right). Stepping error was at baseline levels when the two = – = reference frame within which the location of target N must be upcoming targets were simultaneously visible (t17 0.4, P 0.69, perceived. This leads to the hypothesis that the relevant visual in- d = 0.09; mean difference relative to full vision condition = –0.41, formation is found in the region of the upcoming terrain that in- 95% CI [–2.55, 1.74]), allowing for their relative position to be ac- cludes the two upcoming targets. We refer to this as the two targets curately perceived. When there was a gap in time in the visibility of hypothesis and derive from it the following prediction—that foot upcoming targets (i.e., when relative target position was more diffi- placement should be less accurate when the relative position of the cult to accurately estimate), stepping error was significantly higher = < = two upcoming targets is perceived less accurately. (t17 3.35, P 0.01, d 0.79; mean difference relative to full vision condition = 3.78, 95% CI [1.40, 6.17]). As one would expect, per- Testing the Two Targets Hypothesis (Experiment 2A). Our approach formance was in between in the condition in which target visibility to testing the two targets hypothesis was to vary whether the two was contiguous in time (t17 = 1.79, P = 0.09, d = 0.42; mean dif- upcoming targets were simultaneously visible (see Movie S3 for ference relative to full vision condition = 2.15, 95% CI [–0.38, 4.68]). animations depicting the conditions used in experiment 2A). If the two targets are visible at different points in time, their rel- Ruling out the Influence of Duration of Target Visibility (Experiments ative position should be perceived less accurately, which should 2B). Although the results are consistent with the two targets hy- translate into less accurate foot placement. In one of the con- pothesis, there is an alternative explanation for the aforementioned ditions, depicted in the first row of Fig. 3 and the first animation findings that needs to be considered. To make pairs of consecutive in Movie S3, each of the manipulated targets was visible only targets simultaneously visible, it was necessary to make each target when the subject was between roughly two steps and 1/2 a step visible for a longer duration. As such, the results of experiment 2A from that target. The first two targets were always visible as in do not rule out the possibility that subjects were more accurate in experiment 1. Recalling that the horizontal bars indicate the the Overlap condition simply because targets were visible for more location of the walker’s COM when each target was visible, it is time. To test this possibility, experiment 2 also included another set apparent that each target appeared before the previous target of conditions (experiment 2B) in which the same visibility manip- turned off. As such, there was a brief period, lasting about 1/2 a ulations were applied to just one of the targets rather than to the step, during which the two upcoming targets were simultaneously last four (Fig. 3). In other words, five of the six targets were visible visible. We refer to this as the Overlap condition, as there was an throughout the entire trial, and one (either the fourth or fifth overlap in time in the visibility of the two upcoming targets. In target) was initially off, briefly appeared, and then disappeared (see the Gap condition, depicted in the third row of Fig. 3 and the Movie S4). The point of manipulating the visibility of just one third animation in Movie S3, each of the manipulated targets was target is that the duration of target visibility and the overlap in visible when the subject was between roughly 1 1/2 steps and target visibility do not covary as they do when manipulating the 1 step. The two upcoming targets were not simultaneously visible visibility of two or more consecutive targets (i.e., as in experiment in this condition; rather, there was a gap in time in the visibility 2A). Because the target before the manipulated target is always of the two upcoming targets. There was also a condition in which visible, there is overlap in target visibility in all three conditions, each target turned on around the same time that the previous regardless of whether the manipulated target is visible for 1 1/2, 1, or 1/2 a step. This is evident in the bottom of Fig. 3, which illustrates that when target 5 (the manipulated target, blue bars) is visible, target 4 (magenta bars) is also visible. As such, the difference in CP-3 CP-4 CP-5 CP-6 Experiment 2 stepping error across conditions that was found in experiment 2A when four targets were manipulated should be erased. Mean Absolute Stepping Error Relative to Full Vision Condition The results were consistent with this prediction. As shown in ±95% CI (mm) Fig. 3, stepping error for the manipulated target did not signif- -4 -2 0 2 4 6 8 123 4 5 6 icantly differ across conditions and was not significantly different Overlap = = = (1.9-0.5) from full vision in any condition (t17 0.18, P 0.86, d 0.04 for Contiguous the 1.9–0.5 condition; t17 = 1.20, P = 0.25, d = 0.28 for the 1.9– (1.9-0.9) 0.9 condition; t = 1.62, P = 0.12, d = 0.38 for the 1.5–0.9 con- Gap 17 (1.5-0.9) Experiment 2A

(4 active targets) dition). This allows us to rule out the possibility that subjects were more accurate in the Overlap condition of experiment 2A Overlap (1.9-0.5) simply because targets were visible for more time and bolsters Overlap the conclusion that walkers rely on information about the rela- (1.9-0.9) tive position of the two upcoming targets to modulate the push- Overlap (1.5-0.9)

(1 active target) off force from the trailing foot. Experiment 2B

Fig. 3. (Left) Depiction of target visibility manipulations used in experiments Minimal Visibility Conditions (Experiment 3). Taken together, the 2A (Top)and2B(Bottom). See Movies S3 and S4. Horizontal bars indicate the ’ critical control phase hypothesis and the two targets hypothesis location of the walker s COM when the target of the corresponding color was delineate the minimal visibility conditions that must be satisfied visible. Vertical bands indicate hypothesized critical control phase for each target. The same three visibility manipulations were used in both experiments to attain the same level of foot placement accuracy that is pos- but were applied to targets 3–6 in experiment 2A and to targets 4 or 5 in sible when vision is unrestricted: Visual information about the experiment 2B. (Right) Mean absolute stepping error relative to full vision location of a given target relative to the preceding target must be control condition and 95% CI for each condition in experiments 2A and 2B. available during the last half of the preceding step. When the

E6724 | www.pnas.org/cgi/doi/10.1073/pnas.1611699114 Matthis et al. Downloaded by guest on September 27, 2021 necessary and sufficient conditions are not satisfied, foot place- target foothold and are based on visual information about the po- PNAS PLUS ment accuracy should degrade. Making targets visible beyond sition of the intended target relative to the location where the foot these minimal conditions, including the full-vision condition, will be planted during the step to that target. Importantly, this is not should not result in further reductions in stepping error. merely a descriptive account of when visual information is needed These predictions were tested in experiment 3 by manipulating and where that information is found. Both of these statements the visibility of a single target (as in experiment 2B) across six derive from the hypothesis that walkers attempt to exploit their different conditions that varied in the timing and duration of biomechanical structure when walking over complex terrain, just as target visibility (see Fig. 4). In the Just in Time condition, the they do in flat, obstacle-free environments. manipulated target was visible for 1/2 a step during and only This strategy for visual control offers some distinct advantages. during the critical control phase. Because the previous target was Once the upcoming step is initialized, the walker can land on the always visible, visual information about the location of the ma- upcoming target by simply allowing the body to follow its natural, nipulated target relative to the previous target was available. In pendular trajectory. Because there is no need to actively steer during other words, the minimal conditions were satisfied. As predicted, this phase, the walker can begin to use information about the terrain stepping error was not significantly different from the full vision near the next step, so that that step can also be completed under control condition (t11 = 0.67, P = 0.52, d = 0.19). In the next three passive control. In this sense, the strategy can be envisioned as an conditions in Fig. 4, the duration of target visibility was expanded ongoing cycle of passive movement interspersed with brief intervals to one full step, 1 1/5 steps, and 1 1/2 steps while keeping the of active control, allowing walkers to navigate extended stretches of manipulated target visible during the critical control phase. These complex terrain with the efficiency and stability that best approxi- three conditions represent conditions in which the duration of mates that which is achieved when walking over simple terrain. target visibility is needlessly expanded. Again, stepping error was Further studies are needed to determine whether this con- not significantly different from full vision in any of these condi- clusion generalizes to uneven terrain with raised obstacles and to tions (P > 0.05). In the last two conditions, the manipulated target other locomotor speeds and gaits. Interestingly, Warren (47) was visible for one full step but before or after the critical control reported that runners must have visual information about the phase. Despite the fact that the target was visible for twice as long two upcoming targets during the phase before heel strike as it was in the Just in Time condition, stepping error was signif- on the first target. This is strikingly similar to the results of the present study and suggests that the critical phase hypothesis and icantly greater (t11 = 2.56, P < 0.05, d = 0.74 for the 2.5–1.5 con- two targets hypothesis may generalize to running. dition; t11 = 2.38, P < 0.05, d = 0.69 for the 1.0–0 condition). The results of experiment 3 provide compelling support for The findings also converge with recommendations found in the predicted minimal conditions that were derived from the manuals describing the proper use of the long cane by blind walkers, “ critical control phase and two targets hypotheses. Even a brief which suggest tapping the region of the ground infrontofthefoot ” glimpse of the two upcoming targets is sufficient, as long as that which is about to be brought forward (ref. 48, p. 251; see also refs. information is available at just the right time—that is, during the 49 and 50). This method provides the walker with haptic in- critical phase for the visual control of walking. formation about the terrain relevant to the upcoming step precisely during the critical control phase of the human gait cycle. Discussion Lastly, the findings have implications for gaze behavior and pe- ripheral vision. Although walkers visually control their movements

Summary of Findings. When humans walk over complex terrain, it COGNITIVE SCIENCES PSYCHOLOGICAL AND is necessary to make gait adjustments to accommodate obstacles with respect to one target at a time, the two targets hypothesis and and irregularities in the locations of safe footholds. However, the results of experiments 2 and 3 indicate that the relevant in- that does not mean that each and every detail of the movement formation for each step is found in the position of each target rel- needs to be controlled. Active control may be needed only to ative to the previous target. Because walkers can only look at one occasionally tweak the mechanical state of the body so that the target at a time, peripheral vision may be necessary to detect this movement can be largely under passive control. The findings of information. If the walker had a condition that impaired peripheral the present study suggest that these adjustments happen during vision, the relative positions of consecutive targets could not be the latter part of the stance phase on the leg that is intended for the accurately estimated and stepping accuracy would be expected to degrade as it did in the Gap condition of experiment 2A. Findings from previous studies in which the lower visual field is occluded (e.g., by wearing goggles) are consistent with a role for peripheral vision CP-3 CP-4 CP-5 CP-6 Experiment 3 (e.g., see ref. 35 for a review). Mean Absolute Stepping Error The Critical Control Phase and the Role of Spatial Memory. On the as- Relative to Full Vision Condition ±95% CI (mm) sumption that walkers attempt to moveasballisticallyaspossible -6 -4 -2 0246810 12 14 123 4 5 6 during the single support phase of the gait cycle, the end of the Just in Time critical phase can be demarcated by a discrete change in the me- (1.5-1.0) chanical state of the walker (i.e., toe off). In contrast, the beginning Needlessly Early of this phase is not so clearly defined. Any adjustment to the two (2.0-1.0) Needlessly Early determinants of the ballistic trajectory of the COM during the single and Late (2.0-0.8) support phase (discussed above) must occur before the onset of the Needlessly Early single support phase, and as such, visual information about up- (2.5-1.0) coming targets will become increasingly important leading up to that Too Early (2.5-1.5) point. However, in principle there is no reason why a walker could Too Late not begin making gait adjustments far in advance. Nevertheless, the (1.0-0) empirical result that subjects showed degraded performance in the Too Early condition but not the Just In Time condition shows that Fig. 4. (Left) Depiction of target visibility manipulations used in experiment 3. subjects have difficulty stepping accurately to targets that are not Horizontal bars indicate the location of the walker’sCOMwhenthetargetofthe corresponding color was visible. Vertical bands indicate hypothesized critical visible after midstance of the preceding step. The difference in be- control phase for each target. Target visibility manipulation was applied to either havior in those two conditions is especially notable because the target 4 or target 5. (Right) Mean absolute stepping error relative to full vision difference in timing between those two conditions was extremely control condition and 95% CI for each condition in experiment 3. small (half a step, or roughly 200 ms). Although there are clear

Matthis et al. PNAS | Published online July 24, 2017 | E6725 Downloaded by guest on September 27, 2021 Table 1. Participant height, weight, and leg length Average height, m (range) Average weight, kg (range) Average leg length, m (range)

Experiment 1 1.73 (1.55–1.85) 76 (50–118) 0.91 (0.81–0.98) Experiment 2A and B 1.73 (1.58–1.85) 66 (51–85) 0.94 (0.85–1.04) Experiment 3 1.67 (1.58–1.83) 63 (45–79) 0.89 (0.83–0.99)

mechanical reasons for the difference in performance between the has been extensively investigated in the context of a ball bouncing Too Late and Just in Time conditions, what could account for the task in which the subject repeatedly bounces a ball on a racket decrease in stepping accuracy in the Too Early condition? while attempting to maintain a regular rhythm and bounce height. First, it is important to note that planning multiple steps ahead Ball bouncing is analogous to bipedal walking insofar as it involves will yield diminishing returns for each additional step planned. the periodic regulation of a rhythmic process that is largely de- Mechanical simulation studies of bipedal walking based on a linear fined by its physical dynamics. Schaal et al. (2) identified a simple, inverted pendulum model show that planning more than two or passively stable solution to this problem: If the racket hits the ball three steps ahead yields only negligible increases in control au- during the deceleration phase of the upswing, small variations in thority (51, 52). In addition, due to the inescapable variability of the velocity of the ball at impact will die out on their own over motor behavior, any multistep plan will be subject to accumulating repeated cycles, and peak ball height will return to a constant error on each successive step. In short, even if walkers were able to value. As such, the actor does not need to rely on visual in- plan many steps into the future, doing so might not yield significant formation about the motion of the ball to adjust the motion of the advantages over the shorter planning window entailed by the con- racket on each and every cycle. In principle, the ball-bouncing task trol phase strategy. can be performed with eyes closed. In practice, active, perceptu- With this consideration in mind, it is possible that the control ally guided adjustments do play a role (55–57), but it is also clear phase strategy of visually guided walking represents an attempt to that actors take advantage of the inherent passive stability (2, 56– balance between the conflicting needs to plan for future steps while 58). Taken together, the findings point to a blending of these two simultaneously executing the current step. The interpretation is modes, which Siegler et al. (55) called mixed control,inwhich supported by investigations of gaze behavior in an object manipu- actors actively regulate racket movements based on perceptual lation task, which have found that subjects appear to minimize the information but also attempt to keep the system near the region of use of short-term/working memory during the execution of a hand– passive stability. Relying on mixed control has the advantage of eye coordination task. Rather than storing the task-relevant prop- shortening the time to return to steady-state bouncing after a erties of objects in memory, subjects appeared to use a “just in perturbation and eliminates the need for a switching mechanism time” strategy§ whereby they only acquired necessary information to turn active control on or off depending on whether the system just at the point that it was required for the task (53, 54). When is outside of or within the region of passive stability. Likewise, walking over terrain where each step requires precise placement, a remaining near the maximally stable region has the advantage of walker must visually identify and prepare for upcoming steps while minimizing the adjustments needed to compensate for variability simultaneously stepping toward the remembered location of the and keeps the behavior more stable. target identified in the preceding step. The execution of the guided Whereas in ball bouncing actors attempt to maintain steady- steps may exact its own cost, which may interfere with subjects’ state behavior from cycle to cycle and only use active control to ability to maintain a precise spatial memory of the location of up- respond to perturbations, walking over complex terrain requires coming targets if they are not visible during the time that the nec- the adjustment of every step to accommodate variations in the essary gait modifications need to be made (i.e., during the critical environment. As such, one might expect active control to be the control phase). Thus, because the advantages to planning multiple dominant mode during the walking task, with passive control steps ahead may be minimal anyway, walkers may adopt a strategy used only on steps for which foot placement is unconstrained. based on planning for the upcoming step only during the critical The results of the present study suggest otherwise. In the loco- phase for visual control of the upcoming step. motor task, the role that active control plays and the information In short, there are clear mechanical reasons why the end of upon which it relies is dictated by the imperative to let passive the critical control phase would occur at toe off—any control control dominate the movement. Thus, even for a task that enacted after that point would interfere with the natural, inver- cannot be successfully performed without active control, active ted pendulum-like trajectory of the body during the single sup- control appears to function in the service of passive control. port phase. The reason for the empirically identified onset of the Information is necessary for adaptive flexibility, but—as in ball critical phase (i.e., midstance) is not as obvious. We argue that bouncing—behavior is ultimately organized around solutions this timing may arise from the interaction between visual working that exploit passive forces and passive stability (59). memory and the dual-task nature of visually guided walking over terrain with consecutive constrained footholds, but this expla- Scope, Limitations, and Future Directions. The purpose of this study nation is largely speculative. Further research may be needed to (along with refs. 43, 44, and 46) was to identify the minimum re- identify the factors that contribute to the phase-specific use of quirements for the availability of visual information to support visual information to guide foot placement in complex terrain. normal levels of stepping accuracy when walking over complex terrain. Within the precision of our methods, those minimum re- Blending Active and Passive Modes of Control. The results of the quirements were satisfied in the Just in Time condition of experi- present study also have implications for the more general question ment 3, in which subjects displayed nominal stepping performance of how actors blend active and passive modes of control. This issue when visual information about the position of a target foothold relative to the previous base of support was available during the critical control phase. However, our methodology of precisely coupling the visibility of targets to subjects’ gait cycle afforded ex- perimental control at the cost of ecological validity. Although our §The similar naming of the Just in Time condition in our experiments and the Just in Time gaze allocation strategy in ref. 54 was coincidental, though it is an interesting example results are consistent with many studies of gaze behavior during of convergent research. locomotion, these behaviors may be affected by subtle changes in

E6726 | www.pnas.org/cgi/doi/10.1073/pnas.1611699114 Matthis et al. Downloaded by guest on September 27, 2021 of the COM and lower limbs, walkers use visual information in a PNAS PLUS feedforward manner to tailor the initial conditions of each step to the upcoming path. From a neural perspective, visual signals mediated

1.44 m through the posterior parietal and motor cortices may generate the

4.00 m descending signals needed to modulate the ongoing rhythmic oscil- lations of the subcortical and spinal networks to generate the ap- 5.00 m propriate muscular impulses for the coming step (21, 23). In this way, visual control allows walkers to traverse complex terrain while Fig. 5. Experimental set-up (shown with all targets visible). Reprinted with exploiting the same dynamics that underlie the energetic efficiency permission from ref. 46 (Matthis et al.), copyright Association for Research in of human locomotion in flat, obstacle-free environments. Vision and Ophthalmology. Materials and Methods Participants. Fourteen subjects [5 females, 9 males; mean age = 18.7 (18–21) years] the task when visual information is unconstrained. For instance, it = ’ participated in experiment 1, 18 subjects [7 females, 11 males; mean age 19.1 has been shown that subjects abilitytostepaccuratelyontargetsin (18–21) years] participated in experiment 2, and 12 subjects [5 females, 7 males; an otherwise unconstrained walkway was correlated with the dura- mean age = 20.0 (18–22) years] participated in experiment 3. Data from two tion of the time that they fixated the target (60–62). This finding subjects in experiment 1 were excluded because their average stepping accuracy would appear to conflict with our finding that the availability of across all conditions was more than 2 SDs from the mean. Mean height, weight, visual information about target location is not necessary after toe and leg length for participants in each experiment are given in Table 1. Partic- off, except that the targets in those experiments were always sepa- ipants were recruited from psychology courses and received extra credit for rated by at least two steps’ worth of empty walkway. This separation participating. All participants reported normal or corrected-to-normal vision, and mitigated the dual-task nature of simultaneous step planning and none reported any motor impairments. The protocol was approved by the In- execution that was present in our studies, which may explain the stitutional Review Board at Rensselaer Polytechnic Institute, and all subjects gave informed consent before performing the experiment. differences in behavior. Similarly, subjects were able to successfully step over an obstacle that was occluded during the last five steps of Experimental Apparatus and Setup. All three experiments made use of a the approach (30), but this may not have been possible if there had custom-built experimental apparatus that synchronizes a full body motion been other impediments in the space before the obstacle. capture system to an LCD projector that displays virtual target footholds on Human behavior is nothing if not adaptable, and we should the floor (Fig. 5). The system detects when a subject’s foot lands on one of the expect that subjects’ use of visual information will vary with the virtual targets in real time and can render the terrain visible or invisible based on specific constraints of the task that is being performed (53). the subject’s location in the room. The setup used a 14-camera Vicon Motion Nevertheless, the results of this study reveal the way that the Capture system running Vicon Nexus 1.7 software. The motion capture system minimum requirements for successful locomotion over complex tracked the positions of a set of retroreflective markers that were attached to terrain are shaped by the biomechanics and physical dynamics of black spandex suits worn by subjects in accordance with the Vicon Plug-In Gait bipedal gait. Future research will examine walkers’ gaze behavior Full Body (SACR) marker set included in the Nexus software package. A Sanyo as they traverse rough terrain to understand how the constraints described by the control phase hypothesis extrapolate to more COGNITIVE SCIENCES natural locomotor environments. In addition, although much of A PSYCHOLOGICAL AND our interpretation of the results of these studies on the regulation of the push-off force, we do not directly measure this aspect of subjects’ locomotor behavior. Future work may record muscle activity and ground force during walking over visually specified targets to test the specific biomechanical pre- dictions afforded by the control phase hypothesis about the visual control of foot placement walking over complex terrain. Concluding Remarks A central component of the characterization of walking in- B troduced in this study is that humans attempt to move ballistically during the single support phase, allowing the trajectory of their body to be largely determined by their physical structure, mo- mentum, and the force of gravity. Of course, the muscles do play a role, even during the single support phase when muscle cocon- traction is required to support body weight and maintain stiffness of the knee (63). Although the details of muscle work and ener- getic costs throughout the gait cycle is an area of active discussion (e.g., refs. 64 and 65), it is undeniable that bipedal gait is accom- panied by significant physical forces, and as such these forces may be harnessed for the purposes of minimizing energetic costs. Human motor control is often quite miserly with respect to en- ergetic costs. Walkers readily adjust their preferred locomotor pat- terns to achieve increases in energetic efficiency, even when such increases are small and brought about by artificial manipulations, Fig. 6. Schematic depiction of the target visibility manipulation used in the Just in Time condition applied to target 4. (A) Target 4 becomes visible when such as by applying springs to the lower limbs (66, 67) or by moving ’ the belts on a split belt treadmill at different speeds (68). From this the subject s foot crosses the visibility trigger surrounding target 3 (blue dashed circle) and remains so until the foot crosses the invisibility trigger perspective, it should come as no surprise that the visual control of (red dotted circle). (B) The steps to target 3 (blue arrows) and target 4 (red walking over complex terrain is similarly organized around the goal of arrows) shown with time progressing from left to right. The visibility of minimizing the energetic cost of locomotion. Rather than using vi- target 4 is shown on the black and red arrow, along with the trigger number sual feedback to enact continuous, feedback-driven muscular control associated the changing visibility.

Matthis et al. PNAS | Published online July 24, 2017 | E6727 Downloaded by guest on September 27, 2021 PLC-XP45 projector was used to display virtual obstacles onto the floor at a Fig. 6B is also useful for understanding the visual look-ahead distance resolution of 1024 × 768 with a brightness of 3500 ANSI lumens. The projector that was available to subjects in the different visibility conditions. When the was located 1.75 m behind the end position at a height of 1.15 m pointed at the right (blue-colored) foot crosses the visibility trigger that causes target 4 to ground at a low angle of incidence (∼20 degrees; Fig. 5). The motion capture become visible, the vertical projection of the subject’s head is approximately and projector systems were integrated using custom software. The projector two step lengths from the location of target 4. Thus, the 1.5 trigger used in position was calibrated using a method inspired by the Tsai calibration algo- the Just in Time condition provided subjects with two step lengths of visual rithm (69). For this method, calibration points defined in pixel coordinates (i.e., look ahead. Put another way, the 1.5 trigger in this study is equivalent to the xy coordinates on a computer monitor) were displayed on the laboratory floor, two step-length visibility windows used in Matthis and Fajen (43, 44). and a marker tracked by the motion capture system was placed on each pro- As a result of processing time and the network communication between the jected point. The position of the marker in world coordinates (i.e., as defined by computer running the motion capture system and the computer controlling the the motion capture system) and the position of the corresponding calibration projector, there was an 80–90-ms lag between the instant that a subject point in pixel coordinates were used to calculate the transformation between reached a visibility or invisibility trigger (in the real world) and the instant that screen coordinates (pixels) to world coordinates (millimeters). the display was updated. Thus, a visibility trigger that was set to be 0.5 steps would actually make the target visible at ∼0.35 steps. To compensate for this Task and Procedure. The basic task required subjects to walk across a path of lag, triggers were programmed to be 15% larger than the desired visibility six small circular target footholds (radius, 50 mm) that covered the space manipulation. Thus, to achieve an actual 0.5 step visibility trigger, the trigger between the start and end locations. The path between the start and end would be programmed to be 0.65 steps. To make targets appear or disappear boxes was overlaid by a scintillating white noise texture designed to prevent at the beginning of a step, we placed the trigger around the preceding target subjects from using the carpet texture as a landmark to keep track of targets and set the radius to 0.1 steps. We then delayed the visibility manipulation by after they were made invisible. To define the target configurations used in 130 ms so that the target appeared or disappeared near toe off. All adjusted this study, the experimenter recorded their normal footfall pattern when triggers were tested before the experiment and verified for each subject fol- walking from the start to end location without targets. The target config- lowing data collection. By comparing motion capture data to logs from the urations were then obtained by pseudorandomly placing the targets within a computer controlling the projector, we were able to verify that the experi- × 300 mm 300 mm box centered on each step in this pattern of foot falls. To enced trigger corresponded to the intended visibility windows described here. scale the task to each subject’s body size, the distances between the targets were multiplied by the ratio of the subject’s leg length to the leg length of Design. Each experimental session comprised two blocks of trials with a short break the experimenter who defined the original footfall pattern. between blocks. In each block, trials were presented in subblocks of 12 trials of the Subjects completed the experiment barefoot to avoid any irregularities caused same condition (i.e., with the same visibility and invisibility triggers). The order of by differences in footwear. They were asked to walk at a brisk pace from a start conditions within each block was randomized across subjects and the position of position (marked with tape on a carpeted floor) to an end position 5.0 m away. each target varied across repetitions within a subblock as described in Task and Subjects initiated each trial by clicking a button on a wireless mouse. They were Procedure. The second block was identical to the first, except that the order of instructed to walk so as to place the ball of their foot (specifically, the mocap conditions (i.e., subblocks) was reversed. In total, each subject completed 24 rep- marker attached to the second metatarsal head of the foot) as close to the center etitions of each of the experimental conditions. In experiments 1 and 3, subjects of the target as possible. At each step, if the marker was within the target also completed 12 repetitions of target-free walking before the first subblock. radius, the computer speakers played a high-pitched tone to indicate that the subject had successfully hit the target. Otherwise, a low-pitched tone was Experiment 1 also included three conditions that were not reported in the played to indicate that the target had been missed. If the subject failed to reach main text or Fig. 2, and experiment 2 also included a set of conditions in which the end position in 5 s, the trial was terminated and rerun. This constraint the same manipulations shown in Fig. 3 were applied to two consecutive targets. ensured that subjects had to walk at an average speed of at least 1.0 m/s to The data from these conditions were not reported in the main text or figures complete the trial in time. Trials in which the subject missed one or more targets in the interest of brevity but can be seen in Figs. S2 and S3). The findings from were not rerun. these conditions are consistent with the interpretation given in the main text. The visibility of target footholds was controlled using a combination of an “invisibility trigger” (similar to the one used in ref. 46) and a “visibility trigger.” Analyses. In all experiments, stepping accuracy was calculated by taking the These triggers were circular regions centered on each target that trigger a root mean square of the Euclidean distance between the marker on the change in the target’s visibility when the subject’s foot entered into its radius. second metatarsal head of the foot and the center of the target. Stepping Visibility triggers caused targets to go from invisible to visible, whereas in- accuracy was analyzed for targets 3–6 in experiment 1, as the visibility ma- visibility triggers did the opposite. By combining these two types of triggers, we nipulations did not apply to targets 1 and 2. Because young, healthy adults could ensure that targets were only visible to the subject at a specified phase of reach steady-state walking speed within the first two steps (70), this also their gait cycle as they traversed the path of target footholds (Fig. 6A). helped to ensure that the analyses were based on the portion of each trial Fig. 6B shows the steps to target 3 and target 4 unfolding continuously in time. during which subjects were moving at a constant speed. The right leg (colored blue in Fig. 6B) leaves target 1 and crosses the visibility In experiment 2A, stepping accuracy was only analyzed for targets 4 and trigger centered on target 3 (blue dotted line in Fig. 6A) ∼50% of the way through 5 to allow for comparison between the results of experiment 2A and 2C. In that step (during midstance, when the walker is directly above the supporting limb experiment 2b and experiment 3, stepping accuracy was only analyzed from and roughly two step lengths from target 4). At this point, target 4 becomes the target that was affected by the visibility manipulation. visible and remains so through the double-support phase. It then becomes invisible when the left foot (colored red in Fig. 6B) leaves target 2 at toe off ACKNOWLEDGMENTS. We thank Gaby Ciavardoni and Kevin Sullivan for their and remains so for the entire step toward Target 4 (red dotted arrow). The programming contributions and Evelyn Hinojosa and Dylan Brion for their other visibility manipulations used in this study can also be understood using assistance with data collection. This research was supported by National Science this representation. Foundation Grant 1431078 and National Institutes of Health Grant 1R01EY019317.

1. Bernstein N (1967) The coordination and regulation of movements (Pergamon Press, 10. Neptune RR, Zajac FE, Kautz SA (2004) Muscle mechanical work requirements during New York). normal walking: The energetic cost of raising the body’s center-of-mass is significant. 2. Schaal S, Atkeson CG, Sternad D (1996) One-handed juggling: A dynamical approach J Biomech 37:817–825. to a rhythmic movement task. J Mot Behav 28:165–183. 11. Adamczyk PG, Kuo AD (2009) Redirection of center-of-mass velocity during the step- 3. Rossignol S, Dubuc R, Gossard J-P (2006) Dynamic sensorimotor interactions in loco- to-step transition of human walking. J Exp Biol 212:2668–2678. motion. Physiol Rev 86:89–154. 12. Donelan JM, Kram R, Kuo AD (2002) Simultaneous positive and negative external 4. Cavagna GA, Margaria R (1966) Mechanics of walking. J Appl Physiol 21:271–278. mechanical work in human walking. J Biomech 35:117–124. 5. Cavagna GA, Kaneko M (1977) Mechanical work and efficiency in level walking and 13. Kuo AD (2002) Energetics of actively powered locomotion using the simplest walking running. J Physiol 268:467–481. model. J Biomech Eng 124:113–120. 6. Mochon S, McMahon TA (1980) Ballistic walking. J Biomech 13:49–57. 14. Zelik KE, Takahashi KZ, Sawicki GS (2015) Six degree-of-freedom analysis of hip, knee, 7. Kuo AD (2007) The six determinants of gait and the inverted pendulum analogy: A ankle and foot provides updated understanding of biomechanical work during hu- dynamic walking perspective. Hum Mov Sci 26:617–656. man walking. J Exp Biol 218:876–886. 8. Kuo AD, Donelan JM, Ruina A (2005) Energetic consequences of walking like an in- 15. Kuo AD, Donelan JM (2010) Dynamic principles of gait and their clinical implications. verted pendulum: Step-to-step transitions. Exerc Sport Sci Rev 33:88–97. Phys Ther 90:157–174. 9. Donelan JM, Kram R, Kuo AD (2002) Mechanical work for step-to-step transitions is a 16. Donelan JM, Kram R, Kuo AD (2001) Mechanical and metabolic determinants of the major determinant of the metabolic cost of human walking. J Exp Biol 205:3717–3727. preferred step width in human walking. Proc Biol Sci 268:1985–1992.

E6728 | www.pnas.org/cgi/doi/10.1073/pnas.1611699114 Matthis et al. Downloaded by guest on September 27, 2021 17. McGeer T (1990) Passive dynamic walking. Int J Robot Res 9:62–82. 49. Miller J (1967) Vision, a component of locomotion. Physiotherapy 53:326–332. PNAS PLUS 18. Ahn J, Hogan N (2012) Walking is not like reaching: Evidence from periodic me- 50. Rodgers MD, Emerson RW (2005) Human factor analysis of long cane design: Weight chanical perturbations. PLoS One 7:e31767. and length. J Vis Impair Blind 99:622–632. 19. Ahn J, Hogan N (2012) A simple state-determined model reproduces entrainment and 51. Koolen T, De Boer T, Rebula J, Goswami A, Pratt J (2010) Capturability based analysis phase-locking of human walking. PLoS One 7:e47963. and control of legged locomotion, part 2: Application to three simple gait models. 20. Ahn J, Hogan N (2013) Long-range correlations in stride intervals may emerge from Control (January):1–15. non-chaotic walking dynamics. PLoS One 8:e73239. 52. Rebula J, Koolen IT, De Boer T (2010) Capturability Based Analysis and Control of 21. McVea DA, Pearson KG (2009) Object avoidance during locomotion. Advances in Legged Locomotion, Part 1 : Concepts and Definitions. Control (January):1–11. Experimental Medicine and Biology, ed Sternad D (Springer, Boston, MA), pp 293–315. 53. Hayhoe M, Ballard D (2005) Eye movements in natural behavior. Trends Cogn Sci 9: 22. Andujar JE, Lajoie K, Drew T (2010) A contribution of area 5 of the posterior parietal 188–194. cortex to the planning of visually guided locomotion: Limb-specific and limb- 54. Ballard DH, Hayhoe MM, Pelz JB (1995) Memory representations in natural tasks. independent effects. J Neurophysiol 103:986–1006. J Cogn Neurosci 7:66–80. 23. Drew T, Marigold DS (2015) Taking the next step: Cortical contributions to the control 55. Siegler IA, Bardy BG, Warren WH (2010) Passive vs. active control of rhythmic ball of locomotion. Curr Opin Neurobiol 33:25–33. bouncing: the role of visual information. J Exp Psychol Hum Percept Perform 36:729–750. 24. Kalaska JF (1996) Parietal cortex area 5 and visuomotor behavior. Can J Physiol 56. Sternad D, Duarte M, Katsumata H, Schaal S (2001) Bouncing a ball: tuning into dy- Pharmacol 74:483–498. namic stability. J Exp Psychol Hum Percept Perform 27:1163–1184. 25. Lacquaniti F, Guigon E, Bianchi L, Ferraina S, Caminiti R (1995) Representing spatial in- 57. Wei K, Dijkstra TMH, Sternad D (2007) Passive stability and active control in a formation for limb movement: Role of area 5 in the monkey. Cereb Cortex 5:391–409. rhythmic task. J Neurophysiol 98:2633–2646. 26. Maeda RS, O’Connor SM, Donelan JM, Marigold DS (2016) Foot placement relies on 58. de Rugy A, Wei K, Müller H, Sternad D (2003) Actively tracking ‘passive’ stability in a state estimation during visually guided walking. J Neurophysiol 117:480–491. ball bouncing task. Brain Res 982:64–78. 27. Patla AE, Adkin A, Martin C, Holden R, Prentice S (1996) Characteristics of voluntary 59. Warren WH (2006) The dynamics of perception and action. Psychol Rev 113:358–389. visual sampling of the environment for safe locomotion over different terrains. Exp 60. Chapman GJ, Hollands MA (2007) Evidence that older adult fallers prioritise the Brain Res 112:513–522. planning of future stepping actions over the accurate execution of ongoing steps 28. Patla AE (1997) Understanding the roles of vision in the control of human locomotion. during complex locomotor tasks. Gait Posture 26:59–67. Gait Posture 5:54–69. 61. Chapman GJ, Hollands MA (2006) Evidence for a link between changes to gaze behaviour 29. Barton SL, Matthis JS, Fajen BR (2017) Visual regulation of gait: Zeroing in on a solution and risk of falling in older adults during adaptive locomotion. Gait Posture 24:288–294. to the complex terrain problem. J Exp Psychol Hum Percept Perform, 10.1037/ 62. Young WR, Hollands MA (2010) Can telling older adults where to look reduce falls? xhp0000435. Evidence for a causal link between inappropriate visual sampling and suboptimal 30. Patla AE, Greig M (2006) Any way you look at it, successful obstacle negotiation needs stepping performance. Exp Brain Res 204:103–113. visually guided on-line foot placement regulation during the approach phase. 63. McGowan CP, Neptune RR, Clark DJ, Kautz SA (2010) Modular control of human Neurosci Lett 397:110–114. walking: Adaptations to altered mechanical demands. J Biomech 43:412–419. 31. Marigold DS, Patla AE (2007) Gaze fixation patterns for negotiating complex ground 64. Kuo AD, Donelan JM (2009) Comment on “Contributions of the individual ankle terrain. Neuroscience 144:302–313. plantar flexors to support, forward progression and swing initiation during walking” 32. Patla AE, Vickers JN (2003) How far ahead do we look when required to step on (Neptune et al., 2001) and “Muscle mechanical work requirements during normal specific locations in the travel path during locomotion? Exp Brain Res 148:133–138. walking: The energetic cost of raising the body’s center-of-mass is significant” 33. Patla AE, Vickers JN (1997) Where and when do we look as we approach and step over (Neptune et al., 2004). J Biomech 42:1783–1785, author reply 1786–1789. an obstacle in the travel path? Neuroreport 8:3661–3665. 65. Neptune RR, Zajac FE, Kautz SA (2009) Author’s response to comment on “Contri- 34. Franchak JM, Adolph KE (2010) Visually guided navigation: Head-mounted eye- butions of the individual ankle plantar flexors to support, forward progression and tracking of natural locomotion in children and adults. Vision Res 50:2766–2774. swing initiation during walking” (Neptune et al., 2001) and “Muscle mechanical work 35. Marigold DS (2008) Role of peripheral visual cues in online visual guidance of loco- requirements during normal walking: The energetic cost of raising the body’s center- motion. Exerc Sport Sci Rev 36:145–151. of-mass is significant.” J Biomech 42:1786–1789. 36. Laurent M, Thomson JA (1988) The role of visual information in control of a con- 66. Snaterse M, Ton R, Kuo AD, Donelan JM (2011) Distinct fast and slow processes strained locomotor task. J Mot Behav 20:17–37. contribute to the selection of preferred step frequency during human walking. J Appl 37. Marigold DS, Weerdesteyn V, Patla AE, Duysens J (2007) Keep looking ahead? Re- Physiol (1985) 110:1682–1690. direction of visual fixation does not always occur during an unpredictable obstacle 67. Selinger JC, O’Connor SM, Wong JD, Donelan JM (2015) Humans can continuously COGNITIVE SCIENCES avoidance task. Exp Brain Res 176:32–42. optimize energetic cost during walking. Curr Biol 25:2452–2456. PSYCHOLOGICAL AND 38. Patla AE, Beuter A, Prentice S (1991) A two stage correction of limb trajectory to avoid 68. Finley JM, Bastian AJ, Gottschall JS (2013) Learning to be economical: The energy cost obstacles during stepping. Neurosci Res Commun 8:153–159. of walking tracks motor adaptation. J Physiol 591:1081–1095. 39. Day BL, Reynolds RF (2005) Vestibular reafference shapes voluntary movement. Curr 69. Tsai B (1987) A versatile camera calibration technique for high-accuracy 3D machine vision Biol 15:1390–1394. meteorology using off-the-shelf TV camera and lenses. IEEE J Robot Autom RA-3:323–344. 40. Weerdesteyn V, Nienhuis B, Hampsink B, Duysens J (2004) Gait adjustments in re- 70. Muir BC, Rietdyk S, Haddad JM (2014) Gait initiation: The first four steps in adults sponse to an obstacle are faster than voluntary reactions. Hum Mov Sci 23:351–363. aged 20-25 years, 65-79 years, and 80-91 years. Gait Posture 39:490–494. 41. Voloshina AS, Kuo AD, Daley MA, Ferris DP (2013) Biomechanics and energetics of 71. Garcia M, Chatterjee A, Ruina A, Coleman M (1998) The simplest walking model: walking on uneven terrain. J Exp Biol 216:3963–3970. Stability, complexity, and scaling. J Biomech Eng 120:281–288. 42. Voloshina AS, Ferris DP (2015) Biomechanics and energetics of running on uneven 72. Collins S, Ruina A, Tedrake R, Wisse M (2005) Efficient bipedal robots based on terrain. J Exp Biol 218:711–719. passive-dynamic walkers. Science 307:1082–1085. 43. Matthis JS, Fajen BR (2014) Visual control of foot placement when walking over 73. Holt KJ, Jeng SF, Rr RR, Hamill J (1995) Energetic cost and stability during human complex terrain. J Exp Psychol Hum Percept Perform 40:106–115. walking at the preferred stride velocity. J Mot Behav 27:164–178. 44. Matthis JS, Fajen BR (2013) Humans exploit the biomechanics of bipedal gait during 74. Holt KG, Hamill J, Andres RO (1990) The force-driven harmonic oscillator as a model visually guided walking over complex terrain. Proc R Soc B Biol Sci 280:20130700. for human locomotion. Hum Mov Sci 9:55–68. 45. Zaytsev P, Hasaneini SJ, Ruina A (2015) Two steps is enough: No need to plan far 75. Kuo AD (2001) A simple model of bipedal walking predicts the preferred speed-step ahead for walking balance. 2015 IEEE International Conference on and length relationship. J Biomech Eng 123:264–269. Automation (ICRA) (IEEE, New York), pp 6295–6300. 76. Kuo AD (1999) Stabilization of lateral motion in passive dynamic walking. Int J Robot 46. Matthis JS, Barton SL, Fajen BR (2015) The biomechanics of walking shape the use of Res 18:917–930. visual information during locomotion over complex terrain. JVis15:10. 77. Bauby CE, Kuo AD (2000) Active control of lateral balance in human walking. 47. Warren WH (1988) Action modes and laws of control for the visual guidance of ac- J Biomech 33:1433–1440. tion. Adv Psychol 50:339–379. 78. Donelan JM, Shipman DW, Kram R, Kuo AD (2004) Mechanical and metabolic re- 48. Miller ME, Hoover RE (1946) Foot travel without sight. Outlook Blind 40:244–251. quirements for active lateral stabilization in human walking. J Biomech 37:827–835.

Matthis et al. PNAS | Published online July 24, 2017 | E6729 Downloaded by guest on September 27, 2021