Predicting and Facilitating the Emergence of Optimal Solutions for a Cooperative “Herding” Task and Testing Their Similitude to Contexts Utilizing Full-Body Motion

Home , Synergetics (Haken)

Predicting and Facilitating the Emergence of Optimal Solutions for a Cooperative “Herding”

Task and Testing their Similitude to Contexts Utilizing Full-Body Motion

A dissertation submitted to the

Division of Graduate Education and Research

of the University of Cincinnati

in partial fulfillment of the

requirements for the degree of

DOCTOR OF PHILOSOPHY

in the Department of Psychology

of the McMicken College of Arts and Sciences

Patrick Nalepka

M. A. University of Cincinnati, 2016

January, 2018

Committee Chair: Kevin Shockley, Ph.D.

Committee: Michael J. Richardson, Ph.D. Anthony Chemero, Ph.D.

Rachel W. Kallen, Ph.D.

Paula Silva, Ph.D.

Abstract

Multi-agent activity is an emergent process, with the roles and responsibilities of individual actors a self-organized consequence of task-dynamic constraints and perturbations. The shepherding paradigm, first investigated by Nalepka, Kallen, Chemero, Saltzman, and Richardson,

(2017), was directed towards exploring the emergence of stable multi-agent behavioral modes within dynamically changing task-environments. The task involved pairs of participants using their hands to contain a herd of autonomous and reactive “sheep” within a virtual game field projected on a tabletop display. Initially, all participants employed a search-and-recover (S&R) mode of behavior, moving from sheep-to-sheep to corral the herd to a target containment region in the center of the game field. However, a subset of dyads learned to coordinate their movements, forming an oscillating “wall” that contained the herd (termed coupled oscillatory containment)— a behavioral mode termed coupled oscillatory containment (COC)—which allowed dyads to achieve superior task performance. Experiment 1 investigated a potential control parameter to promote the emergence of COC behavior, as well as determine whether changes in oculomotor behavior might predict its emergence using recurrence quantification analysis (RQA). Experiment

2 sought to validate weather S&R and COC behavior also defined a more realistic herding situation that involved the full-body movement of participants in a large task space. Results indicated that manipulating task difficulty, by controlling how fast the sheep could move, promoted the use of more coordinated modes of behavior (Experiment 1 and 2). Significant changes in the determinism and complexity (entropy) of oculomotor behavior was observed two trials prior to the discovery of COC behavior using RQA (Experiment 1). In full-body herding (Experiment 2), a subset of dyads discovered a coordinated mode of behavior that involved a joint circling pattern in a fixed direction, with the frequency with which this behavioral mode emerged increasing with increases iii

in task difficulty. Future work will investigate the informational basis that leads to the discovery of optimal solutions (i.e., oscillatory behavioral coordination) in the shepherding task, as well as the design of flexible, adaptive artificial (machine) systems that can learn and work alongside humans in these unstable task-environments.

Keywords: multi-agent coordination, shepherding, strategy discovery, recurrence quantification analysis

Acknowledgments

Special thanks to the committee who reviewed this work, my instructors, both past and present, for giving me the tools needed to carve my own path, and to my friends and family for supporting me throughout my academic career. I would like to acknowledge Elliot Saltzman and

Maurice Lamb, who are collaborators on the “shepherding” project. I would also like to acknowledge Carl Bou Mansour, Christopher Riehm, Guilherme Sanches De Oliveira and Riley

Mayr, who assisted in either equipment setup, data collection or data post-processing on one or more of the “shepherding” experiments throughout the years. Finally, I would like to acknowledge

Melissa Kathryn Ridgley for her love and support the past four years that made my time in

Cincinnati unforgettable.

This research was supported by the National Institutes of Health (R01GM105045). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Contents List of Figures ...... ix

CHAPTER 1 Introduction...... 1

Current Dissertation ...... 5

CHAPTER 2 Experiment 1 Facilitating and Predicting the Emergence of Coupled Oscillatory

Containment ...... 8

Dynamics of Behavioral Change ...... 8

Predictors to Behavioral Change ...... 11

Current Experiment ...... 16

Method ...... 18

Results ...... 25

Discussion ...... 38

CHAPTER 3 Experiment 2 Shepherding Dynamic Similitude to Full-Body Movement ...... 44

Dynamic Similitude of Shepherding and Herding Behavior ...... 45

Dynamic Motor Primitives of Task-Dynamics ...... 47

Current Experiment ...... 48

Method ...... 48

Results ...... 52

Discussion ...... 59

CHAPTER 4 General Discussion ...... 67

Summary of Findings in Experiment 1 ...... 67

Summary of Findings in Experiment 2 ...... 68

vii

General Conclusions ...... 69

References ...... 73

viii

List of Figures

Figure 1. Depiction of experiment setup from Nalepka, Kallen et al. (2017)...... 2

Figure 2. Illustrations of the behavioral modes observed in Nalepka, Kallen et al., (2017)...... 4

Figure 3. Example gear-problems from Stephen, Boncoddo et al., (2009)...... 13

Figure 4. Change in Entropy during a phase transition in a Lorenz system indexed using recurrence quantification analysis. Taken from Stephen, Dixon and Isenhower (2009)...... 14

Figure 5. Example recurrence plot used for recurrence quantification analysis (RQA). Taken from

Webber & Zbilut (2005)...... 15

Figure 6. Depiction of Experiment 1...... 19

Figure 7. Behavior classification for Experiment 1...... 22

Figure 8. Summary plots depicting within-subject differences in behavioral mode on shepherding performance.for Experiment 1...... 28

Figure 9. Summary plots depicting within-subject differences on ocular descriptive measures for unsuccessful performance, successful S&R performance, and successful COC performance for

Experiment 1...... 30

Figure 10. Summary plots depicting within-subject differences on ocular dynamics for unsuccessful performance, successful S&R performance, and successful COC performance for

Experiment 1...... 31

Figure 11. Summary plots depicting within-subject differences on ocular descriptive measures for unsuccessful performance, successful S&R performance, and successful COC performance for

Experiment 1 using a 0.5 Hz cutoff criterion for COC trials...... 33

Figure 12. Summary plots depicting within-subject differences on ocular dynamics for unsuccessful performance, successful S&R performance, and successful COC performance for

Experiment 1 using a 0.5 Hz cutoff criterion for COC trials...... 34

Figure 13. Summary plots depicting ocular dynamics at different time-points (on the 3rd, 2nd and last trial before first COC success [labeled 1]) for Experiment 1...... 36

Figure 14. Summary plots depicting ocular descriptive measures at different time-points (on the

3rd, 2nd and last trial before first COC success [labeled 1]) for Experiment 1...... 37

Figure 15. Changes in Entropy and %Determinism from 8 trials before first successful use of COC

(labeled 1) to the subsequent 2-8 trials that followed for Experiment 1...... 41

Figure 16. Simulations of sheepdog behavior. Taken from Strömbom et al., (2014)...... 46

Figure 17. Depiction of virtual task field for Experiment 2...... 49

Figure 18. Summary plots on effect of task-difficulty on successful shepherding performance for

Experiment 2...... 54

Figure 19. Exemplar time-series displaying unwrapped angular position and x,y position for each behavioral mode observed in Experiment 2...... 56

Figure 20. Summary plots for the effect of behavioral mode on shepherding performance for

Experiment 2...... 57

Figure 21. Summary plots for the effect of behavioral mode on interpersonal coordination using relative phase analysis for Experiment 2...... 59

Figure 22. Circling behavior example from Nalepka, Kallen et al., (2017)...... 63

Figure 23. Cooperative collision avoidance task used in Richardson et al., (2015). Taken from

Richardson et al., (2016)...... 64

CHAPTER 1

Introduction

Many human behaviors are cooperative, requiring two or more actors to come together to coordinate their behavior as a collective unit or team. Such coordination occurs during collaborative dance, team sports, or simply when two or more individuals are clearing a dinner table or ushering a group of school children through a crowded museum. The fundamental question for researchers interested in understanding such coordinated, joint-action, is how individuals can achieve these kinds of dynamic, multi-agent-environment behaviors so successfully and across so many task situations?

Over the last two decades, attempts at answering this question has focused on identifying the cognitive and neural mechanisms that allow for the prediction of other agents’ actions and for the development of shared intentional states (Knoblich, Butterfill, & Sebanz, 2011; Vesper,

Schmitz, Safra, Sebanz, & Knoblich, 2016). However, a growing body of work has also demonstrated that neurocognitive mechanisms alone cannot explain adapted multi-agent coordination and that such behavior is often self-organized, emerging from the embodied and embedded constraints of agent-environment systems (Richardson et al., 2016; Shockley,

Richardson, & Dale, 2009). Of particular relevance here, is that this latter research has also indicated that the stable patterns of coordinated multi-agent activity can be understood to emerge from low-dimensional, nonlinear task-dynamical laws (e.g., Richardson et al., 2015, 2016; also see Saltzman & Kelso, 1987; Warren, 2006).

An excellent example of the self-organized, task-dynamic emergence of multi-agent coordination is provided by the recent work of Nalepka, Kallen et al., (2017), in which the 1

behavioral dynamics of a two-person shepherding task was investigated. The task required naïve participants, recruited as dyads, to collect and contain 3, 5 or 7 autonomous, virtual sheep (Figure

1, c-e), within a target containment area for 45 seconds (Figure 1, b). The task space was displayed on a translucent glass tabletop, with participants standing on opposite sides of the tabletop and corralling the virtual sheep by controlling the position of virtual sheepdogs (a blue or red colored box) using hand-held motion-tracking sensors (see Figure 1, a). Sheep movement was governed by Brownian motion dynamics. However, when sheep were within a certain distance of a participant’s hand (sheepdog box), the sheep were repelled away from the participant’s hand location, moving directly away from the participant’s hand location (Nalepka, Kallen, Chemero,

Saltzman, & Richardson, 2017)

Figure 1. (a) Depiction of experiment setup from Nalepka, Kallen et al. (2017). (b) Depiction of the task/game space. Initial arrangement of the 3 (c), 5 (d) and 7 (e) initial sheep arrangement. Taken from Nalepka, Kallen et al., (2017). See text for more details.

During the early stages of task performance, participants implicitly subdivided the task-

(table-) space in half and attempted to corral the sheep by moving from furthest-sheep to furthest- sheep, adopting a Search-and-Recover [S&R] strategy (Figure 2, top). Some dyads were able to complete the task successfully utilizing this strategy, especially when task difficulty was low; i.e., when herding only 3 or 5 sheep. However, a certain subset of dyads, predominately those in the 7- sheep condition, also discovered another behavioral solution, one that involved the participants oscillating their hands (sheepdogs) in a semi-circular manner around the sheep herd, forming a virtual ‘wall’ around the sheep. This mode of behavior was termed coupled oscillatory containment [COC], because participant pairs also synchronized their oscillatory movements in either an in-phase (i.e., spatially symmetrical; Figure 2, bottom right) or antiphase (i.e., keeping to opposite sides of the circle; Figure 2, bottom left) manner, consistent with the stable modes of bimanual rhythmic coordination known to define visual and interpersonal rhythmic coordination more generally (Haken, Kelso, & Bunz, 1985; Schmidt, Carello, & Turvey, 1990). Importantly,

COC behavior was much more effective in herding the sheep within the target area compared to

S&R behavior, such that those dyads that discovered COC were able to achieve near-optimal task performance (Nalepka, Kallen, et al., 2017; Nalepka, Lamb, et al., 2017).

Figure 2. Illustrations of the behavioral modes observed in Nalepka, Kallen et al., (2017).

Regarding the underlying task-dynamics of the shepherding task, Nalepka, Kallen et al.,

(2017) illustrated how a low-dimensional dynamical (differential equation) model composed of point-attractive mass-spring functions and nonlinear oscillators could produce the same S&R and

COC behaviors exhibited by participants. The model also revealed how the spontaneous transition between S&R and COC behavior was due to a task-dependent Hopf Bifurcation—a Hopf bifurcation occurs when a fixed-point attractor (characteristic of S&R) loses stability and, at a critical point, switches to a periodic, limit-cycle behavior (characteristic of COC). More recently,

Nalepka and colleagues have demonstrated how an adapted version of this model implemented within the control structure of an artificial, computer-controlled humanoid avatar is also capable of performing the shepherding task with novice human actors. The effectiveness of the model in capturing human-like S&R and COC behavior was revealed by the fact that the novice human actors were not only able to learn effective S&R and COC behavior when completing the shepherding task with the model-controlled artificial agent, but produced a final level of herding performance that was comparable to when novice participants completed the shepherding task with 4

an expert human player (Nalepka et al., 2016). Finally, Nalepka, Lamb et al., (2017) have demonstrated that the same S&R and COC behaviors also emerged when sheep are randomly scattered throughout the game environment (field) and participants are simply instructed to corral and contain the sheep to a self-defined target region; i.e., no instructed containment location was given, rather participants were free to nonverbally negotiate amongst themselves where to contain the sheep within the game field.

Current Dissertation

The objective of the current dissertation project was to further explore the behavioral dynamics of the multi-agent shepherding task developed by Nalepka, Kallen et al., (2017). The reason why the shepherding task was of interest was because it embodies many of the processes that characterize multi-agent activity. More specifically, the shepherding task is a goal-directed social task that requires multiple actors to actively work together to control an unstable, dynamically changing environment. Moreover, successful performance requires that co-actors implicitly negotiate task roles and decide who is responsible for what and when, as well as cooperatively learn different behavioral strategies to complete the task optimally.

Of specific interest in the current dissertation was identifying: (1) why and how dyads discovered the COC behavioral mode during ongoing activity, and (2) whether the S&R and COC behavioral modes observed in Nalepka, Kallen et al., (2017) generalize to other shepherding task contexts.

With regards to (1) why and how certain dyads discover and transition to COC behavior during ongoing activity, previous research on behavioral learning more generally has revealed that participants search for new, more optimal solutions when current strategies do not lead to task success (Jacobs, Runeson, & Michaels, 2001). Consistent with this, Nalepka, Kallen et al., (2017) 5

found that participants who completed the difficult 7-sheep condition (i.e., compared to the easier

3- and 5-sheep conditions) or participants who found successful S&R shepherding more difficult in general (i.e., exhibit overall poor S&R performance), were more likely to discover and transition to COC behavior. The implication is that increases in task difficulty might operate to promote the search for and discovery of COC behavior. Experiment 1 (Chapter 2) investigated this possibility with the expectation that task difficulty would act as “control parameter”, decreasing the stability of S&R behavior as task difficulty increased and, in turn, enhancing the potential emergence of

COC behavior.

Finding that task difficulty promotes COC discovery will provide insights into why dyads discover and transition to COC behavior. However, this potential finding would not answer the question of how dyads discovered COC behavior. Interestingly, work by Stephen, Boncoddo,

Magnuson and Dixon (2009) provide evidence that the transition to more efficient solutions may be the result of a re-organization of oculomotor behavior that may lead to the discovery of new behaviors. This re-organization is marked by a rise and subsequent fall of the complexity (entropy) of ocular behavior. Accordingly, the eye-movements of participants in Experiment 1 were also recorded to determine whether there were ocular behavioral bio-markers that might indicate or predict the discovery of COC. Based on research by Stephen et al., (2009), it was expected that changes in ocular entropy would occur prior to the transition to COC behavior.

Regarding (2) whether S&R and COC behavior generalize to different task contexts –

Experiment 2 (Chapter 3) investigated whether the S&R and COC behavioral modes observed in

Nalepka, Kallen et al., (2017) are unique to the task context employed in that work (i.e., tabletop shepherding), or whether they represent the “universal” behavioral modes of shepherding more generally and thus, should also be observed across different experimental contexts. Experiment 2 6

investigated this by requiring the full-body movement of participants to contain fleeing sheep in a large task space. It is well known that qualitatively similar movement solutions may emerge in different task contexts, so long as key physical and informational task constraints remain invariant

(i.e., if the same task-dynamics are maintained; Saltzman & Kelso, 1987; Turvey, 2007; Turvey,

1990; Warren, 2006). For example, the dynamics of coupled nonlinear oscillators define the task- dynamics of intrapersonal interlimb coordination (Haken et al., 1985), as well as the dynamics of two people coordinating their limb movements (Schmidt et al., 1990). These dynamics are also observed in other contexts (intentionally or unintentionally), such as a when individuals are walking side-by-side, or rock side-by-side in rocking chairs (Richardson, Marsh, Isenhower,

Goodman, & Schmidt, 2007; Schmidt & Richardson, 2008). In another set of examples, the hand movements of partners engaged in a pick-and-place task (Lamb et al., 2017) has been shown to reflect the same task-dynamics involved in human route navigation using full-body movement

(Fajen & Warren, 2003). Accordingly, the expectation in Experiment 2 was that the same modes of behavior observed with hand movements in Nalepka, Kallen et al., (2017), would be observed in a full-body movement context. Importantly, qualitative differences in the behavior modes observed would indicate differences in the underlying task-dynamic constraints that differentiated the environmental contexts.

CHAPTER 2

Experiment 1

Facilitating and Predicting the Emergence of Coupled Oscillatory Containment

Experiment 1 was designed to investigate how co-acting human agents discover and transition to new behavioral modes in a complex multi-agent task context. The shepherding paradigm, introduced in the first chapter, will be utilized to understand the processes that enable behavioral change in such social, goal-directed contexts. To understand how these processes unfold, the experiment seeks to investigate what variables or constraints (1) promote and (2) predict behavioral discovery and change.

Dynamics of Behavioral Change

How can the discovery of the more effective (potentially optimal), coupled oscillatory containment (COC) behavior be steered? The notion of “optimal control” for complex agent- environment systems is often difficult to reconcile as complex systems, by definition, are governed by nonlinear processes, with seemingly small or non-obvious factors often having large, unpredictable consequences for the organization and stability of system order. However, promoting the discovery of more stable or effective task solutions can be achieved by manipulating key task parameters—control parameters—that increase the relatively stability or existence of possible behavioral modes, while decreasing or extinguishing others (Kelso, 1995). For the shepherding task, Nalepka, Kallen et al., (2017) found that as the number of sheep that needed to be corralled and contained increased (i.e., 3 to 5 to 7 sheep), dyad search-and-recover behavior

(S&R) performance decreased and, moreover, dyads were more likely to discover and adopt COC

behavior. The implication is that manipulating task difficulty provides a way of enhancing the emergence of COC by decreasing the task stability of S&R behavior.

The stability of behavior, and the transition between behavioral modes, can be understood with tools and concepts from dynamical systems theory and dissipative systems. Research over the past 40 years has provided support for the notion that the human movement system is organized into coordinative structures, or synergies. That is, the components of the motor system (i.e., neurons, muscles, limbs, etc.) are temporarily constrained to function as task-specific units of organization (e.g., Kelso, 2009; Kugler, Kelso, & Turvey, 1980; Latash, 2008; Saltzman & Kelso,

1987; Turvey, 1990). The function of coordinative structures is to maximize interactions amongst components while minimizing intervention from the central nervous system, simplifying the control of the many degrees of freedom (DoF) of the human movement system (Bernstein, 1967;

Latash, 2008). The behavior of these synergies can be understood as living in an n-dimensional state space where each dimension represents a DoF of the coordinative structure. During task learning or ongoing performance, movement along these dimensions within the state space typically converges towards regions of stability (i.e., attractors) and away from regions of instability (i.e., repellors) (Kugler, Kelso, & Turvey, 1980). The regions of stability for these coordinative structures are the result of an interaction of animal, environmental and task constraints that define what behaviors are possible and or efficient (Newell, 1986). For example, the mode of locomotion selected for a quadruped (e.g., walk, trot, gallop) is the one that is most energetically efficient for the animal (an animal and environmental constraint) that satisfies the task constraint (i.e., the speed to achieve) (Collins & Stewart, 1993).

An avenue to control the behavioral mode that a coordinative structure exhibits is to manipulate the constraints acting upon the task-dynamic system. A well-studied example in the 9

coordination literature is the self-organization of stable modes in rhythmic limb movement. When participants are tasked to oscillate their index fingers, two stable modes of behavior are observed: in-phase, where the fingers are symmetrical in their movement (i.e., maintain a 0° phase relationship), and antiphase, where the fingers are at opposite points of the phase cycle (180°). At comfortable oscillatory frequencies, the system can exhibit both modes of coordination – the system is bi-stable and following a perturbation, the system will return to one of these two coordinative modes.

How can one bias the system to exhibit a specific mode of behavior? Control parameters are an avenue to both alter the attractor landscape of the stable end-state dynamics of a system, while simultaneously gaining an understanding of the underlying structure of the system itself

(Kelso, 1995). When task constraints are manipulated during the interlimb coordination paradigm, such that the needed tempo or frequency of oscillation is increased beyond a critical point, a bifurcation occurs (i.e., the attractor landscape has changed) and the attractor at antiphase becomes annihilated. As such, in-phase coordination becomes the only stable solution and individuals producing antiphase coordination spontaneously transition from antiphase behavior to in-phase behavior.

In the shepherding task, S&R and COC behavior are both stable states of behavior and represent a bi-stable system. By preferentially destabilizing S&R behavior with the use of a control parameter, it is expected that more dyads will transition to COC behavior as the more stable solution. As described above and in Chapter 1, the results of Nalepka, Kallen et al., (2017) suggest that task difficulty may operate as a potential control parameter, with increases in task difficultly reducing the stability of S&R behavior and enhancing the discovery of COC behavior.

Predictors to Behavioral Change

How can one predict when a behavioral mode or strategy will be discovered and utilized?

Changes in ocular behavior may predict or anticipate the adoption of new behavioral modes, because producing a stable behavior is co-specific with the ability of individuals to detect information that specifies or stabilizes that behavioral possibility (Riley, 2002; Turvey, Carello, &

Kim, 1990). A reason why in-phase and antiphase coordinative modes are stably produced, for example, is because there is information in the optic array that allows for the perceptual control of these modes of coordination (Bingham, 2004). Evidence for this is found when participants are tasked to learn an inherently unstable coordinative mode, such as maintaining a 90° phase relationship between the limbs (Schöner, Zanone, & Kelso, 1992). When participants are trained to differentiate a 90° phase relationship on a computer screen, this perceptual proficiency translated to the successful control of maintaining this relationship with the limbs (Wilson, Snapp-

Childs, Coats, & Bingham, 2010; Wilson, Snapp-Childs, & Bingham, 2010). Other work has demonstrated that learning difficult polyrhythm patterns, such as a 5:3 or 4:3 tapping pattern, can be produced easily when the perceptual consequences of one’s movement is transformed to reflect a 1:1 pattern; simplifying the variable that need to be controlled (Kovacs, Buchanan, & Shea, 2010;

Mechsner, Kerzel, Knoblich, & Prinz, 2001). In optical-based tasks, such as the shepherding task, ocular dynamics may display certain signatures as the agent is transitioning from detecting information that specifies one behavioral mode over to another.

A measure that may provide predictive power is the complexity or “entropy” of the dynamics of ocular behavior. Pertinent to this dissertation is the work by Stephen, Boncoddo,

Magnuson, & Dixon, (2009), who investigated ocular behavior during a moment of insight when solving a mathematical puzzle task. The task, referred to as the gear-problem task, required 11

participants to determine the spin direction of the final gear in a series (see Figure 3, left). At the start, participants utilized a strategy that involved tracing the spin directions of each gear to arrive at their answer. However, after some time, a subset of participants discovered a parity strategy

(i.e., determining if there were an even or odd number of gears) that was optimal, as measured by the number of seconds needed to complete a trial. Participants who discovered this parity strategy exhibited changes in the dynamics of ocular-movement prior to realizing and transitioning to the new strategy. Specifically, an increase in eye movement entropy, as measured by recurrence quantification analysis (see below), occurred prior to the discovery of the strategy (see Figure 3, right), followed by a subsequent decrease in entropy. This change in entropy is characteristic of a nonlinear phase transitions in behavioral stability more generally. For example, the Lorenz system

(a nonlinear system) exhibits a rise and fall in entropy during the transition from fixed point to limit cycle behavior (see Figure 4). Stephen et al., (2009) argued that like the phase transitions in nonlinear dynamical models (i.e., like the Lorenz system), participants perception of the gear- problem (and the subsequent strategy they use) undergoes a reorganization of behavior that lead to the discovery of the parity strategy as a stable solution. Adding noise to the system close to the critical point of transition (e.g., randomly moving the display) can also push the system to re- reorganize sooner (Stephen, Dixon, & Isenhower, 2009)

Figure 3. Example gear-problems from Stephen, Boncoddo et al., (2009) (left). Participants who discovered the parity (odd-even) strategy to solve the task had an increase and subsequent drop in entropy prior to the first use of the strategy (right).

Figure 4. Change in Entropy during a phase transition in a Lorenz system indexed using recurrence quantification analysis. Taken from Stephen, Dixon and Isenhower (2009).

Understanding System Dynamics with Recurrence Quantification Analysis. To understand what the Entropy measure employed by Stephen et al., (2009) means with respect to the dynamics of ocular behavior, a brief tutorial of recurrence quantification analysis (RQA) is needed. RQA gives insight into the properties of a dynamical system by measuring the patterning of trajectory re-visitation in a reconstructed n-dimensional state space (Webber & Zbilut, 2005).

The patterning of state space re-visitation is plotted on a 2-D recurrence plot (see Figure 5) where the n-dimensional reconstructed time series trajectory is plotted on each axis. The central diagonal or line of identity corresponds to self-recurrences and is therefore ignored, with the symmetrical structures either side of the line of identify examined to quantify the nature of the systems dynamics.

Figure 5. Example recurrence plot used for recurrence quantification analysis (RQA). Taken from Webber & Zbilut (2005).

Several measures can be obtained from the recurrence plot to quantify a system’s underlying dynamics. The first measure, %Recurrence, is simply the percentage of points that are considered recurrent (again, ignoring the line of identity). Best practices for human data suggest that RQA parameters be chosen so that the mean %Recurrence remains under 5% in order to detect subtle differences in system dynamics between trials (Shockley, 2005)—i.e., to avoid indexing false recurrences. %Recurrence has been shown to index the amount of coupling within a dynamic system (Shockley, Butwill, Zbilut, & Webber, 2002) and is sensitive to changes in the amount of noise (Richardson, Schmidt, & Kay, 2007). %Determinism is a measure of the percent of recurrent points that fall along diagonal lines and indexes how deterministic are the dynamics of the underlying system. Thus, a perfect sine wave for example would exhibit 100% determinism, whereas white noise would exhibit 0% determinism. %Determinism is expected to be affected by

task-difficulty – as task difficulty increases, %Determinism is also expected to increase which is consistent with past research that apply additional constraints that limit participant behavior

(Balasubramaniam, Riley, & Turvey, 2000; Davis, Pinto, & Kiefer, 2017). Exhibiting deterministic structure may be the result of a strategy to increase predictability considering perturbations induced by increasing task difficulty. Maxline represents the longest diagonal line length and is inversely related to the most positive Lyapunov exponent – greater values of Maxline indicate more stable systems (Riley & Turvey, 2002). In rhythmic bimanual coordination, for example, Maxline is found to be larger for in-phase coordination, than antiphase coordination, consistent with the notion that in-phase coordination is the more stable antiphase coordination

(Richardson et al., 2007; Schöner, Haken, & Kelso, 1986).

Finally, Entropy, the measure of most interest here, is the Shannon information (Shannon,

1948) entropy of the distribution of diagonal line lengths. Entropy is a measure of the complexity of coordination. Recent research has indicated that Entropy increases with task difficulty (Davis et al., 2017). More importantly, Entropy was found to change prior to the discovery of the new cognitive structure to solve a math-based gear problem-solving task investigated by Stephen et al.,

2009. Stephen et al., (2009) argued that entropy taps into the re-organization of a systems dynamical landscape. During a phase transition, constraints that bind components begin to break and re-organize. The consequence of this is an increase in Entropy, as these unchained components are free to move independently. The reduction in Entropy seen in Figure 3, is the result of new constraints being formed between components that increases the predictability of the system.

Current Experiment

Experiment 1 had two aims. First, to test a potential control parameter that enhances the emergence of COC behavior by destabilizing the utility of S&R behavior. The selected control 16

parameter was task difficulty. In Nalepka, Kallen et al., (2017), task difficulty was manipulated by changing the number of sheep that needed to be contained. To control for differences in inter- sheep collisions that were afforded to 7, but not 3 sheep, the same number of sheep was utilized here (i.e., 7 sheep). Instead, difficulty was manipulated by changing the maximum speed the sheep could move. S&R behavior involves the selection and pursuit of sheep furthest from the task goal.

By increasing sheep speed, the intended consequence is that S&R would be an ineffective strategy as corralling an individual sheep would allow other sheep to escape the task space sooner. This is also expected to have a consequence to increase task-related variability, thereby facilitating the discovery and transition to COC. Greater utilization of COC behavior was expected in high task- difficulty conditions (i.e., high maximum sheep speed) than in low task-difficulty conditions.

Consistent with previous research utilizing the shepherding task (Nalepka, Lamb, et al., 2017;

Nalepka, Kallen, et al., 2017), COC behavior was expected to produce more stable herding performance, by minimizing the herd area, the amount of herd travel and the mean sheep speed.

The second aim of the experiment was to assess oculomotor predictors for the transition and use of COC behavior. Consistent with prior research investigating variability prior to behavioral mode transitions, changes in the complexity (i.e., entropy) of ocular dynamics were expected to occur prior to a behavioral mode transition to COC behavior. In particular, the entropy of ocular dynamics is expected to increase and subsequently decrease prior to the discovery and use of COC behavior. Entropy of ocular movements was examined with the use of recurrence quantification analysis, consistent with methods used in prior research (Stephen, Dixon, et al.,

2009).

Method

Participants. Sixty-six right-handed undergraduate students from the University of

Cincinnati (M = 19.17 years, SD = 1.82), recruited into one of 33 dyads, participated in the experiment (50 female, 16 same-sex, same-race dyads). Participants received research credit towards completion of a Psychology course requirement.

Materials and Task. The shepherding task was reconfigured to accommodate the use of

Ergoneers Dikablis eye-tracking headsets (Ergoneers GmbH, Germany). The visual qualities of the task were made with high-contrast and the sheep were replaced with QR-markers for object identification in eye-data post-processing (see Figure 6). The task was redesigned to be projected on a translucent glass tabletop from a projector unit placed on the floor (Viewsonic PJD6683ws).

The displayed image measured 1.15 × 0.65 m. The view of the task is a top-down, orthographic view. The QR-marker sheep measured 22.09 cm2 and were placed on top of spheres, invisible to participants, to maintain the same sphere-to-sphere dynamics in Nalepka, Kallen et al., (2017).

The spheres had a radius of 4.7 cm. The participant-controlled cubes measured 8.63 cm2.

Participants controlled their respective cube by using a Polhemus LATUS motion-tracking sensor operating at 96 Hz (Polhemus Ltd. Vermont, USA). Participants moved these sensors on top of the glass tabletop to control their cube, which tracked closely underneath the Polhemus sensor. The task-space was surrounded by a white fence (thickness = 0.6 cm). If a sheep collided with this fence, the trial ended prematurely.

Figure 6. Depiction of experiment. Calibration and inter-trial screen (left), initial arrangement of objects at the start of a trial (right). Participants held hand-held sensors that controlled the blue (orange) square.

Before a trial, participants were shown the inter-trial/calibration screen which consisted of a QR-maker (edge = 11.75 cm) and four calibration crosses organized on the corners of a 47 ×

23.5 cm rectangle (see Figure 6, left). To initiate a trial, participants moved their sensor to their respective start location (x,y = 0, ± 33.5 cm). Once initiated, the seven QR-markers (sheep) appeared clustered on the ground between both participants (cluster center x,y = 0,0; cluster radius

= 9.4 cm) (see Figure 6, right). The sheep were repelled away from either participant when within

11.75 cm of the participant-controlled cube (the dynamics of this repulsion was consistent with the method reported in Nalepka, Kallen et al., [2017]). Participants were tasked to contain all the sheep within a radial distance of 14.1 cm, centered at the sheep’s center of mass. Participants received optical feedback when all the sheep were within this radial distance of each other by changing the color of the surrounding fence from white to green. Each herding trial was a maximum of one-minute long. Participants were informed that trials would end prematurely if any sheep collided with the surrounding fence and that successful trials were deemed to have occurred if the sheep were contained within the containment circle for at least 70% of the last 45 seconds

(i.e., for 31.5 seconds). Following a trial, the sheep would disappear, and participants moved to

their respective start location to initiate the next trial. The overall goal for participants was to complete eight successful trials within the 45-minute experimental period.

Design and Measures. Dyads were randomly assigned to one of two difficulty conditions.

Task difficulty was defined by the maximum speed the spherical sheep could move along the game

푐푚 푐푚 field: low difficulty = 6.9 and high difficulty = 16.1 . Dependent measures to assess 푠 푠 shepherding performance were the following: containment time (s) – the amount of time the sheep were within the required circular region (i.e., were red), sheep area/spread (cm2) – the area of the convex hull formed by all sheep (the convex hull is defined by the smallest convex, n-sided polygon that encompasses all objects, e.g., like a rubber-band around a set of objects), herd travel

푐푚 (cm) – the distance the herd’s center of mass (COM) moved during the trial, and sheep speed ( ) 푠

– the magnitude of the mean sheep’s velocity. Dependent measures to assess gross oculomotor

푓푖푥푎푡푖표푛 behavior were the following: normalized number of fixations ( ), mean fixation duration 푠

(ms), mean saccade angle (°), and standard deviation in saccade angle (°). Finally, in addition to measuring Entropy from recurrence quantification analysis, differences in %Recurrence,

%Determinism, normalized max line length and mean line length were also assessed to fully capture the oculomotor dynamics during the shepherding task.

Behavioral Mode Classification and Data Preprocessing. Search-and-Recover (S&R) and Coupled Oscillatory Containment (COC) behavioral mode classification was distinguished by taking the power spectra for the detrended and z-score normalized time-series of the angular movement with respect of the sheeps’ center of mass. MATLAB’s pwelch function was used with a 50% overlap window of 512 samples. For a given trial, the average peak frequency within the dyad was plotted versus the average number of directional changes the dyad exhibited for the given

trial (the time series was down-sampled to a 250 ms resolution to avoid misclassification in directional changes). See Figure 7 for a plot from the present experiment. The data were detrended from the expected number of direction changes for a given average frequency (orange line in

Figure 7, top). The data was then z-score normalized and then classified into two clusters using the k-means clustering algorithm from MATLAB (Mathworks, Natick, MA). See Figure 7, bottom for the classification result.

For oculomotor measures, the x-position (in pixels) of the pupil was used as the input variables for recurrence quantification analysis (RQA). The pupil x-position allowed for more variability due to the x-dimension of the task space being larger than the y-dimension. Time series from both participants were synchronized (error ≤ 17 ms). For each time series, outliers beyond 3 standard deviations beyond the mean were removed to account for large deviations due to eye- blinking. Linear interpolation was then applied to missing data and a Hampel filter was applied afterwards to smooth the time series, replacing outliers beyond 3 times the median absolute deviation with the local median (window of 6 samples). Finally, time series were trimmed to exclude missing values at the beginning and end of the time series.

140

120

100

40 Dyad Behavior

Number of Direction Changes Direction of Number 20 Expected COC Behavior 0 -0.4 0.1 0.6 1.1 1.6 Avg. Peak Frequency (Hz)

120

100

60 S&R 40 COC Expected 20

0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

Number of Direction Changes from Changesfrom Direction of Number -20

-40 Avg. Peak Frequency (Hz)

Figure 7. Behavior classification for Experiment 1. Trials that lasted at least 30 seconds were included. Average peak frequency was plotted versus the number of angular direction changes (at a 250 ms resolution). The orange line (top) indicates the expected number of directional changes if the average frequency was maintained for the duration of the trial. The bottom figure illustrates data linearly detrended from the expected number of directional changes and classified into one of two clusters from k-means clustering. 22

Recurrence Quantification Analysis. Time series were submitted to recurrence quantification analysis (RQA). Insight into the dynamics of higher dimensional state spaces, given a one-dimensional input signal, is possible due to the unfolding of delayed copies of the time series into higher dimensions (Takens, 1981). Assuming the inputted time series is coupled to the rest of the system, the time series contains information about the activity of other unmeasured components that impart influences on the measured time series at different time lags. To retrieve the most information about the influences of these other components, average mutual information is conducted at various sample lags to obtain a lag that yields the most information gain (by minimizing average mutual information). Once an appropriate sample lag t is obtained, a false nearest-neighbors analysis is conducted to uncover the dimensionality of the dynamic system

(Kennel, Brown, & Abarbanel, 1992). Here, dimensions k are added by taking the time series lagged at k × t until the percentage of data points that were previously near each other in a lower dimension no longer change when the new dimension is added. The reason for a “false” nearest neighbor is due to the problem of projection of higher dimensional objects to a lower dimension.

This allows you to uncover the dimensions of, for example, a moving body, when one is only given the shadow of that object. Here, what may look like an object moving in the y-direction may

(actually) be moving in the z-direction (such is the case of one’s shadow when one is jumping) when the appropriate dimension is recovered.

To select the appropriate lag and embedding dimension, every dyad time series that had at least 30 seconds of data (448 trials) were used. Average Mutual Information analysis was conducted at sample lags below 3 seconds and the lag that produced the first local minima was used, averaged within each dyad (M = 40 samples). Next, false nearest neighbors analysis was conducted. The percentage of false nearest neighbor was determined as the percentage of data 23

points whose distance to their nearest neighbor in dimension k +1 changed by 15 times the distance in dimension k. The embedding dimension for a given time series was determined by the dimension k where the change in false nearest neighbors in dimension k + 1 was less than 1% (M = 6 dimensions). For RQA, time series were z-score normalized. Data points were recurrent (i.e., occupy the same space) if their distance was within 10% of the maximum distance between data points.

Procedure. Following informed consent, participants were taken to the testing room, fitted with the eye-tracking headsets and calibrated, given the motion-trackers to their right hand, and then received task instructions. Dyads were told that they had 45 minutes to complete eight successful trials and that once eight successful trials had been completed the experiment would finish early in exchange for full research participation. This “game” type methodology for the shepherding task was used as an incentive for participants to work together. If participants did not complete eight successful trials the shepherding game would automatically end after 45 minutes.

Participants were told the rules of the task. They were informed that aim of the game was to herd the sheep together and then keep them together, and that the surrounding fence would turn green when they had been herded close enough together. The experimenter reinforced to participants that to move the sheep, participants had to position their cube (controlled by their hand-held controller) so that the sheep would be repelled away from them. The experimenter also reinforced that there was no pre-defined goal location – only that the sheep needed to be herded together somewhere on the task field.

At the end of each one-minute trial, the trial “score’ (i.e., percentage of time herded together) was displayed to participants on the table. Participants were informed that a successful herding trial required a score of 70 or better. If a trial ended prematurely due to a sheep exiting the 24

task field, a “Try Again!” message would instead appear. At the end of each trial, participants initiated the next trial by moving to their respective start location. Finally, participants were informed that they could not communicate during the task or during any breaks and the experimenter in the room would enforced this no-talking policy.

Results

Sixteen dyads were in the low task-difficulty conditions and 17 in the high task-difficulty condition. 15 (93.75%) and 11 (64.71%) dyads in the low and high-task-difficulty condition, respectively, achieved the 8 points needed to complete the experiment (time to completion: low task-difficulty M = 14.46 min, SD = 5.19; high task-difficulty M = 15.98 min, SD = 6.85). The proportion of successful dyads was not equally distributed across condition, X2 (1, N = 33) = 4.16, p = .04, as fewer dyads in the high task-difficulty condition were able to meet task expectations, although those who did succeed did so at a time comparable to those in the low task-difficulty condition, t(24) = -.65, p = .53. Shepherding performance during failure trials was also worse for dyads in the high task-difficulty condition. Independent samples t-tests were conducted for every dyad that had at least 1 failure trial (all but 1 dyad in the low task-difficulty condition met this criteria). Dyads in the high task-difficulty condition contained the sheep for less time (M = 9.86 s,

SD = 5.76; low task M = 16.56 s, SD = 6.17), t(30) = 3.18, p = .003, kept them to a larger area (M

= 1145.53 cm2, SD = 497.20; low task M = 494.55 cm2, SD = 433.48), t(30) = -3.92, p < .001, and

푐푚 were unable to keep the sheep’s speed under control (M = 4.62 , SD = 0.64; low task M = 3.99 푠

푐푚 , SD = 0.39), t(30) = -3.30, p = .003. The amount of herd travel was less for dyads in the high 푠 task-difficulty condition (M = 60.57 cm, SD = 10.10; low task M = 70.29 cm, SD = 9.16), t(30) =

2.84, p = .008, possibly due to the sheep equally dispersing from the center causing the herd’s

center of mass to remain relatively stable. Manipulating the maximum sheep speed was effective in increasing task difficulty by increasing the number of dyads who were unable to complete the shepherding task.

Effect of Task Difficulty on Successful Shepherding Performance and Use of COC.

The first question was whether increasing task difficulty by manipulating maximum sheep speed was a sufficient control parameter to aid in the destabilization of S&R behavior and aid in the discovery and adoption of COC behavior. Only dyads who achieved at least 1 successful trial were included in analysis (2 dyads in the high task-difficulty condition were excluded). There were no significant differences between task difficulty groups on containment time, t(29) = .70, p = .49, sheep area/spread, t(29) = -.3, p = .77, or herd travel, t(29) = 1.55, p = .13. Dyads in the high task-

푐푚 difficulty group contained the sheep to a lower mean speed (1.73 , SD = 0.79) than those in the 푠

푐푚 low task-difficulty group (2.27 , SD = 0.78), although not statistically significant, t(29) = 1.91, 푠 p = .07.

Seven (43.75%) of 16 dyads in the low-task condition utilized COC at least once successfully while 10 out of 15 (66.67%) of dyads in the high task-difficulty condition utilized

COC behavior at least once. This distribution of use of COC was greater for dyads in the high task- difficulty condition, although not statistically significant, X2 (1, N = 31) = 1.64, p = .2, η = .23.

Dyads in the low task-difficulty condition utilized COC 32.03% of all successful trials (SD = 0.39) while those in the high task-difficulty condition utilized COC more (M = 48.33% of trials, SD =

0.40). A Mann-Whitney test indicated a trend towards a significant difference, U = 91.50, p = .24.

Although dyads in both task-difficulty conditions utilized COC in statistically equal proportions, the mean peak frequency of oscillation may be impacted by the difficulty of the task. An

independent samples t-test was conducted to test this possibility and found no difference in mean peak frequency between task-difficulty conditions, t(15) = -.45, p = .66 (low task-difficulty M =

1.81 Hz, SD = 1.30; high task-difficulty M = 0.86, SD = .19).

Effects of Behavioral Mode on Successful Containment. Paired samples t-tests were conducted to verify that COC behavior was more optimal in containing the sheep than S&R behavior during successful performance. For dyads that exhibited both successful S&R and COC trials (low task-difficulty N = 6; high task-difficulty N = 7), COC behavior proved to be more effective in sheep containing on all dependent measures (see Figure 8 for summary plots). Dyads performing COC behavior contained the sheep together for longer (M = 44.44 s, SD = 1.15; S&R

M = 41.27, SD = 2.92), t(12) = -3.98, p = .002, contained them to a smaller area (M = 71.67 cm2,

SD = 14.53; S&R M = 136.86 cm2, SD = 61.28), t(12) = 3.89, p = .002, minimized the sheeps’ center of mass movement (M = 29.42 cm, SD = 8.11; S&R M = 50.99, SD = 15.94), t(12) = 5.44,

푐푚 푐푚 p < .001, and kept the sheep to a lower mean speed (M = 1.05 , SD = 0.30; S&R M = 2.22 , 푠 푠

SD = 0.74), t(12) = 5.64, p < .001.

Figure 8. Summary plots depicting within-subject differences in behavioral mode on shepherding performance. Error bars represent standard error. ** = p < .01, *** = p < .001.

Effects of Behavioral Mode on Ocular Descriptive Variables. Within-subject analyses of variance (ANOVAs) were conducted to describe oculomotor differences between behavioral modes when completing the shepherding task. Dependent measures were: the normalized number of fixations, mean fixation duration, mean saccade angle, and the variability of saccade angle.

Ocular behavior during successful COC trials were compared to successful S&R trials.

Unsuccessful trials were added as a third level for baseline comparisons. Dyads who had each of these trial types and had 30 seconds of data following post-processing were included in analysis

(low task-difficulty N = 4; high task-difficulty N = 7). Plots summarizing results can be found in

Figure 9. Differences within trial type (failure, correct S&R, correct COC) were detected across several measures. Normalized fixation number was smallest during successful COC trials (M = .83

푓푖푥푎푡푖표푛 푓푖푥푎푡푖표푛 , SD = .20), followed by successful S&R trials (M = 1.11 , SD = .11), and then 푠 푠

푓푖푥푎푡푖표푛 2 failure trials (M = 1.35 , SD = .13), F(2, 20) = 41.68, p < .001, ηp = .81. All comparisons 푠 were significant at p ≤ .006. Accordingly, mean fixation duration was greatest during successful

COC trials (M = 1048.68 ms, SD = 268.27), followed by successful S&R trials (M = 844.07 ms,

2 SD = 256.25), and then failure trials (M = 628.10 ms, SD = 101.61), F(2, 20) = 14.47, p < .001, ηp

= .59. Comparisons between failure trials with S&R and COC trials were significant at p < .02. No significant difference between S&R and COC trials (p = .13). Mean saccade angle differences were due to greater saccade angles during failure trials (M = 6.79°, SD = 1.10) compared to either successful trial type (S&R: M = 5.14°, SD = .80; COC: M = 4.87°, SD = 1.03) (comparisons with failure at p < .001. No differences between S&R and COC behavior [p = .92]), F(2, 20) = 26.48,

2 p < .001, ηp = .73. Finally, the standard deviation of saccade angle followed the same pattern, with performance during failure trials producing greater variability in saccade angle (M = 5.16°, SD =

1.10) compared to S&R (M = 3.76°, SD = 1.07) and COC performance (M = 3.71°, SD = 1.32) (all comparisons against failure trials at p ≤ .001. No difference between S&R and COC trials [p =

2 1.00]), F(2, 20) = 14.04, p < .001, ηp = .58.

Figure 9. Summary plots depicting within-subject differences on ocular descriptive measures for unsuccessful performance, successful S&R performance, and successful COC performance. Error bars represent standard error. *** = p ≤ .001.

Effect of Behavioral Mode on Ocular Dynamics. Next, within-subject ANOVAs were conducted to assess the differences in ocular dynamics on the behavioral mode utilized in the shepherding task as assessed by RQA. The ocular dynamics of successful COC trials were compared to successful S&R trials and failure trials (for baseline comparison). Dyads who had each of these trial types and had 30 seconds of data following post-processing were included in analysis (low task-difficulty N = 4; high task-difficulty N = 7). The following dependent measures computed from auto-recurrence analysis were tested: %Recurrence, %Determinism, normalized maximum line length and Entropy. Measures were averaged across partners for a mean dyad score.

Figure 10. Summary plots depicting within-subject differences on ocular dynamics for unsuccessful performance, successful S&R performance, and successful COC performance. Error bars represent standard error. *** = p ≤ .001.

Summary of results are presented in Figure 10. The ocular dynamics during successful

COC trials were less deterministic (M = 92.80%, SD = 1.92) than during successful S&R trials (M

= 94.68%, SD = 1.55) (p = .054) and during failure (M = 95.80%, SD = 1.05) (p = .001), F(2, 20)

2 = 14.11, p < .001, ηp = .59. Differences between S&R trials and failure was not significant (p =

2 .14). No significant difference in Entropy were detected, F(2, 20) = 2.32, p = .12, ηp = .19, nor for any of the other measures.

Second Round: Effects of Behavioral Mode on Ocular Descriptive Variables. A second round of analyses were conducted utilizing the original trial classification criterion used in

Nalepka, Kallen et al., (2017) instead of the classification method introduced in this experiment.

The original criterion classified trials as predominately exhibiting COC behavior if the average 31

peak oscillatory frequency was ≥ 0.5 Hz. Using this criterion, the sample size for analyses assessing oculomotor differences increased from 11 to 14 dyads. Note that each trial classification method measures different aspects of COC behavior. The classification method introduced in this experiment favors trials that have low variability in oscillatory behavior, while the 0.5 Hz cutoff criterion favors any behavior that is oscillatory. Using the 0.5 Hz criterion did not influence the results in the shepherding performance measures discussed above but led to differences in analyses involving ocular measures. The following addresses these differences.

Within-subject ANOVAs were again conducted to describe oculomotor differences between behavioral modes when completing the shepherding task. Dyads who had at least one failure, S&R and COC trial and had 30 seconds of data following post-processing were included in analysis (low task-difficulty N = 5; high task-difficulty N = 9). Plots summarizing results can be found in Figure 11. Differences between trial type (failure, correct S&R, correct COC) were detected across all assessed dependent measures. Normalized fixation number was smallest during

푓푖푥푎푡푖표푛 successful COC trials (M = .89 , SD = .32), followed by successful S&R trials (M = 1.12 푠

푓푖푥푎푡푖표푛 푓푖푥푎푡푖표푛 , SD = .25), and then failure trials (M = 1.36 , SD = .22), F(1.11, 18.21) = 37.53, p 푠 푠

2 < .001, ηp = .74 (Greenhouse-Geisser corrected). All comparisons were significant at p ≤ .005.

Accordingly, mean fixation duration was greatest during successful COC trials (M = 998.74 ms,

SD = 305.92), followed by successful S&R trials (M = 793.77 ms, SD = 256.41), and then failure

2 trials (M = 611.27 ms, SD = 129.67), F(2, 26) = 19.34, p < .001, ηp = .60. All comparisons were significant at p ≤ .024. Mean saccade angle differences were due to greater saccade angles during failure trials (M = 6.98°, SD = .98) compared to either successful trial type (S&R: M = 5.28°, SD

= .97; COC: M = 4.61°, SD = .89) (comparisons with failure at p < .001. No differences between

2 S&R and COC behavior [p = .11]), F(2, 26) = 40.25, p < .001, ηp = .76. Finally, the standard deviation of saccade angle followed the same pattern, with performance during failure trials producing greater variability in saccade angle (M = 5.31°, SD = .1.09) compared to S&R (M =

4.01°, SD = 1.41) and COC performance (M = 3.26°, SD = .93) (all comparisons against failure trials at p ≤ .001. No difference between S&R and COC trials [p = .28]), F(2, 26) = 18.88, p <

2 .001, ηp = .59.

Figure 11. Summary plots depicting within-subject differences on ocular descriptive measures for unsuccessful performance, successful S&R performance, and successful COC performance using the 0.5 Hz cutoff criterion for COC trials. Error bars represent standard error. *** = p ≤ .001.

Second Round: Effect of Behavioral Mode on Ocular Dynamics. Next, within-subject

ANOVAs were conducted to assess the differences in ocular dynamics on the behavioral mode utilized in the shepherding task as assessed by RQA. Again, trials were classified utilizing the 0.5

Hz cutoff criterion utilized in Nalepka, Kallen et al., (2017). The ocular dynamics of successful 33

COC trials were compared to successful S&R trials and failure trials (for baseline comparison).

Dyads who had each of these trial types and had 30 seconds of data following post-processing were included in analysis (low task-difficulty N = 5; high task-difficulty N = 9). The following dependent measures computed from auto-recurrence analysis were tested: %Recurrence,

%Determinism, normalized maximum line length, and Entropy. Measures were averaged across partners for a mean dyad score.

Figure 12. Summary plots depicting within-subject differences on ocular dynamics for unsuccessful performance, successful S&R performance, and successful COC performance using the 0.5 Hz cutoff criterion for COC trials. Error bars represent standard error. * = p < .05, ** = p < .01, *** = p < .001.

Summary of results are presented in Figure 12. The ocular dynamics during successful

COC trials were less deterministic (M = 92.21%, SD = 2.19) than during successful S&R trials (M

= 94.26%, SD = 1.91) (p = .004) and during failure (M = 95.48%, SD = 1.29) (p < .001), F(2, 26)

2 = 25.52, p < .001, ηp = .66. S&R trials were less deterministic than trials resulting in failure (p =

.04). The coordinative dynamics during successful COC trials were also less complex, as measured by Entropy, (M = 3.78 bits, SD = .26) compared to the dynamics during task failure (M = 4.03 bits,

2 SD = .19) (p = .007), F(2, 26) = 5.18, p = .01, ηp = .29. There was no difference between COC trials and S&R trials (M = 3.91, SD = .35) (p = .4), nor between S&R and during failure (M = 4.03, p = .19) (p = .54).

Predictors for Discovery and Use of COC. Differences in ocular dynamics were found between trial types for %Determinism and in Entropy when using the 0.5 Hz criterion. However, can these dynamics predict the transition to COC behavior? Within-subject ANOVAs were conducted with the independent variable time-point (4 levels: 3, 2, and 1 trials before first COC use, and the first COC trial). To improve the sensitivity to uncover changes in dynamic variables prior to COC use, all trials that lasted at least 15 seconds were included in analysis. All dyads who had at least one successful COC trial were included in analysis (low task-difficulty N = 6, high task-difficulty N = 10). See Figure 13 for summary of results. Differences for %Determinism was

2 found leading up to the first successful use of COC behavior, F(3, 45) = 16.66, p < .001, ηp = .53, such that %Determinism was greatest (M = 96.36%, SD = 1.45) on the second to last 15-second trial before first successful use of COC. Compared to this time-point, %Determinism was significantly less on the trial before first success (M = 94.44%, SD = 2.13) (p = .01) and the first successful trial (M = 93.02%, SD = 2.42) (p < .001). %Determinism on the first successful COC trial was also less than %Determinism on the third trial before success (M = 95.52%, SD = 2.08)

(p < .001).

Figure 13. Summary plots depicting ocular dynamics at different time-points (on the 3rd, 2nd and last trial before first COC success [labeled 1]) using the 0.5 Hz cutoff criterion for COC trials. Error bars represent standard error. * = p < .05, *** = p < .001.

Importantly, changes in Entropy from RQA showed significant differences across the four

2 time-points, F(3, 45) = 5.97, p = .002, ηp = .29. Entropy was greatest on the second to last trial before COC success (M = 4.19 bits, SD = 0.38) than both the trial before (M = 3.90 bits, SD = .32)

(p = .02) and the first successful COC trial (M = 3.87 bits, SD = 0.36) (p = .02). In summary,

%Determinism and Entropy was significantly reduced between the second to last to last trial before

the first use of COC behavior, showing support that these measures afford predictive power in detecting the transition to COC behavior. Note, however, that these differences were only detected when COC trials were classified using the 0.5 Hz cutoff criterion. No differences were detected when using the criterion introduced in the Method section of this experiment.

Figure 14. Summary plots depicting ocular descriptive measures at different time-points (on the 3rd, 2nd and last trial before first COC success [labeled 1]) using the 0.5 Hz cutoff criterion for COC trials. Error bars represent standard error. * = p < .05, *** = p ≤ .001.

How do the summary nonlinear measures from RQA compare to more traditional oculomotor measures (e.g. fixation duration, saccade angle variability) in predicting the discovery of COC? Additional within-subject ANOVAs were conducted to detect trial-to-trial differences between normalized fixation number, fixation duration, saccade angle, and standard deviation in saccade angle for the three trials leading to and including the first successful COC trial. Summary 37

plots are found in Figure 14. Significant differences were detected within the variable time-point

(3, 2, and 1 trials before first COC use, and the first COC trial) on all measures, F(3, 45) ≥ 4.95, p

2 ≤ .005, ηp ≥ .25. However, none of these measures had significant changes prior to the first successful COC trial (all p ≥ ,18). Instead, the differences were due to differences with the first successful COC trial, affording no predictive power for these descriptive ocular measures.

Discussion

The aims of Experiment 1 were to investigate the conditions which promote the discovery and transition to COC behavior, as well as to determine oculomotor predictors that anticipate its emergence. As expected, the results revealed that manipulating the maximum speed the sheep could move could act as a control parameter promoting a search towards more optimal solutions, in this case COC behavior. Second, significant changes in oculomotor entropy occurred prior to the first use of COC behavior. However, this difference depended on how COC trials were classified. A discussion of dyad shepherding performance and ocular dynamics are discussed in turn.

Behavioral Dynamics. Dyads were able to discover and transition to the use of COC behavior. Consistent with Nalepka, Kallen et al., (2017), COC behavior was superior to S&R behavior, not only producing a higher score, but also maintaining the sheep to a smaller area within the target containment region. On average, a greater number of dyads in the high task-difficulty condition discovered COC behavior and had more successful COC trials than dyads in the low task-difficulty group. However, this difference was not significant. Thus, manipulating task difficulty could act as a control parameter to promote the discovery and transition to COC behavior. This question will be pursued further in Experiment 2.

The task difficulty manipulation was designed to preferentially destabilize the effectiveness of S&R behavior specifically, as it would require more active movements to ensure no sheep escaped the task space. Observations of participant performance indicated that a solution to corral sheep in the high task-difficulty condition was to keep the speed of the sheep to a manageable value where one could then switch between multiple sheep to contain them in the target containment region. This strategy was necessary to ensure that sheep did not gain too much momentum and, in turn, become impossible to corral (as it would hit the bordering fence before participants could apply enough repulsion). Some evidence for this is provided by the fact that dyads appeared to limit sheep speed in the high task-difficulty condition—dyads in the high task-

푐푚 푐푚 difficulty condition kept the sheep to a mean speed of 1.73 (SD = 0.79), compared to 2.27 푠 푠

(SD = 0.78) for dyads in the low task-difficulty condition. Although not statistically significant (p

푐푚 = .07), that is still potentially meaningful, as sheep were able to move up to 16.1 in the high 푠

푐푚 task-difficulty condition (compared to 6.9 in the low-task difficulty condition). It is also 푠 possible that the pre-COC back-and-forth movement between sheep to limit their speed may have also enhanced the discovery of stable COC behavior in the high task-difficulty condition.

Ocular Dynamics. Oculomotor results were dependent on the method of classifying COC trials. These differences, however may be due to a small sample size as the pattern of results are similar across classification criteria (14 dyads are included as opposed to 11 dyads when the 0.5

Hz cutoff criterion was utilized). Utilizing the 0.5 Hz criterion, distinct differences were found in oculomotor behavior between S&R and COC behavior. Ocular behavior during COC trials involved fewer fixations for a longer duration. This suggests that participants treated the sheep as a “collective” as opposed to individual agents – the number of fixations decreased, and the average

fixation duration increased as compared to S&R behavior. However, the result of this behavior may be due to a circular process – COC behavior kept the sheep to a smaller area and speed which may have led to fewer saccades compared to S&R behavior.

No statistical differences in %Recurrence and Maxline in both intrapersonal and interpersonal coordination indicated that the stability and coupling of the dynamic system did not differ between trial types. Instead, the complexity of coordination, as assessed by %Determinism and Entropy was impacted by trial type. The decrease in %Determinism in both intrapersonal and interpersonal coordination may be a consequence of a relaxation of the dynamics from difficult task constraints. This is consistent with previous research demonstrating differences in

%Determinism as a function of changes in task difficulty. In postural tasks, deterministic structure under task constraints may be a movement strategy for the postural system to maintain predictable balance to achieve task demands without falling. For example, postural stability has been shown to be more deterministic compared to quiet standing when participants had to perform a cognitive task (Pellecchia & Shockley, 2005). In a supra-postural pointing task, postural stability was also shown to be more deterministic along the dimension responsible for maintaining an upright stance without falling (Balasubramaniam et al., 2000). This deterministic structure was also found for patients with Parkinson’s disease compared to healthy adults for maintaining an upright stance

(Schmit et al., 2006). Both %Determinism and Entropy also increased during quiet standing on increasingly difficult surfaces (Mazaheri, Salavati, Negahban, Sanjari, & Parnianpour, 2010).

Within the joint action literature, %Determinism and Entropy have been found to increase in a joint supra-postural task as task difficulty is increased (Davis et al., 2017). In a two-person model car construction task, %Determinism was greatest in over-constrained turn-taking conditions compared to where the players’ roles were undefined. In tasks where ocular behavior 40

was recorded, %Determinism was found to differ between novices and experts looking at medical images, such that experts had less deterministic searching behavior (Vaidyanathan, Pelz, Alm, Shi,

& Haake, 2014). Consistent with what has been found elsewhere in the literature, deterministic structure in the shepherding task during failure and S&R trials may be due to a highly task- constrained system whose behavior is dictated by the functioning of individual sheep (see Figure

15, right for differences in %Determinism before and after the discovery of COC behavior). Once these constraints were relaxed, a search for more effective herding solutions may have been afforded to participants.

Figure 15. Changes in Entropy and %Determinism from 8 trials before first successful use of COC (labeled 1) to the subsequent 2-8 trials that followed. COC trials were determined using the 0.5 Hz cutoff criterion for the above plots. Trials that lasted at least 15 seconds were considered, and all dyads who had partial data were included who had at least 1 successful COC trial. Red line is drawn to assist in visualizing behavior prior to first COC success (left of red line), and trials after (and including) the first COC success (right of red line). See text for more details.

Ocular Predictors to COC. %Determinism and Entropy showed significant changes prior to COC discovery while the more traditional measures (i.e., fixation number, duration, saccade angle and variability in saccade angle) did not. The relaxing of task-constrained dynamics may have afforded participants with freedom to detect information to guide the use of COC behavior

(Jacobs & Michaels, 2007). However, not all dyads succeeded the task, especially when in the high

task-difficulty condition. The ability to utilize COC behavior may be an example of a hierarchical process that can only be detected during S&R behavior (Saltzman & Caplan, 2015; Wagman,

Caputo, & Stoffregen, 2016). This is worth noting because at the beginning of a trial, the sheep were already located within the containment area. The discovery and use of COC behavior, then, appears to be emergent from participant interactions during unsuccessful trials on the shepherding task. Although Entropy did significantly change prior to the discovery of COC behavior, the characteristic rise and fall of Entropy was not detected as seen in Stephen et al., (2009) using the gear-problem task (see Figure 3). Although this may be a statistical issue due to sample size, as a

“bump” can be observed when partial data is included (see Figure 15, left). Future work will attempt to fine-tune windowed epoch analysis of Entropy within trials to provide more statistical certainty for when dyads begin to discover and implement COC behavior. The windowed analysis is expected to detect critical fluctuations in system behavior, a defining behavior of a system prior to a phase transition (e.g., Scholz, Kelso, & Schöner, [1987] for interlimb coordination).

Finally, the use of COC behavior was found in both task-difficulty conditions. It may appear counter intuitive why dyads in the low task-difficulty condition would transition to COC behavior as it required more active arm movements (M = 24.62 m travel, SD = 6.58) compared to

S&R behavior (M = 13.86 m travel, SD = 5.98). The transition to COC behavior in the easy task- difficulty condition may be the result of the system preferentially selecting solutions that maximizes prediction, or reduces the effects of perturbation, as opposed to minimizing energy output of the arm movement (Huang, Kram, & Ahmed, 2012; Kistemaker, Wong, & Gribble, 2010;

Nasseroleslami, Hasson, & Sternad, 2014; Thorp, Kording, & Mussa-Ivaldi, 2017). A potential consequence is a reduction in needed sensorimotor processing (Koeppen, Huber, Sternad, &

Hogan, 2017). This would suggest COC behavior, once discovered, is not only more efficient, but 42

also provides simpler less explicit control compared to S&R behavior. Support for this can be found in the oculomotor behaviors in dyads - COC behavior required fewer saccades compared to

S&R behavior as the sheep were kept more closely contained with a lower mean speed.

CHAPTER 3

Experiment 2

Shepherding Dynamic Similitude to Full-Body Movement

The aim of Experiment 2 was to determine whether the dynamics of “herding” seen in

Nalepka, Kallen, et al., (2017) also define more realistic herding scenarios that involved the use of full-body motion to contain sheep. Specifically, Experiment 2 was designed to test whether S&R and COC behavior are invariant across different herding contexts—i.e., “universal” herding behaviors defined by the task-dynamics of “herding” more generally. Indeed, qualitative behavioral similarities are found in various contexts where the task dynamics remain invariant.

For example, maintaining in-phase or antiphase behavior between the limbs of one individual defines limb coordination between two people (Schmidt, Carello, & Turvey, 1990; Schmidt &

Richardson, 2008). In another example, the dynamics that result when one steers towards a goal while locomoting (Fajen & Warren, 2003) is also found when one steers their hand to a goal location in a pick-and-place task (Lamb et al., 2017). Below, discussed in turn, is that (1) qualities of S&R and COC behavior found in Nalepka, Kallen et al., (2017) also appear to be exhibited in herding behaviors in different agent-environment contexts (i.e., sheepdogs and wolves). Further,

(2) additional support for the potential universality of S&R and COC behavior across task contexts is underscored by the notion of dynamic motor primitives (Ijspeert, Nakanishi, Hoffmann, Pastor,

& Schaal, 2013), which correspond to the basic building blocks of behavior and may underlie the composition of all synergistic perceptual-motor behaviors.

Dynamic Similitude of Shepherding and Herding Behavior

An appropriate starting point to determine whether dynamics of S&R and COC behavior define full-body shepherding contexts is to understand how animals, in-fact, herd and contain other agents. The first example, fittingly, comes from shepherding by sheepdogs itself. Work by

Strömbom et al., (2014) demonstrated that sheepdogs also exhibit similar modes of behavior to those observed in Nalepka, Kallen et al., (2017). Strömbom et al., (2014) refered to these modes as collecting and driving behavior. Collecting behavior involves the pursuit of the sheep that is furthest from the herds’ center-of-mass and repels them towards the rest of the herd. This behavioral mode is not unlike the S&R behavior described in Nalepka, Kallen et al., (2017) and involves the same pursuit of sheep furthest from the center of the containment region. Once the herd is sufficiently collected, sheepdogs then swtich to driving behavior, which involves the dogs moving behind the herd so as to repel the herd towards the goal location. When one plots the timeseries of simulations of the sheepdog’s performance, the alternation between collecting and driving behavior produces long time-scale oscillatory behavior (see Figure 16 for a simulation of this). The reason for this is that as the sheepdog drives the herd, sheep at the flanks of the herd begin to be pushed laterally from the herd, causing the sheepdog to transition to collecting behavior to correct for the perturbations caused by driving behavior. Here, oscillatory behavior is a natural result of the interaction between the sheepdog and the herd, and does not need to be explicitely programmed.

Figure 16. Simulations of sheepdog behavior. The sheepdog alternates between “collecting” and “driving” behavior to herd the sheep. Taken from Strömbom et al., (2014).

Another relevant example is work done on the emergent hunting strategies of wolves

(Muro, Escobedo, Spector, & Coppinger, 2011). The coordination of multiple wolves to hunt down prey larger than an individual wolf may suggest that communication and planning between the wolf agents are required, but many aspects of wolf-pack hunting could be explained through two simple rules for each wolf agent: 1) pursue the prey until a safe distance is reached close to the animal and 2) once this safe distance is reached, move away from neighboring wolves. Here, the other wolves are treated as repellors for the individual wolf-agent. These rules, when simulated, are able to capture seemingly complex phenomena such as ambushing and relay hunting (where wolves appear to alternative between their respective roles in a hunt). Although wolf-pack hunting does not produce the robust rhythmic behavior seen in Nalepka, Kallen et al., (2017), both wolf- pack hunting behavior and participant behavior in the shepherding task require the herding agents to form a “containment wall” around the prey/sheep, thereby cancelling any repulsive perturbations caused by any singular agent.

Dynamic Motor Primitives of Task-Dynamics

Human behavior can be classified into one of two general classes of movement: discrete movement (e.g., reaching, grasping) and rhythmic behavior (e.g., walking, chewing), whose functioning are separable and distinct (Schaal, Sternad, Osu, & Kawato, 2004). These dynamic motor primitives are argued to be the basic building blocks that compose human motor behavior

(e.g., writing in cursive is one example of a complex action requiring both discrete and rhythmic movements) (Hogan & Sternad, 2007, 2012; Ijspeert et al., 2013). The benefit for such simplicity is that these behaviors can be modeled as either point or limit-cycle attractors, allowing complex behavior to be modeled by a series of damped-mass springs and oscillators common in the dynamical systems modeling toolbox. Such modeling efforts have been fruitful in the motor behavior literature, including, but not limited to models of human reaching and object interception, and route navigation and obstacle avoidance (Fajen & Warren, 2003; Saltzman & Kelso, 1987;

Zaal, Bootsma, & van Wieringen, 1999). Further, discrete and rhythmic behaviors define S&R and

COC behavior, respectively. Indeed, S&R and COC behavior has been modeled as a nonlinear oscillator with parameter dynamics controlling the amount of excitation/inhibition of the oscillator to enable it to resemble both fixed-point and rhythmic behavior (Nalepka et al., 2016; Nalepka,

Kallen, et al., 2017). Therefore, a general prediction could be that the transition from discrete to rhythmic dynamical primitives could define the behavioral modes in herding more generally, and that the use of a control parameter should facilitate the transition between primitives. For example, in an unimanual Fitts-like task where participants were tasked to move a pointer between two locations, human behavior transitioned to oscillatory behavior as task difficulty (defined as time to reach target) increased (Sternad et al., 2013).

Current Experiment

As mentioned previously, this experiment investigated whether the S&R and COC modes of behavior observed in Nalepka, Kallen et al., (2017) are not specific to hand movements on a tabletop display, but are task-specific realizations of the general class of shepherding or herding behaviors. In Experiment 2, participants, recruited into dyads were instructed to complete the same shepherding task described in Nalepka, Kallen et al., (2017) but in an immersive, first-person virtual reality environment. Instead of holding hand-held controllers, participants moved by running around the task space. Given the same general task goal as in Nalepka, Kallen et al.,

(2017), the same modes of behavior (S&R and COC) were expected to emerge. As in Experiment

1, task difficulty was manipulated by controlling the maximum speed the sheep could move. This served to further test whether manipulating maximum sheep speed could be a useful control parameter to promote the use of more coordinated modes of behavior. As in Experiment 1, increases in task difficulty (i.e., higher maximum sheep speed) were expected to have a negative effect on S&R herding performance, and, in turn, promote the emergence of COC-like behavior.

Method

Participants. Eighty undergraduate students from the University of Cincinnati (M = 18.93 years, SD = 0.90), recruited into one of 40 dyads, participated in the experiment (50 female, 16 same-sex, same-race dyads). Participants received research credit towards completion of a

Psychology course requirement.

Figure 17. Depiction of virtual task field for Experiment 2.

Materials and Task. The shepherding task was designed as an immersive virtual reality experience (Unity 5.6.12, Unity Technologies. San Francisco, USA) where dyads were able to locomote in a shared physical and virtual space (6 × 3.48 m) to retrieve and contain autonomously moving spherical objects (.24 m radius) or sheep (see Figure 17). A WLAN connection was utilized to transfer the participant and objects’ positions to each other using Unity’s UNET server- authoritative networking protocol. Participants were allowed free movement within an 8 × 6 m space (more than the 6 × 3.48m required for the task). Participants wore portable computer backpacks (MSI VR-One, Micro-Star International, Taiwan) equipped with a HTC Vive virtual- reality headset (HTC Inc., Taiwan). When worn, participants were able to see their partner in the shared virtual task environment. To ensure participant safety, a fence barrier would appear to participants as they were exiting the safe tracking space. This barrier was placed outside the task space and did not interfere with the task. Participants could walk outside the task space but were instructed to never pass the safety barrier.

Before a trial, participants moved to a start location (x,y = 0,±1.2) on either side of the virtual field to initiate a trial. Once initiated, seven sheep appeared clustered on the ground between both participants (cluster center x,y = 0,0; cluster radius = 0.36m). The native movement of the sheep was governed by Brownian motion. However, the sheep were repelled away from either participant when within 0.6 m of the participant (the dynamics of this repulsion was consistent with the method reported in Nalepka, Kallen et al., [2017], and in Experiment 1). Within the virtual environment, participants were represented as crash-test dummy avatars, calibrated to the participants’ eye-height (shoulder width = 0.42 m). The avatars were controlled by the participants’ head movement and an inverse kinematics controller was responsible for rotating the avatar’s torso

(FinalIK, Rootmotion, Estonia). The avatars’ bodies, as well as the shadows they cast were included to provide information to participants regarding their partner’s location in space. At the base of each avatar was a “cube” (edge = 0.15m). This cube was controlled by the x,y-position of the participants’ head position and the position of this cube defined the location of the participant regarding sheep repulsion. Note that participants could not contact the sheep (i.e., push the sheep).

Rather, when participants moved close enough to the sheep, the sheep moved directly away from participants’ cube’s location (i.e., like the repulsion of a magnet).

Participants were informed that the aim of the task was to contain all sheep close together.

Successful containment was defined as herding all the sheep together within a radial distance of

0.72 m, centered at the current sheeps’ center of mass at each time step. Participants received optical feedback when all the sheep were within this radial distance of each other by changing the color of the sheep from white to red. Each herding trial was a maximum of two-minutes long.

Participants were informed that trials would end prematurely if any sheep exited the playing field and that successful trials were deemed to have occurred if the sheep were contained within the 50

containment circle (i.e., were colored red) for at least 70% of the last 90 s of a trial (i.e., for 63 seconds). The overall goal for participants was to complete eight successful trials within the 45- minute experimental period.

Design and Measures. Dyads were randomly assigned to one of three difficulty conditions. Task difficulty was defined by the maximum speed the spherical sheep could move

푚 푚 along the game field: low difficulty = 0.12 ; medium difficulty = 0.20 , high difficulty = 0.28 푠 푠

푚 . Dependent measures to assess shepherding performance were the following: containment time 푠

(s) – the amount of time the sheep were within the required circular region (i.e., were red), sheep area/spread (m2) – the area of the convex hull formed by all sheep (the convex hull is defined by the smallest convex, n-sided polygon that encompasses all objects, e.g., like a rubber-band around a set of objects), herd travel (m) – the distance the herd’s center of mass (COM) moved during the

푚 trial, and sheep speed ( ) – the magnitude of the mean sheep’s velocity. 푠

Procedure. Following informed consent, participants were taken to the testing room and received task instructions. Dyads were told that they had 45 minutes to complete eight successful trials and that once eight successful trials had been completed the experiment would finish early in exchange for full participation credit. This “game” type methodology was used as an incentive for participants to work together. If participants did not complete eight successful trials the shepherding game would automatically end after 45 minutes. Participants were told the rules of the task. They were informed that aim of the game was to herd the sheep together and then keep them together, and that the sheep would turn red when they had been herded close enough together.

The experimenter reinforced to participants that to move the sheep, participants had to position their cube so that the sheep would be repelled away from them. The experimenter also reinforced

that there was no pre-defined goal location – only that the sheep needed to be herded together somewhere on the task field.

At the end of each two-minute trial, the trial “score” (i.e., percentage of time herded together) was displayed to participants. Participants were informed that a successful herding trial required a score of 70 or better. If a trial ended prematurely due to a sheep exiting the task field, a

“Try Again!” message would instead appear. At the end of each trial, participants initiated the next trial by moving to their respective start location. Finally, participants were informed that they could not communicate during the task or during any breaks and the experimenter in the room would enforced this no-talking policy.

Results

One dyad was removed from analysis due to premature program closure. Of the remaining

39 dyads (13 in each condition), 13 (100%), 10 (76.92%) and 3 (23.08%) in the low, medium and high-difficulty conditions, respectively, completed the experiment within the 45-minutes allotted to them. The distribution of success rates was not equally distributed across difficulty conditions,

X2 (2, N = 39) = 18.23, p < .001. Mean times to completion were 16.91 (SD = 1.23), 24.83 (SD =

4.73) and 32.02 (SD = 7.30) minutes for slow, medium and fast, respectively. A Kruskal-Wallis test was conducted and a significant difference between task-difficulty conditions were found,

H(2) = 18.94, p < .001. Mean ranks were 7.19, 18.75 and 23.33 for the low, medium and high task- difficulty conditions, respectively. Together, manipulating the maximum velocity of the sheep was effective in making the shepherding task more difficult.

Effects of Task-Difficulty on Successful Herding Performance. Next, between-subject

ANOVAs were conducted to test the effects of task-difficulty on successful herding performance.

Dyads who achieved at least one successful trial were included in analyses, resulting in 13, 13 and 52

8 dyads in the low, medium and high-difficulty conditions. Plots summarizing results can be found in Figure 18. Significant differences were found across conditions on containment time, F(2, 31)

2 2 = 9.63, p = .001, ηp = .38, herd travel, F(2, 31) = 9.68, p = .001, ηp = .38, and sheep speed, F(2,

2 31) = 7.72, p = .002, ηp = .33. Bonferroni-corrected pairwise comparisons indicated that for containment time, dyads in the low task-difficulty condition contained the sheep within the required containment area (circle radius = 0.72m) for a longer (88.52 s, SD = 1.63) time than those in the medium task-difficulty condition (M = 81.58, SD = 4.66) (p < .001). This time was also greater than for those dyads in the high task-difficulty condition (M = 83.94 s, SD = 5.66), although this difference was not significant (p = .054). For herd travel, dyads in the low task-difficulty condition kept the herd more stably located (M = 3.21 m, SD = 0.32) than those in the medium task-difficulty condition (M = 4.22 m, SD = 0.80) (p < .001). No differences for dyads in the high task-difficulty group compared to other groups (M = 3.69 m, SD = .53). Finally, dyads in both the

푚 low task-difficulty condition (M = 0.07 , SD = 0.007) (p = .003) and high task-difficulty condition 푠

푚 (M = 0.08 , SD = 0.02) (p = .02) contained the sheep to a lower speed than dyads in the medium 푠

푚 task-difficulty condition (M = 0.09 , SD = 0.02). No other differences were found. In summary, 푠 dyads in the low and high task-difficulty conditions had the best containment performance, while those in the medium task-difficulty condition had more difficulty maintaining control over the sheep.

Figure 18. Summary plots on effect of task-difficulty on successful shepherding performance. Error bars represent standard error. * = p < .05; ** = p < .01; *** = p < .001.

Behavioral Modes and Classification. Preliminary analysis revealed that participants adopted two modes of herding behavior. The first mode of behavior, collecting behavior, was very similar to the search-and-recover behavior observed by Nalepka, Kallen et al., (2017) and involved participants in a dyad acting relatively independently, moving from sheep to sheep to collect the sheep together as a group. The second behavioral mode was circling containment, where participants would maintain a circling direction with no, or very little, switching. In addition, participants would position themselves on opposite sides of the circle. Like behavior found in

Nalepka, Kallen et al. (2017), this behavioral mode was akin to stable oscillatory containment.

Figure 19 provides an illustration of each type of behavior.

Collecting and circling containment behavior have functional similarities to S&R and COC behavior from Experiment 1. The same classification criteria from Experiment 1 was used, where a k-means clustering algorithm was utilized to identify two clusters (collecting/S&R and circling/COC) on trials lasting at least 50% of the trial time (1 minute). Similarly to Experiment 1, the average dyad peak frequency of angular movement was plotted against the average number of directional changes for the trial (at a 1 second resolution). Unlike Experiment 1, trials were not detrended as perfect circling containment performance was expected to exhibit zero directional changes regardless of the dyad’s average peak frequency. A total of 433 trials met the 1-minute minimum criteria. Following clustering analysis, 247 trials (57.04%) were classified as collecting behavior and 186 trials (42.96%) were classified as circling containment. A Kruskal-Wallis test demonstrated that the proportion of successful trials that exhibited circling containment was not the same across all three difficulty conditions, H(2) = 6.29, p = .04. Dyads in the low task-difficulty condition exhibited circling containment on 44.23% (SD = 41.96, mean rank = 13.96) of trials, whereas it was 58.65% (SD = 34.67, mean rank = 19.19) for dyads in the medium task-difficulty condition and 92.50% (SD = 21.21, mean rank = 20.50) for dyads in the high task-difficulty condition.

Figure 19. Exemplar time-series displaying unwrapped angular position (left) and x,y position (right) for each behavioral mode during the last 90 seconds of a successful trial. The collecting behavior is from a dyad in the low task-difficulty condition. The dyad exhibiting circling containment behavior is from the high task-difficulty condition.

Behavioral Mode and Shepherding Performance. Within-subject ANOVAs were conducted to assess the effectiveness of each behavioral mode in containing the sheep in the shepherding task. Dyad averages computed from all the trials submitted to the clustering analysis were done, including both failure and successful trials. A total of 29 dyads were submitted to analysis who exhibited each of the behavioral modes (7, 12, 10 dyads in the low, medium and high task-difficulty conditions, respectively). Figure 20 presents a summary of the results. Significant differences were found on all dependent measures except herd travel: containment time, t(28) =

7.27, p < .001, herd area, t(28) = -6.29, p < .001, and sheep speed, t(28) = -7.28, p < .001.

Figure 20. Summary plots for the effect of behavioral mode on shepherding performance. Error bars represent standard error. *** = p < .001.

Trials in which dyads utilized the circling containment behavior mode was the best at containing the sheep in all cases. For the dependent measure containment time, dyads utilizing circling containment behavior contained the sheep for 75.86 s (SD = 19.98) on average compared to 45.38 s (SD = 27.08) when using collecting behavior. For the dependent measure, herd area, circling containment trials kept the spread of the sheep to a smaller area (M = 0.40 m2, SD = .29) than collecting trials (M = 0.89 m2, SD = 0.51). Finally, circling containment behavior excelled at

푚 minimizing mean sheep speed (M = 0.09 , SD = 0.03) compared to collecting behavior (M = 0.13 푠

푚 , SD = 0.03). In summary, the use of circling containment led to superior shepherding 푠 performance in the present experiment.

Behavioral Mode and Participant Behavior. Qualitatively, circling containment behavior consisted of coordinated movement between participants to maintain stable circling behavior. To quantify the stability of this coordination, an instantaneous relative phase analysis was conducted to assess the mean and variability of the phase angle between participants with respect to the sheeps’ center of mass using the Hilbert transform (see Pikovsky, Rosenblum, &

Kurths, [2001]). Here, a phase angle of 0° indicated both participants were in the same physical space, while a phase angle of 180° indicated that the participants were on opposite sides of the herd such that a line between them would bisect the sheeps’ center of mass. Paired-samples t-tests indicated that participants within a dyad kept the same average relative phase (collecting M =

152.51°, SD = 19.22; circling containment M = 156.86°, SD = 14.50), t(28) = 1.00, p = .33, but circling containment behavior led to a more stable relative phase compared to collecting behavior, t(28) = -7.03, p < .001 (see Figure 21). Dyads had lower relative phase variability when performing circling containment (M = 47.29°, SD = 3.63) than when performing collecting behavior (M = 55.69°, SD = 5.70). In summary, all participants maintained a phase relationship to their partner > 90° when completing the task. Further, the increased performance in containing the sheep utilizing circling containment behavior was accompanied with greater degree of coordination between participants, as measured by their reduction in relative phase variability.

Figure 21. Summary plots for the effect of behavioral mode on interpersonal coordination using relative phase analysis. * = p < .05; ** = p ≤ .01; *** = p < .001.

Discussion

Experiment 2 sought to test whether the same behavioral modes in the shepherding task found in Nalepka, Kallen et al., (2017), as well as in Experiment 1, would also define two-person shepherding behavior in a new environmental context. In Experiment 2, the same task was utilized, but participants had a different point-of-view and utilized different effectors (i.e., their whole body as opposed to their hand) to complete the task. As in Experiment 1, maximum sheep speed was also manipulated to further test the hypothesis that maximum sheep speed could act as a control parameter to promote the emergence of more stable COC-like behvaior. Results indicated that qualitatively similar, althought not identical, modes of behavior were observed. Further, the task difficulty manipulation had an effect on what behavioral modes were observed, with higher task difficulty conditions leading to more stable modes of coordinatied behavior. Next, a detailed discussion of the observed behaviors, as well as a possible mechanism for behavioral mode transitions will be discussed.

Behavioral Dynamics and Comparisons to Experiment 1. Two modes of behvaior were observed: collecting, which involved the collection of individual sheep, and circling containment behavior, which involved the coordinated circling around the entire herd. It is important to note that these modes of behaviors are qualitatively like the modes of behavior observed in sheepdog and wolf-pack hunting work, as well as the modes observed in Nalepka, Kallen et al., (2017). Like the collecting behavior found in Strömbom et al., (2014), collecting behavior in the present work is a task requirement for containment to begin (i.e., if one does not go for the furthest sheep from the herd, a sheep herd would never form). For both S&R behavior and collecting behavior, participants appeared to work individually to corral the sheep that is furthest from the center on their side of the field. Similarly to S&R behavior, collecting behavior was also found to occur predominately in low task-difficulty conditions. With regard to circling containment, the behavior shares common features with wolf-pack hunting. Like wolf-pack hunting, participants stayed equidistant from each other when containing the sheep. As task difficulty increased, so did the increased use of circling containment behavior, as it was more optimal in containing the sheep compared to collecting behavior. Similar to Experiment 1, both circling containment and COC behavior resulted in higher containment time, kept the sheep to a smaller area, reduced the amaount of travel by the sheep, and kept them to a lower mean speed. The formation of a contiual spatial- temporal “wall” for both COC and cicrcling containment behavior was effective in canceling the forces acting upon the herd by both participants, as well as minimizing the possibility of inter- agent collision.

The use of circling containment behavior in all three task-difficulty conditions may be due to dyads exploiting the most stable and energetically efficient solution to meet task demands. The defining characteristic of this behavior is the maintainence of the same rotation direction during 60

the entire trial. What are the benefits of maintaining a fixed direction when performing circling movements? First, collecting behavior resulted in lower average velocities, either due to the result of stopping and waiting for a particular sheep to be repelled back towards the center, or because of changes in velocity due to directional changes. By decreasing one’s velocity, the required dyad- herd relationship for successful containment may no longer be maintained, especially as average sheep speed increased at higher task difficulties. This may be why collecting behavior was prevalent in low and medium task-difficulty condition, but almost non-existent in the high task- difficulty condition. Further, there is an energetic cost associated with changing walking speed more generally (Seethapathi & Srinivasan, 2015). Second, as task difficulty increases, circling containment is a more optimal solution, as it relates to demands on sensorimotor processing by the participant (Koeppen et al., 2017). Consistent with observations from Experiment 1, maintaining a fixed direction of travel may reduce the necessity to constantly search for individual sheep to corral, simplifying control.

An explanation for the emergence of circling containment behavior in dyad performance as opposed to the rhtyhmic osccilatory behavior seen in Nalepka, Kallen et al., (2017) may be due to a difference in constraints between tasks. Utilizing Newell’s (1986) constraint trichotomoy where behavior results from the interaction between individual, environmental, and task constraints, the constraints acting upon the individual were different between task contexts (task constraints were held constant, and environmental constraints defined as influences of gravity, temperature, etc. were the same in both contexts). The individual constraint that changed was the constraint of arm collisions, which limited the coordinative modes possible in the tabletop shepherding task investigated in Nalepka, Kallen et al., (2017). That is, rythmic oscillatory behavior was observed because joint circling behavior was impossible when participants operated 61

handheld controllers as the participants’ forearms would collide with their partner. Circling behavior was observed in Nalepka, Kallen et al., (2017), but this was only observed when one participant performed circling motions around the herd, while their partner would sit idly to prevent collision (see Figure 22). The coupled oscillatory containment behavior, then, can be thought of as a special case of the more generally, less constrained circling containment behavior found in the current experiment.

Both behaviors achieved the task demand of maintaining a spatial-temporal circle around the herd, but differed in executation due to a difference in individual constraints. One can expect that if the present experiment incorporated the additional constraint found in the tabletop version of the task, the same rhtyhmic oscillatory behavior would be expected. This could be done by artificially adding a “forearm” to the participants’ avatar bodies. For example, a polygon can be added to the avatar’s body that extends outside the task space. By telling participants that they could not intersect this polygon, circling behavior would no longer be possible and rhtyhmic oscillatory behavior would be expected to emerge. Conversely, if participants were instructed to complete the tabletop version of the shepherding task with the use of joysticks, as opposed to hand- held sensors, a circling mode of behavior would likely occur (with the caveat that participants’ may not necessarily maintain seperation there would be no physical collision). Despite differences between COC and circling containment behavior, both represent limit-cycle behaviors and a dynamic similitude between environmental contexts on the level of dynamic motor primitives.

Figure 22. Circling behavior example from Nalepka, Kallen et al., (2017). One participant (orange) would make circling movements with their hand to contain the sheep, while their partner (blue) would wait idly to avoid collision (right). Radial and angular position for the blue participant (top left) and for the orange participant (bottom left).

Symmetry-Breaking as a Mechanism for Behavioral Change. Qualitatively, the transition from collecting to circling containment behavior is a transition from collecting individual sheep to containing the herd. Coordination is required between both members in the dyad in order for circling behavior to be successful (i.e., without one participant colliding with their partner). A symmetry-breaking event may be responsible for the transition from collecting to circling containment behavior (Richardson & Kallen, 2015). In work by Richardson et al., (2015), for example, dyads were instructed to move from one corner of a rectangle diagonal to another at the same time while avoiding collision with their partner when passing the center of the rectangle

(see Figure 23). Collisions with the partner led to task failure. To avoid failure, a symetry-breaking event occurred that broke the symetrical nature of the participants’ behavior. Instead of colliding with each other, one participant in the dyad would form a circular trajectory that would avoid the collision in the center of the rectangle (see Figure 23, bottom). Once this event occurred, participants settled on their new, assymetric spatial mode of behavior. Results indicated that the 63

assymetry in behavior was due to an asymetry in the degree of repulsive coupling between participants that the task demanded (Richardson et al., 2015). When the “avoid collision” task demand is minimized, behavior becomes more symetrical (Eiler et al., 2015).

Figure 23. Cooperative collision avoidance task used in Richardson et al., (2015). Participants were tasked to move between respective diagonal squares while avoiding collision (top). Example behavioral trajectories (bottom). Taken from Richardson et al., (2016).

The transition from collecting to circling containment behavior could be due to a similar symetry-breaking process. In the joint action literature, symmetry breaks are often associated with an increase in role differentiation where the actions of each is not symetrical to their partner such as the behavior in the collision avoidance task described above (Richardson & Kallen, 2015).

Although it may be counter-intuitive how a break in symmetry could produce a more symmetrical, less differentiated state (i.e., the creation of a circling pattern where the behavior of each participant is similar), the hope is that the following will convince the reader of its plausibility.

Collecting behavior may be regarded as a symetrical behavior in the sense that the decision to move clockwise or counterclockwise is equally distributed across time. As can be seen in Figure 64

19 for the collecting example, there is no bias in spin direction over the course of 90 seconds. With the discovery of circling containment, there is a clear preference to maintain a particular rotation.

In Figure 19, bottom, for example, the dyad maintained their preferred direction for at least 15 revolutions in the last 90 seconds of the trial. This differentiation to a specific spin direction is akin to what is seen in a Rayleigh-Bénard convection system. In this system, a liquid is heated from below. With increasing temperatures, the activity of specific molecules increases but is uncorrelated (i.e., there is equal probability as to the molecule’s direction of movement). However, once a criticial temperature is reached, a phase transition occurs and the molecules begin to coordinate and maintain a particular spin direction as a method to expel the energy gradient more efficiently.

The symmetry-breaking event in the shepherding task may arise in one of two ways. The first possibility is a symmetry-break in the locomotory mode participants utilized to collect and contain the sheep. Although not explicitely analyzed here, anecodately there are two movement solutions to contain a particular sheep. The first solution is to walk so as to maintain one’s body orientation towards the center of the herd. Here, to corral a sheep, one would perform a side or crossover shuffle so that the sheep was in-between the participant and the goal location. This movement is symmetrical with no preferrence towards movement to the left or right. The other behavioral solution is to walk so that one’s body is facing tangently to the containment circle, affording forward locomotion to the targetted sheep. This behavioral mode is not symmetrical in its ability to move forward or back, breaking the symmetry by biasing movement towards the forward direction. This is because it is energetically more expensive to walk backwards than forward (Reilly & Bowen, 1984). The utility of forward locomotion is also greater than side shuffling, as side shuffling is more energeticallty expensive and has a lower optimal speed 65

compared to forward locomotion (Handford & Srinivasan, 2014). Therefore, participants may select a forward locomotory strategy to contain the sheep as more energeticaly efficient, biasing the preferred direction of rotation around the herd. Similar to the transition to COC behavior from

S&R behavior in Experiment 1, the transition from a side shuffle mode of behavior to forward locomotion may be considered a Hopf bifurcation (i.e., a transition from fixed point control to limit-cyrcle behavior).

A second possible symmetry-breaking event, which is not exclusive from the first possibility, is a change in the repulsive term between participants in the dyad. An increased repulsive term by one participant in the dyad may result in one participant reacting to the movement direction of their partner. As one participant maintained their movement direction, their partner may be repelled and move to the other side of the circle (to avoid collision). If the participant maintains this behavior, their partner would be forced to continue walking in a circle in order to avoid collision with their partner, allowing the circling containment behavior to emerge.

This symetry-break could either be unintentional or intentional. Anecdotally, one participant did report following their partner closely to encourage them to maintain the circling pattern, although sometimes that may not be enough to induce a behavioral mode transition. One participant, out of frustration, would make circling motions with their arm to try to communicate with their partner

(although arm movements were not able to be seen while wearing the virtual reality headsets).

CHAPTER 4

General Discussion

Broadly stated, the aim of the current work was to investigate the processes that result in the self-organized emergence of coordinated, multi-agent behavior within complex, dynamically changing environments. More specifically, a multi-agent shepherding task paradigm was employed to investigate the conditions that promote the discovery of more optimal behavioral modes within a social action context and to determine what, if any, ocular biomarkers might predict this discovery. The shepherding task, first investigated in Nalepka, Kallen et al., (2017) involved pairs of participants working together to contain reactive, autonomous “sheep” to a small area while corralling any that tried to flee. For this task, dyads typically adopted a search-and-recover

(S&R) behavior to corral the sheep, which involved participants attempting to contain sheep on their side of the task space (herding field). However, some dyads discovered a more optimal mode of behavior termed coupled oscillatory containment (COC). This behavior involved both participants performing rhythmic oscillatory movements around the entire herd to keep them contained, treating them as a collective as opposed to individual sheep.

Summary of Findings in Experiment 1

In Experiment 1, participants, recruited as dyads, completed the shepherding task using handheld sensors on a tabletop display while wearing eye-tracking headsets. The maximum speed the sheep could move was manipulated as a “control parameter” to test whether it could facilitate the emergence of COC behavior. The same S&R and COC modes of behavior were present as described above. As task difficulty increased, there was a greater utilization of COC behavior to contain the herd. The use of COC behavior led to superior herding performance compared to S&R 67

behavior – consistent with findings from Nalepka, Kallen et al., (2017). The transition to COC behavior was predicted by oculomotor measures two trials prior to discovery. Specifically, significant changes in Entropy, measured by RQA, was detected prior to the discovery of COC behavior. This is consistent with findings from Stephen et al., (2009) which also found changes in

Entropy preceding the discovery of a more optimal strategy in a math-based problem-solving.

%Determinism also significantly changed prior to the first use of COC behavior. Taken together, changes in both Entropy and %Determinism suggested that constraints due to task-difficulty were relaxed leading up to the first use of COC behavior. This relaxation of ocular dynamics may have allowed for the detection of information that guided the discovery and use of COC behavior. Future works seeks to investigate the informational basis for the discovery and adoption of COC behavior.

Possibly, there is information available to the agent that guides them to more optimal solutions

(Jacobs, Ibáñez-Gijón, Díaz, & Travieso, 2011; Jacobs & Michaels, 2007; Michaels, Gomes, &

Benda, 2016).

Summary of Findings in Experiment 2

In Experiment 2, dyads were tasked to complete the same shepherding task in a new environmental context. Instead of performing the task on a tabletop display, participants were put in an immersive virtual reality environment that afforded full-body movement throughout the task- space including the ability to run. Experiment 2 tested whether the behavioral modes discovered in Nalepka, Kallen et al., (2017), and in Experiment 1, were specific to an experimental context, or if they represented general global “herding” solutions. Functionally similar modes were discovered when participants completed this version of the task. The first primary mode of behavior, collecting behavior, was akin to S&R behavior in that it involved both participants individually corralling sheep furthest from the herd and directing them towards the group. The 68

second primary mode, circling containment involved coupled and coordinated circling patterns around the entire sheep collective in a fixed direction. The stability of this coordination was greater than what was found in collecting behavior. The circling containment behavioral mode was also found to be best at completing the full-body shepherding task, as it allowed for the sheep to be contained for a longer period and to a smaller area. This circling behavior was not unlike what is seen in wolf-pack hunting where individual wolves will form a circle around their prey while also staying equidistant to neighboring wolves (Muro et al., 2011). Also, recent work on simulating shepherding performance by multiple sheepdogs using a decentralized control algorithm indicated that, given only local information about the nearest sheep and a desire to avoid collision with sheep or each other sheepdog, circling behaviors also emerged once the sheep were collected (Lee &

Kim, 2017).

Circling containment is not the same as the COC mode of behavior found in Experiment 1 and in Nalepka, Kallen et al., (2017). Nevertheless, circling containment may be an unconstrained version of COC, where circling was impossible due to the arm acting as a hard constraint on the task not found in Experiment 2. Also, circling containment and COC behavior both require the coordinated actions of both participants to maintain a circle around the entire herd. Consistent with findings from Experiment 1, the greater use of more coordinated modes of behaviors (circling containment here, COC in Experiment 1) were found as task-difficulty increased.

General Conclusions

Understanding what is controlled in a task or encounter (Warren, 2006; Warren & Shaw,

1985; Wilson & Golonka, 2013) is a useful unit of analysis for the understanding of the animal in ecological psychology theory (Baggs, 2014). Research in this area involves identifying the information that specifies action boundaries for encounters such as climbing a staircase, applying 69

brakes to an automobile to avoid collision, or throwing a ball to a target (Fajen, 2007; Warren et al., 1984; Wilson, Weightman, Bingham, & Zhu, 2016). To reduce the active degrees-of-freedom of the motor system to simplify control, animals are hypothesized to organize themselves to form

“special-purpose” devices that are constrained by information to solve a task (Bernstein, 1967;

Runeson, 1977; Turvey, 1990). The solution, more specifically, is for animals to form coordinative structures that couples the activity of the components responsible for the control of information for action (Bingham, 1988; Kugler, Kelso, & Turvey, 1980; Kugler & Turvey, 1987; Saltzman &

Kelso, 1987).

Tasks, like physical constraints on an animal’s morphology, play a constraining role in the organization and guidance of behavior (Turvey, Saltzman, & Schmidt, 1991). The interactions between these constraints are what allows for the emergence of complex behavior like that found in passive-dynamic walkers (Collins, Ruina, Tedrake, & Wisse, 2005), termite mound-building behavior (Kugler & Turvey, 1987), and soccer (Glazier, 2015). The task-constraints inherent in the shepherding paradigm pursued in the present set of experiments demonstrate that certain behaviors are afforded to participants when acting within a dyad. Although differences in physical constraints acting on participants resulted in different behaviors (e.g., COC behavior in Experiment

1, circling containment in Experiment 2), there was dynamic similitude to the kinds of behavioral solutions exhibited by dyads in either experiment. Specifically, a subset of dyads learned to transition from a fixed-point solution (e.g., S&R, collecting) to a rhythmic, limit-cycle solution

(e.g., COC, circling containment). Discrete (i.e., fixed-point) and rhythmic (i.e., limit-cycle) behaviors constitute two general classes of behavior that underlie the building blocks of many complex behaviors (Hogan & Sternad, 2007, 2012; Ijspeert et al., 2013; Saltzman & Kelso, 1987).

Although the behaviors were instantiated in different ways, the same underlying task-dynamic 70

model may span between both experimental contexts as a transition from fixed-point to limit-cycle behavior. Finally, given differences in task-constraints (e.g., task difficulty) participants learned to select the solutions that were best in meeting task-demands while potentially also simplifying control.

Future Research. There is future opportunity for the use of assistive systems in collaborating with humans and steering their behavior, as teachers, towards more optimal solutions. Researchers have developed artificial systems that can couple its movements in such a way as to steer participants to maintain a 90° phase relationship in rhythmic coordination tasks

(Dumas, de Guzman, Tognoli, & Kelso, 2014). Similarly, a task-dynamic model of the shepherding task has been created and tested with novices (Nalepka et al., 2016). However, not every participant discovered the preferred mode of coordination intended in the interaction. This opens an opportunity for the future development of adaptive systems that can tune their parameter- dynamics to structure the task environment for the detection of information specifying the preferred behavior. The use of artificial systems to steer complex systems may be an effective approach for behavior learning and modification, as learning skills are social in many instances

(i.e., mother-child, teacher-student interactions). In this way, artificial agents could serve a role to manipulate the constraints of a task online (e.g., by adjusting the difficulty or degree of autonomy) to keep the learner in the “sweet-spot” for learning (e.g., their proximal zone of development

[Vygotsky, 1978]), akin to what is done in the constraints-led approach to skill development

(Davids, Button, & Bennett, 2008). There may be additional benefits to scaffolding the novice’s learning as opposed to simply presenting them the solution. Self-discovery, associated with an

“aha!” movement has been suggested to lead to better memory recall of the solution in the future

(Danek, Fraps, von Müller, Grothe, & Öllinger, 2013). Further, the use of human-inspired, low- 71

dimensional coordinative models may allow for more fluid human-robot interaction (Lorenz,

Mörtl, & Hirche, 2013; Richardson et al., 2016). Because these models are low-dimensional in nature, machine learning techniques to select parameters will have their computational cost decreased for use of other, more computationally expensive tasks such as computer vision.

The shepherding task investigated thus far focused on symmetrical capabilities of the actors. Some recent work has begun to understand coordination given asymmetries in the agents

(Davis et al., 2017; Richardson et al., 2015; Wallot, Mitkidis, McGraw, & Roepstorff, 2016). In work by Davis et al., (2017), for example, when partners were asymmetric in their abilities to act in a supra-postural task, the dynamics of the over-constrained individual became more deterministic and led the interaction while the unconstrained and more flexible partner adapted to ensure success of the task. In a two-person “pong” task, the area of responsibility may be defined by the skills of the players, with the more able partner being responsible for a larger portion of the task (Benerink, Zaal, Casanova, Bonnardel, & Bootsma, 2016). For the shepherding task, future work could investigate how roles form and evolve in face of perturbations. For example, if a new sheep appears away from the herd, who in the dyad becomes responsible for maintaining the current herd, and who becomes responsible for retrieving the new sheep? Understanding how individual responsibilities are decided and changed in face of new constraints or perturbations, with both the aid of changing constraints and the use of adaptive assistive agents in experimental contexts, will give insight to the robustness and fluidity of multi-agent activity seen in everyday activities.

References

Baggs, E. (2014). The task-oriented approach in psychology : A solution to Fodor’s problem.

Proceedings of the Annual Meeting of the Cognitive Science Society, 36, 128–133.

Balasubramaniam, R., Riley, M. A., & Turvey, M. T. (2000). Specificity of postural sway to the

demands of a precision task. Gait and Posture, 12–24.

Benerink, N. H., Zaal, F. T. J. M., Casanova, R., Bonnardel, N., & Bootsma, R. J. (2016). Playing

“pong” together: Emergent coordination in a doubles interception task. Frontiers in

Psychology, 7(DEC). https://doi.org/10.3389/fpsyg.2016.01910

Bernstein, N. A. (1967). The co-ordination and regulation of movements: Conclusions towards the

Study of Motor Co-ordination. Biodynamics of Locomotion.

https://doi.org/10.1097/00005072-196804000-00011

Bingham, G. P. (1988). Task-specific devices and the perceptual bottleneck. Human Movement

Science, 7(2–4), 225–264. https://doi.org/10.1016/0167-9457(88)90013-9

Bingham, G. P. (2004). A Perceptually Driven Dynamical Model of Bimanual Rhythmic

Movement (and Phase Perception). Ecological Psychology, 16(1), 45–53.

https://doi.org/10.1207/s15326969eco1601_6

Collins, J. J., & Stewart, I. N. (1993). Coupled nonlinear oscillators and the symmetries of animal

gaits. Journal of Nonlinear Science, 3(1), 349–392. https://doi.org/10.1007/BF02429870

Collins, S., Ruina, a, Tedrake, R., & Wisse, M. (2005). Efficient Bipedal Robots Based on Passive-

Dynamics Walkers. Science, 307(5712), 1082–1085.

https://doi.org/10.1126/science.1107799

Danek, A. H., Fraps, T., von Müller, A., Grothe, B., & Öllinger, M. (2013). Aha! experiences leave

a mark: Facilitated recall of insight solutions. Psychological Research, 77(5), 659–669. 73

https://doi.org/10.1007/s00426-012-0454-8

Davids, K. W., Button, C., & Bennett, S. J. (2008). Dynamics of skill acquisition : a constraints-

led approach. Faculty of Health; Institute of Health and Biomedical Innovation; School of

Exercise & Nutrition Sciences. https://doi.org/10.1260/174795408784089333

Davis, T. J., Pinto, G. B., & Kiefer, A. W. (2017). The Stance Leads the Dance: The Emergence

of Role in a Joint Supra-Postural Task. Frontiers in Psychology.

https://doi.org/10.3389/fpsyg.2017.00718

Dumas, G., de Guzman, G. C., Tognoli, E., & Kelso, J. A. S. (2014). The human dynamic clamp

as a paradigm for social interaction. Pnas, 1407486111-.

https://doi.org/10.1073/pnas.1407486111

Eiler, B., Kallen, R. W., Harrison, S. J., Saltzman, E. L., Schmidt, R. C., & Richardson, M. J.

(2015). Behavioral dynamics of a collision avoidance task; How asymmetry stabilizes

performance. In D. C. Noelle, A. S. Warlaumont, J. Yoshimi, T. Matlock, C. D. Jennings, &

P. P. Maglio (Eds.), Proceedings of the 37th Annual Meeting of the Cognitive Science Society.

Austin, TX: Cognitive Science Society.

Fajen, B. R. (2007). Affordance-Based Control of Visually Guided Action. Ecological Psychology,

19(4), 383–410. https://doi.org/10.1080/10407410701557877

Fajen, B. R., & Warren, W. H. (2003). Behavioral Dynamics of Steering, Obstacle Avoidance, and

Route Selection. Journal of Experimental Psychology: Human Perception and Performance,

29(2), 343–362. https://doi.org/10.1037/0096-1523.29.2.343

Glazier, P. S. (2015). Towards a Grand Unified Theory of sports performance. Human Movement

Science. https://doi.org/10.1016/j.humov.2015.08.001

Haken, H., Kelso, J. A. S., & Bunz, H. (1985). A theoretical model of phase transitions in human 74

hand movements. Biological Cybernetics, 51(5), 347–356.

https://doi.org/10.1007/BF00336922

Handford, M. L., & Srinivasan, M. (2014). Sideways walking: preferred is slow, slow is optimal,

and optimal is expensive. Biology Letters, 10(1), 20131006–20131006.

https://doi.org/10.1098/rsbl.2013.1006

Hogan, N., & Sternad, D. (2007). On rhythmic and discrete movements: Reflections, definitions

and implications for motor control. Experimental Brain Research, 181(1), 13–30.

https://doi.org/10.1007/s00221-007-0899-y

Hogan, N., & Sternad, D. (2012). Dynamic primitives of motor behavior. Biological Cybernetics.

https://doi.org/10.1007/s00422-012-0527-1

Huang, H. J., Kram, R., & Ahmed, A. A. (2012). Reduction of Metabolic Cost during Motor

Learning of Arm Reaching Dynamics. Journal of Neuroscience, 32(6), 2182–2190.

https://doi.org/10.1523/JNEUROSCI.4003-11.2012

Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P., & Schaal, S. (2013). Dynamical Movement

Primitives: Learning Attractor Models for Motor Behaviors. Neural Computation, 25(2),

328–373. https://doi.org/10.1162/NECO_a_00393

Jacobs, D. M., Ibáñez-Gijón, J., Díaz, A., & Travieso, D. (2011). On Potential-Based and Direct

Movements in Information Spaces. Ecological Psychology, 23(September 2013), 123–145.

https://doi.org/10.1080/10407413.2011.566046

Jacobs, D. M., & Michaels, C. F. (2007). Direct Learning. Ecological Psychology, 19(4), 321–349.

https://doi.org/10.1080/10407410701432337

Jacobs, D. M., Runeson, S., & Michaels, C. F. (2001). Learning to visually perceive the relative

mass of colliding balls in globally and locally constrained task ecologies. Journal of 75

Experimental Psychology: Human Perception and Performance, 27(5), 1019–1038.

https://doi.org/10.1037/0096-1523.27.5.1019

Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior.

Kelso, J. A. S. (2009). Synergies: Atoms of brain and behavior. Advances in Experimental

Medicine and Biology, 629, 83–91. https://doi.org/10.1007/978-0-387-77064-2_5

Kennel, M. B., Brown, R., & Abarbanel, H. D. I. (1992). Determining embedding dimension for

phase-space reconstruction using a geometrical construction. Physical Review A, 45(6),

3403–3411. https://doi.org/10.1103/PhysRevA.45.3403

Kistemaker, D. A., Wong, J. D., & Gribble, P. L. (2010). The Central Nervous System Does Not

Minimize Energy Cost in Arm Movements. Journal of Neurophysiology, 104(6), 2985–2994.

https://doi.org/10.1152/jn.00483.2010

Knoblich, G., Butterfill, S., & Sebanz, N. (2011). Psychological Research on Joint Action: Theory

and Data. In The Psychology of Learning and Motivation (Vol. 54, pp. 59–101). Burlington:

Academic Press.

Koeppen, R., Huber, M. E., Sternad, D., & Hogan, N. (2017). Controlling physical interactions:

Humans do not minimize muscle effort. In Proceedings of the ASME 2017 Dynamic Systems

and Control Conference (pp. 1–8). Tysons, Virginia. https://doi.org/10.1115/DSCC2017-

5202

Kovacs, A. J., Buchanan, J. J., & Shea, C. H. (2010). Impossible is nothing: 5:3 and 4:3 multi-

frequency bimanual coordination. Experimental Brain Research, 201(2), 249–259.

https://doi.org/10.1007/s00221-009-2031-y

Kugler, P. N., Kelso, J. A. S., & Turvey, M. T. (1980). 1 On the Concept of Coordinative Structures

as Dissipative Structures: I. Theoretical Lines of Convergence. Advances in Psychology, 1(C), 76

3–47. https://doi.org/10.1016/S0166-4115(08)61936-6

Kugler, P. N., & Turvey, M. T. (1987). Information, natural law, and the self-assembly of rhythmic

movement. Resources for Ecological Psychology, 481.

Lamb, M., Kallen, R. W., Harrison, S. J., Di Bernardo, M., Minai, A., & Richardson, M. J. (2017).

To Pass or Not to Pass: Modeling the Movement and Affordance Dynamics of a Pick and

Place Task. Frontiers in Psychology, 8(1061), 1–23.

https://doi.org/10.3389/fpsyg.2017.01061

Latash, M. L. (2008). Synergy. New York: Oxford University Press.

Lee, W., & Kim, D. (2017). Autonomous shepherding behaviors of multiple target steering robots.

Sensors (Switzerland), 17(12). https://doi.org/10.3390/s17122729

Lorenz, T., Mörtl, A., & Hirche, S. (2013). Movement synchronization fails during non-adaptive

human-robot interaction, 189–190. Retrieved from

http://dl.acm.org/citation.cfm?id=2447556.2447631

Mazaheri, M., Salavati, M., Negahban, H., Sanjari, M. A., & Parnianpour, M. (2010). Postural

sway in low back pain: Effects of dual tasks. Gait and Posture, 31(1), 116–121.

https://doi.org/10.1016/j.gaitpost.2009.09.009

Mechsner, F., Kerzel, D., Knoblich, G., & Prinz, W. (2001). Perceptual basis of bimanual

coordination. Nature, 414(6859), 69–73. https://doi.org/10.1038/35102060

Michaels, C. F., Gomes, T. V. B., & Benda, R. N. (2016). A Direct-Learning Approach to

Acquiring a Bimanual Tapping Skill. Journal of Motor Behavior, 2895(January), 1–18.

https://doi.org/10.1080/00222895.2016.1247031

Muro, C., Escobedo, R., Spector, L., & Coppinger, R. P. (2011). Wolf-pack (Canis lupus) hunting

strategies emerge from simple rules in computational simulations. Behavioural Processes, 77

88(3), 192–197. https://doi.org/10.1016/j.beproc.2011.09.006

Nalepka, P., Kallen, R. W., Chemero, A., Saltzman, E., & Richardson, M. J. (2017). Herd Those

Sheep: Emergent Multiagent Coordination and Behavioral-Mode Switching. Psychological

Science, 28(5), 630–650. https://doi.org/10.1177/0956797617692107

Nalepka, P., Lamb, M., Kallen, R. W., Saltzman, E., Chemero, A., & Richardson, M. J. (2017).

First step is to group them: Task-dynamic model validation for human multiagent herding in

a less constrained task. In G. Gunzelmann, A. Howes, T. Tenbrink, & E. Davelaar (Eds.),

Proceedings of the 39th Annual Meeting of the Cognitive Science Society. London, UK.

Nalepka, P., Lamb, M., Kallen, R. W., Shockley, K., Chemero, A., & Richardson, M. J. (2016). A

bio-inspired artificial agent to complete a herding task with novices. In C. Gershenson, T.

Froese, J. M. Sigueiros, W. Aguilar, E. J. Izquierdo, & H. Sayama (Eds.), Proceedings of the

Artificial Life Conference 2016 (pp. 656–663). Cancun, Mexico: MIT Press.

Nasseroleslami, B., Hasson, C. J., & Sternad, D. (2014). Rhythmic Manipulation of Objects with

Complex Dynamics: Predictability over Chaos. PLoS Computational Biology, 10(10).

https://doi.org/10.1371/journal.pcbi.1003900

Newell, K. (1986). Constraints on the development of coordination. Motor Development in

Children: Aspects of Coordination and Control. https://doi.org/10.1016/S0166-

4115(08)62541-8

Pellecchia, G., & Shockley, K. (2005). Application of recurrence quantification analysis: Influence

of cognitive activity on postural fluctuations. Tutorials in Contemporary Nonlinear Methods

for the Behavioral Sciences, 95–141. https://doi.org/10.1.1.138.5373

Pikovsky, A., Rosenblum, M., & Kurths, J. (2001). Synchronization: A Universal Concept in

Nonlinear Sciences. Cambridge: Cambridge University Press. 78

https://doi.org/10.1063/1.1554136

Reilly, T., & Bowen, T. (1984). Exertional costs of changes in directional modes of running.

Perceptual and Motor Skills, 58, 149–150.

Richardson, M. J., Harrison, S. J., Kallen, R. W., Walton, A., Eiler, B. A., Saltzman, E., & Schmidt,

R. C. (2015). Self-organized complementary joint action: Behavioral dynamics of an

interpersonal collision-avoidance task. Journal of Experimental Psychology: Human

Perception and Performance, 41(3), 665–79. https://doi.org/10.1037/xhp0000041

Richardson, M. J., & Kallen, R. W. (2015). Symmetry-breaking and the contextual emergence of

human multiagent coordination and social activity. In E. Dzhafarov, S. Jordan, R. Zhang, &

V. Cervantes (Eds.), Contextuality from Quantum Physics to Psychology (pp. 229–286).

World Scientific Publishing Co.

Richardson, M. J., Kallen, R. W., Nalepka, P., Harrison, S. J., Lamb, M., Chemero, A., … Schmidt,

R. C. (2016). Modeling embedded Interpersonal and multiagent coordination. In Proceedings

of the 1st International Conference on Complex Information Systems (pp. 155–164). Rome,

Italy. https://doi.org/10.5220/0005878101550164

Richardson, M. J., Marsh, K. L., Isenhower, R. W., Goodman, J. R. L., & Schmidt, R. C. (2007).

Rocking together: Dynamics of intentional and unintentional interpersonal coordination.

Human Movement Science, 26(6), 867–891. https://doi.org/10.1016/j.humov.2007.07.002

Richardson, M. J., Schmidt, R. C., & Kay, B. A. (2007). Distinguishing the noise and attractor

strength of coordinated limb movements using recurrence analysis. Biological Cybernetics,

96(1), 59–78. https://doi.org/10.1007/s00422-006-0104-6

Riley, M. A., & Turvey, M. T. (2002). Variability and Determinism in Motor Behavior. Journal

of Motor Behavior, 34(2), 99–125. https://doi.org/10.1080/00222890209601934 79

Riley, M. A., Wagman, J. B., Santana, M. V., Carello, C., & Turvey, M. T. (2002). Perceptual

behavior: Recurrence analysis of a haptic exploratory procedure. Perception, 31(4), 481–510.

https://doi.org/10.1068/p3176

Runeson, S. (1977). On the possibility of “smart” perceptual mechanisms. Scandinavian Journal

of Psychology, 18(1), 172–179. https://doi.org/10.1111/j.1467-9450.1977.tb00274.x

Saltzman, E., & Caplan, D. (2015). A graph-dynamic perspective on coordinative structures, the

role of affordance-effectivity relations in action selection, and the self-organization of

complex activities. Ecological Psychology, 27(4), 300–309.

https://doi.org/10.1080/10407413.2015.1086228

Saltzman, E., & Kelso, J. A. S. (1987). Skilled Actions: A Task-Dynamic Approach. Psychol Rev,

94(1), 84–106. https://doi.org/10.1037/0033-295X.94.1.84

Schaal, S., Sternad, D., Osu, R., & Kawato, M. (2004). Rhythmic arm movement is not discrete.

Nature Neuroscience, 7(10), 1136–1143. https://doi.org/10.1038/nn1322

Schmidt, R. C., Carello, C., & Turvey, M. T. (1990). Phase transitions and critical fluctuations in

the visual coordination of rhythmic movements between people. Journal of Experimental

Psychology. Human Perception and Performance, 16(2), 227–247.

https://doi.org/10.1037/0096-1523.16.2.227

Schmidt, R. C., & Richardson, M. J. (2008). Dynamics of interpersonal coordination.

Understanding Complex Systems, 2008, 281–308. https://doi.org/10.1007/978-3-540-74479-

5_14

Schmit, J. M., Riley, M. A., Dalvi, A., Sahay, A., Shear, P. K., Shockley, K. D., & Pun, R. Y. K.

(2006). Deterministic center of pressure patterns characterize postural instability in

Parkinson’s disease. Experimental Brain Research, 168(3), 357–367. 80

https://doi.org/10.1007/s00221-005-0094-y

Scholz, J. P., Kelso, J. A. S., & Schöner, G. (1987). Nonequilibrium phase transitions in

coordinated biological motion: Critical slowing down and switching time. Physics Letters A,

123(8), 390–394. https://doi.org/10.1016/0375-9601(87)90038-7

Schöner, G., Haken, H., & Kelso, J. A. S. (1986). A stochastic theory of phase transitions in human

hand movement. Biological Cybernetics, 53(4), 247–257.

https://doi.org/10.1007/BF00336995

Schöner, G., Zanone, P. G., & Kelso, J. A. S. (1992). Learning as change of coordination dynamics:

theory and experiment. Journal of Motor Behavior, 24(1), 29–48.

https://doi.org/10.1080/00222895.1992.9941599

Seethapathi, N., & Srinivasan, M. (2015). The metabolic cost of changing walking speeds is

significant, implies lower optimal speeds for shorter distances, and increases daily energy

estimates. Biology Letters, 11(9). https://doi.org/10.1098/rsbl.2015.0486

Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical

Journal, 27(July 1928), 379–423. https://doi.org/10.1145/584091.584093

Shockley, K. (2005). Cross recurrence quantification of interpersonal postural activity. In Tutorials

in Contemporary Nonlinear Methods for the Behavioral Sciences (pp. 142–177). Retrieved

from http://www.nsf.gov/sbe/bcs/pac/nmbs/nmbs.jsp

Shockley, K., Butwill, M., Zbilut, J. P., & Webber, C. L. (2002). Cross recurrence quantification

of coupled oscillators. Physics Letters, Section A: General, Atomic and Solid State Physics,

305(1–2), 59–69. https://doi.org/10.1016/S0375-9601(02)01411-1

Shockley, K., Richardson, D. C., & Dale, R. (2009). Conversation and Coordinative Structures.

Topics in Cognitive Science, 1(2), 305–319. https://doi.org/10.1111/j.1756- 81

8765.2009.01021.x

Stephen, D. G., Boncoddo, R. A., Magnuson, J. S., & Dixon, J. A. (2009). The dynamics of insight:

Mathematical discovery as a phase transition. Memory & Cognition, 37(8), 1132–1149.

https://doi.org/10.3758/MC.37.8.1132

Stephen, D. G., Dixon, J. A., & Isenhower, R. W. (2009). Dynamics of representational change:

Entropy, action, and cognition. Journal of Experimental Psychology: Human Perception and

Performance, 35(6), 1811–1832. https://doi.org/10.1037/a0014510

Sternad, D., Marino, H., Charles, S. K., Duarte, M., Dipietro, L., & Hogan, N. (2013). Transitions

between discrete and rhythmic primitives in a unimanual task. Frontiers in Computational

Neuroscience, 7(July), 1–13. https://doi.org/10.3389/fncom.2013.00090

Strömbom, D., Mann, R. P., Wilson, A. M., Hailes, S., Morton, A. J., Sumpter, D. J. T., & King,

A. J. (2014). Solving the shepherding problem: Heuristics for herding autonomous,

interacting agents. Journal of The Royal Society Interface, 11(100).

https://doi.org/10.1098/rsif.2014.0719

Takens, F. (1981). Detecting strange attractors in turbulence. In Lecture Notes in Mathematics,

Vol. 898, Dynamical Systems and Turbulence, Warwick 1980 (pp. 366–381). Berlin:

Springer-Verlag.

Thorp, E. B., Kording, K. P., & Mussa-Ivaldi, F. A. (2017). Using noise to shape motor learning.

Journal of Neurophysiology, 117(2), 728–737. https://doi.org/10.1152/jn.00493.2016

Turvey, M. T., Carello, C., & Kim, N. G. (1990). Links between active perception and the control

of action. In Synergetics of cognition (pp. 269–295). Berlin Heidelberg: Springer.

Turvey, M. T. (1990). Coordination. American Psychologist, 45(8), 938.

https://doi.org/10.1037//0003-066X.45.8.938 82

Turvey, M. T. (2007). Action and perception at the level of synergies. Human Movement Science,

26(4), 657–697. https://doi.org/10.1016/j.humov.2007.04.002

Turvey, M. T., Saltzman, E., & Schmidt, R. C. (1991). Dynamics and task-specific coordinations.

In N. I. Badler, B. A. Barsky, & D. Zeltzer (Eds.), Making them move: Mechanics, control,

and animation of articulated figures (pp. 157–170). San Mateo, CA: Morgan Kaufmann.

Vaidyanathan, P., Pelz, J., Alm, C., Shi, P., & Haake, A. (2014). Recurrence Quantification

Analysis Reveals Eye-Movement Behavior Differences between Experts and Novices. ETRA

’14 Proceedings of the Symposium on Eye Tracking Research and Applications, (MARCH),

303–306. https://doi.org/10.1145/2578153.2578207

Vesper, C., Schmitz, L., Safra, L., Sebanz, N., & Knoblich, G. (2016). The role of shared visual

information for joint action coordination. Cognition, 153, 118–123.

https://doi.org/10.1016/j.cognition.2016.05.002

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes.

Mind in Society The Development of Higher Psychological Processes, Mind in So, 159.

https://doi.org/10.1007/978-3-540-92784-6

Wagman, J. B., Caputo, S. E., & Stoffregen, T. A. (2016). Hierarchical nesting of affordances in

a tool use task. Journal of Experimental Psychology: Human Perception and Performance,

42(10), 1627–1642. https://doi.org/10.1037/xhp0000251

Wallot, S., Mitkidis, P., McGraw, J. J., & Roepstorff, A. (2016). Beyond synchrony: Joint action

in a complex production task reveals beneficial effects of decreased interpersonal synchrony.

PLoS ONE, 11(12). https://doi.org/10.1371/journal.pone.0168306

Warren, W. H. (2006). The dynamics of perception and action. Psychological Review, 113(2),

358–389. https://doi.org/10.1037/0033-295X.113.2.358 83

Warren, W. H., & Shaw, R. E. (1985). Events and Encounters as Units of Analysis for Ecological

Psychology. Proceedings of the First International Conference on Event Perception, 1–27.

Warren, W. H., Shaw, R., Turvey, M., Todd, J., Kugler, P., Lee, D., & Kelso, S. (1984). Perceiving

Affordances : Visual Guidance of Stair Climbing. Journal of Experimental Psychology:

Human Perception and Performance, 10(5), 683–703.

Webber, C. L. J., & Zbilut, J. P. (2005). Recurrence quantification analysis of nonlinear dynamical

systems. In Tutorials in Contemporary Nonlinear Methods for the Behavioral Sciences (pp.

26–94). Retrieved from htt://www.nsf.gov/sbe/bcs/pac/nmbs/nmbs.jsp

Wilson, A. D., & Golonka, S. (2013). Embodied Cognition is Not What you Think it is. Frontiers

in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00058

Wilson, A. D., Snapp-Childs, W., & Bingham, G. P. (2010). Perceptual learning immediately

yields new stable motor coordination. Journal of Experimental Psychology. Human

Perception and Performance, 36(6), 1508–1514. https://doi.org/10.1037/a0020412

Wilson, A. D., Snapp-Childs, W., Coats, R., & Bingham, G. P. (2010). Learning a coordinated

rhythmic movement with task-appropriate coordination feedback. Experimental Brain

Research, 205(4), 513–520. https://doi.org/10.1007/s00221-010-2388-y

Wilson, A. D., Weightman, A., Bingham, G. P., & Zhu, Q. (2016). Using task dynamics to quantify

the affordances of throwing for long distance and accuracy. Journal of Experimental

Psychology: Human Perception and Performance, 42(7), 965–981.

https://doi.org/10.1037/xhp0000199

Zaal, F. T. J. M., Bootsma, R. J., & van Wieringen, P. C. W. (1999). Dynamics of reaching for

stationary and moving objects: Data and model. Journal of Experimental Psychology: Human

Perception and Performance, 25(1), 149–161. https://doi.org/10.1037/0096-1523.25.1.149 84