Using Closed Feedback Loops to Evaluate Autonomous Juggling Performance Tomas´ Collado Hamzah Farooqi [email protected] [email protected] Tashu Gupta Ashwindev Pazhetam Robert Taylor [email protected] [email protected] [email protected] Kevin Ge* [email protected] New Jersey’s Governor’s School of Engineering and Technology July 18, 2020 *Corresponding Author Abstract—Juggling requires humans to intake a vast amount of This project aspires to study the complex movements of visual information, analyze it, and act accordingly, all at a speed humans, particularly in juggling, and recreate them through a that seems to surpass normal human reaction time. To study simulated robot. Once a juggling robot is completed, the goal these dexterous human movements, this project aims to recreate juggling through a simulated robot that inputs feedback similar of the project is to determine the minimum amount of feedback to how a human would. In this project, a robot was developed necessary to complete a successful juggle. By incrementally using a game development engine called Unity that performed a limiting the amount of information the robot receives, the three ball cascade based on knowledge of the positions of the balls necessity of certain data will be determined which will allow at all times, and, when this knowledge was restricted, was able to for the creation of more dexterous and efficient feedback- juggle using the initial velocities of the balls and simulated visual feedback. Comparing the two feedback systems yielded a better dependent robots. understanding as to how humans juggle, as well as how robots can be improved to perform advanced, dexterous movements. II. BACKGROUND A. Juggling Logistics I. INTRODUCTION The complex act of juggling can be summarized through a In the age of automation, advancements in robotics are es- series of mathematical algorithms and physics principles. The sential for higher production rates and increased productivity, pattern in which balls are thrown and caught is repetitive and along with countless other benefits. [1]. However, the tasks predictable, allowing it to be represented simply. However, this needed in a wide range of industries are relatively complex project aims to mimic the practical human ability to juggle, for modern technology, and thus trained human beings are and while the mathematical algorithms that comprise juggling required to take up those positions. For instance, biomedical theory describe juggling incredibly well, they also assume products are often produced with high levels of complexity an ideal environment. Therefore, while the algorithms would to mimic complicated human actions. Dexterous robots are produce perfect juggling patterns, the simulation would be needed to fulfill roles that would otherwise be occupied by highly unrealistic. Juggling theory refuses to take into account trained human beings. Juggling, which requires the training of external environmental factors such as dissimilar throws and one’s motor skills and hand-eye coordination, is an incredibly poor catches. Ideal juggling theory also assumes that there are complex act for a dexterous machine to perform. This project only “throws” and “catches” within a juggle and neglects the seeks to program a robot to juggle on Unity by modeling “dwell time” or holding period, in which the ball rests in a complex juggling movements and providing information that hand between throws and catches [2]. Realistically, humans are humans would normally input. By doing so, the project not capable of instantaneously throwing a ball after catching it, opens doors to robotic automation in numerous industries, and compensate by adjusting the initial velocity of the throw from performing medical procedures to packaging commercial to preserve the pattern [3]. Humans also attempt to juggle in products. just two dimensions, keeping the horizontal distance between 1 their body and the balls constant, to reduce the complexity of the act. The juggling robot will face a similar issue to humans as it’s impossible to completely forego carrying time and other factors that may compromise an ideal environment. Therefore, while juggling theory is consistent enough for a robot to be constructed based off of it, this implementation does not provide insight on how humans juggle. Thus, this robot will use juggling theory to define the structure of the throws, but Fig. 1. Three ball cascade ladder diagram will not rely on theory to reproduce realistic behaviors. B. Complexity of Juggling Another way to represent juggles is the vanilla siteswap notation, designed by Paul Klimek from the University of Part of the reason why juggling is so difficult for robots California, Bruce Tiemann from the California Institute of is its implementation of multiple complex human behaviors. Technology, and Michael Day from the University of Cam- Humans possess a unique spatial awareness that allows them bridge in 1985 [5]. This notation represents a sequence of to track the position and predict the path of the balls without throws with each number describing the ticks between the dedicating all their focus on any individual ball. One theory throwing and catching of the same ball. For further explanation as to why humans can juggle so accurately is dynamical on siteswaps, a 441 juggle explains that one ball is in the air prediction. The theory suggests that humans compute parabolic for four ticks after being thrown from one hand, the second trajectories through an innate form of kinematic equations. ball in the other hand is thrown immediately at the next tick Another theory is that the motor cortex stimulates multiple for four ticks, and then a third ball is passed from one hand to muscles to perform complex movements [4]. The simulated another for a single tick. The three ball cascade is represented robot within this project seeks to recreate these complex by the siteswap notation 333, meaning that each ball is thrown human behaviors successfully, tackling the first theory with for three ticks. It may be useful to note that the amount of balls the velocity-based juggle and the latter with the position-based used in a pattern can be found by taking the average of the juggle. numbers in the notation. For example, the 333 juggle would require three balls since those numbers average to three, while C. Siteswaps and Ladders a juggle like the 71 averages to four, meaning it requires four A simple way to visually depict the throws and catches of balls. the vast varieties of juggles is through a ladder diagram, which D. Feedback Loops uses different colored lines to represent the paths of different balls throughout the juggle. Every time a colored line is in Humans are able to juggle using sensory feedback to predict contact with one of the black lines, the ball is “caught” in the landing position of all three balls. By keeping their eyes at either the left hand (the top black line) or the right hand (the a fixed position above their hands, humans take in information lower black line). When the colored line is between the two about the balls’ locations to correctly position their hands. For black lines, the ball is in the air. It is possible to determine a robot to simulate this, it needs to implement a looping control how long the ball is in the air by looking at the vertical dashed system of which there are two types: open-loop systems lines, each representing one tick, which is an arbitrary unit and closed-loop systems. An open loop system is a system of time used by jugglers to represent the time it takes to independent of feedback, meaning the output of the system has pass a ball from one hand to another. Each number on the no effect on the input, causing every loop to be identical. In diagram represents the numbered tick, starting at 1 and ending contrast, a closed loop system implements a feedback signal, at 16. Figure 1 displays the ladder diagram for a 333 juggle, as the output of the system depends on the input, and the or a three ball cascade, which is the primary focus of this input changes based on surrounding conditions. This allows project. This diagram shows that the first ball (represented the robot to react to sudden changes. [6]. by the red line) is thrown immediately from the right hand. Based on theory alone, juggling can be performed with One tick after this, the second ball (blue line) is thrown from an open-loop system. Due to the theoretical predictability of the left, and, a tick after that, the third ball (yellow line) is the juggling motions, every throw and catch should be able thrown from the right hand. One tick after all three balls are to be timed perfectly by separating the throws and catches thrown, the first ball is caught in the left hand, and then thrown with the appropriate number of ticks for the juggle that is to immediately. Every three ticks, the pattern of throwing and be attempted. Based on this notion, an open loop juggling catching is repeated, and the pattern is offset by one tick for robot was attempted at the New Jersey Governor’s School each ball, causing one catch and throw to occur at every tick. of Engineering and Technology in 2019 in a project titled With this repetitive, constant pattern, each ball passes through Autonomous Juggling Robot. The researchers constructed a both hands equally and spends the same amount of time in robot that eventually attempted a 3 ball cascade and found that, the air. despite theory, their throws were inconsistent. This was partly 2 due the transistor heating up within their circuit, increasing resistance [7]. Once a single error occurred, the robot could not correct it, and the error eventually compounded and resulted in a missed catch.