ENGINEERING PERSONALITY INTO ROBOTS EMERGING TECHNOLOGY

Robots that have personalities and interact with humans have long been the preserve of sci-fi films, although usually portrayed by actors in costumes or CGI. However, as the field of robotics develops, these robots are becoming real. Technology journalist Richard Gray talks to Matt Denton, electrical software and robotics engineer, and Lee Towersey, amateur robot builder, about the scene-stealing, real-life Star Wars droids.

proving it was real robot; it was the 1927 German silent movie, mechanisms were not as science fiction turned into reality. Metropolis. However, in this, like precise as they are now,” recalls This little ball-shaped robot so many films that came after, Towersey. “They did try using highlights how Hollywood is the robot was merely an actor robots but, towards the end, now at the cutting edge of in a suit. crew members were having robotics development. The film Even in the original Star to pull them around on string industry’s demands are almost Wars movies, hailed for their to get them to move in a unique in the robotics world – innovative special effects when straight line.” few other applications require they were made in the 1970s, This lack of control generally machines to move in lifelike the droids C-3PO and R2-D2 led to robotics being used to ways, react to the humans were brought to life by humans bring onscreen monsters to life. around them and, perhaps most inside suits. Famously, the late Think about the shark bursting Star Wars: The Force Awakens droid BB-8 poses in front of press and fans at the Los Angeles premiere of the film. Originally thought to be computer generated, there was surprise and awe when it was revealed that the robot was real © Lucasfilm Ltd. & TM. All Rights Reserved importantly, have a personality. actor Kenny Baker spent hours from the water in Jaws and the BB-8 is a perfect example inside R2-D2 in the original films, fearsome Tyrannosaurus rex in of how the skills and tenacity wearing the robotic shell almost Jurassic Park. The animatronics With stars such as Harrison Ford of engineers can transform like a rucksack and controlling used to create the dinosaur in and Carrie Fisher walking the red lifeless hunks of electronics the movements from inside. Steven Spielberg’s 1993 movie carpet at the world premiere of and motors into something In The Force Awakens, the were cutting edge at the time, the most recent Star Wars film, magical to capture viewers’ R2-D2 that appears on the but it could essentially do just The Force Awakens, it would have hearts. “Basically, we are just screen is a real robot. Towersey three things: blink, move its taken something very special given drawings by the concept and his colleague Oliver Steeples head and open its mouth. to steal the spotlight. Yet this is artists and some guidance from were amateur robot builders ENGINEERING exactly what happened when the director, then it is up to us who were recruited by the a two-foot-tall robot rolled out to work out how to make it all production team to use their NEW TECHNOLOGY onto the red carpet, in front of work,” explains Lee Towersey, a skills to create a real working As servomotors (which allow the world’s media. member of the creature effects R2-D2 droid. for precise control of angular Named BB-8, this spherical team on the new Star Wars films It highlights the drive or linear position, velocity and droid, dreamed up by the film’s who built many of the droids. within the film industry to take acceleration), and perhaps more PERSONALITY director JJ Abrams, was a huge advantage of the huge changes importantly, the microcontrollers hit with fans. However, most in technological capability needed to coordinate their had believed it to be the work ROBOTS ON SCREEN that have taken place since actions have improved and of clever computer-generated Robots have been appearing the original Star Wars movies grown smaller, the possibilities graphics. The droid’s surprise in films for nearly 100 years. were made. in robotics have opened up. appearance at the premiere in The first known appearance of “The technology wasn’t Suddenly, directors have found INTO ROBOTS Los Angeles stunned many by one on the big screen was in really there then and the control that they can get their robotic

12 INGENIA INGENIA ISSUE 69 DECEMBER 2016 13 ENGINEERING PERSONALITY INTO ROBOTS EMERGING TECHNOLOGY

stars to convey emotion and As BB-8 is a round droid that’s body moves in all directions while its head stays still, creating the robot proved even personality. defeat. A separate radio-control challenging for the engineers. The ball-bearings on the underside of its head attach it magnetically to the top of the “There was only so much unit was used in each version to body, which allows it to move freely when the body rotates © Lucasfilm Ltd. & TM. All Rights Reserved we could do with R2-D2 to give allow the head to twist and tilt him character,” says Towersey. on the body on cue, giving BB-8 “He can drive around, we can its precocious and often comic rotate his head and he can make characterisation. sounds. The off-road versions had “However, BB-8 is much more stabilisers fixed to the back, interactive. I think a lot of kids turning the robot into a three- who grew up with the prequel wheel-drive trike with the films, which had extensive CGI head fixed on a bracket, while in them, expected BB-8 to be another version was essentially computer generated, so when a wheelbarrow pushed by a they saw that it was a real robot puppeteer. They also had a moving about, they were blown lightweight version that could away.” Computer-generated be carried and a ‘bowling ball’ graphics became a major BB-8’s personality is engineered through the ability to tilt and turn its head. Throughout the film, these simple version that could be thrown movements show the viewer the droid’s emotions and give it an almost childlike personality © Lucasfilm Ltd. & TM. component of the Star Wars films All Rights Reserved into shot but remain upright. when director George Lucas But it was only during an decided to make the prequels. new ways of showing emotion The problem Denton turnaround prototypes that had enforced gap in filming, caused The decision to revert and personality. You can move and Lee faced was that their to work. Usually, they only have by actor Harrison Ford breaking back to so-called ‘practical the body around slowly to point BB-8 was going to face some to work for a few weeks but BB-8 his leg on set, that the pair effects’ (what physical props the other way, you can dip unpredictable and difficult had to survive a whole year’s returned to the idea of building are called in the special FX the head a little, cock it to one conditions, yet had to stand up shooting.” a fully working sphere robot. world) left the engineers in the side. There is a huge amount of to the rigours of a frantic filming In the end, the pair created “We started thinking about what During filming, eight different versions of the droid were created to be used in a variety of locations and situations creature workshop with a major emotion that effectively comes schedule. “We had to operate eight different versions of the would happen when we got to © David James (top), John Wilson (bottom) Lucasfilm Ltd. & TM. All Rights Reserved challenge. Director JJ Abrams from three axes of movement.” on sand and rough terrain,” robot: some that could be used the red carpet,” recalls Denton. had sketched out his idea on a There were existing examples explains Denton. “But none of in rough outdoor terrain, one “Everyone would want to see The head ‘floats’ on top of the to take over some of the job of the servomotors in a head into napkin for a ball-shaped robot of spherical robots on the the methods out there would that would stay stationary and BB-8 but they wouldn’t want to body so that it can be tilted delivering emotion in robotic different expressions so you can and asked the team to see what market. One, built by researchers work in all those conditions some that would work inside. see a static version. That is what and turned, giving BB-8 its characters. “We are starting to get some really interesting facial they could come up with. Matt at Uppsala University in Sweden, while also hitting marks over and The simplest of these was drove us to figure out how it distinctive, almost childlike have control systems that sit movements. We have also been Denton, an electrical software features cameras on either side over again. something they nicknamed was going to work. When we personality. It seemed so lifelike on a robotic head and pick up working on automated blinks or and robotics engineer, and his and has been tested by the US “There is a temptation to the ‘wiggler’, which could be first rolled it out on stage in that photographers on the red the movement of the actor,” he breath cycles, so you can have long-time colleague Josh Lee Military as a security droid. It over engineer it, but sometimes bolted to the floor and the 2015, no-one was expecting carpet at the premiere called explains. “So if it is wearing a the robot working away were the people who ultimately works using a pendulum inside you just have to look for the ball could twist and tilt while it. There were people in the its name to get it to look their mask, for example, the control by itself.” built BB-8. the ball that is suspended from simplest solution. You are its head could turn around on audience whose jaws just hit way while actor Warwick Davies system will take control of the Advances in movement “We had to figure out how an axis. A motor moves the wasting a huge amount of a shaft through the middle. the floor.” posed for a selfie with it. Yet eyes or the face so that it looks tracking are also allowing to make it work,” says Denton. pendulum forward, displacing money if the device is broken It was this that allowed the Their final robot uses a BB-8 still requires human help to in the correct direction and engineers to produce robots “There has never been a round the centre of mass and causing down and the film crew is team to get many of the iconic motor-driven pendulum to come alive, with Denton driving blinks when the actor moves. with eyes that automatically robot with a head on it, mainly the ball to roll forward. Moving waiting for you to fix it. It can be close-up shots that show BB-8 propel the beach-ball-sized its movements by radio control. “With new microcontrollers I follow other actors in a scene, because why would you do it from side to side allows it to embarrassing if it doesn’t work. peeking around the door of the sphere from the inside to a However, Denton believes can only see it improving. They making them look like they are that? But having a head gave us steer left and right. In that sense, we were doing fast Millennium Falcon or sagging in maximum speed of 4.3 mph. that new technology is starting allow you to mix the action of all part of the action.

14 INGENIA INGENIA ISSUE 69 DECEMBER 2016 15 ENGINEERING PERSONALITY INTO ROBOTS EMERGING TECHNOLOGY

HOW BB-8 WAS BROUGHT TO LIFE

SOFTWARE There is no universal programming language for robots so manufacturers of robotics hardware tend to develop their own unique software to achieve the results they want. In films, robots and animatronic characters are usually controlled using joysticks, which often means that their movement can look unnatural as the joysticks have little weight and so can be moved quickly around while the robot has to act against gravity and other forces in the real world. Matt Denton has spent years studying how real organisms move in nature to find ways of replicating this within the control software. By including feedback loops and data filtering into the The engineering behind BB-8 was revealed at a Star Wars celebration event* in London in July 2016. This still from a video shown at the event software, he has found that it is possible to remove the twitchiness shows the pendulum mechanism inside the robot’s body Denton, who runs his own the Personal Robots Group that can make a robot move in a less realistic way. Other tricks animatronics company, at Massachusetts Institute Matt Denton and Josh Lee toyed with several approaches to include programming in a feature that means wherever a Micromagic Systems, developed of Technology has been cracking the problem presented to them by director JJ Abrams for character is looking is forward. In a robot such as BB-8, this can Matt Denton’s Hexapod robot responds to people in front of it by using facial a six-legged robot that works developing what she describes The Force Awakens: a spherical robot with a head that ‘floats’ on make controlling it much easier. tracking and gesture recognition software. It’s ‘eye’ (or camera) can follow without a puppeteer altogether. as sociable robots. It began with movement and it swings back on its legs – or shies away – if people get too top of the main body. close © Matt Denton An earlier version of his Hexapod an anthropomorphic robotic it will try to make them happy in one film, Sayonara, about They opted for a pendulum that sits inside the ball-shaped robot appeared as one of the head called Kismet that could with unrelenting cheerfulness. the aftermath of a nuclear magnetism, and much of the problem of getting BB-8’s head shell, which is kept upright by gyroscopes. The spherical body weird magical creatures in the express itself by adjusting its Toyota has also unveiled a power plant meltdown. While technology needed to build to stay on top of its spherical itself is driven by a rotating hub that spins the ball while the Harry Potter movies when it was gaze or facial movement but diminutive robot called Kirobo the performance is a little it, such as the motors and body while still letting it move movement of the pendulum insides creates momentum to alter dressed up like a six-legged turtle. has since progressed into a mini, which has been designed mechanical, it shows a sign of gyroscope sensors, are available around freely. Although Denton the direction. The robot has an axle running horizontally through His updated model uses facial fluffy 2.5-foot-tall robot called to be a ‘communication partner’ what may come. off the shelf. does say that they had obvious the middle of the ball, which is driven by motors to rotate hubs tracking and gesture recognition Leonardo, which features a and has since been touted “If you give robots a script One area that they are quiet help to crack that problem – on either side of the ball. A shaft is suspended vertically from the technology so that it responds visual tracking system so that by others as a companion for and get them to hit their mark, on the details about is the the Force. axle and this has the motor, the battery and actuators suspended to what someone standing in it can interact with children. childless couples. With large eyes they should be able to do it on the bottom, and acts like the mass at the end of a pendulum front of it is doing**. Looking More recently, her team has that blink, a high-pitched voice better than humans as they inside the ball. When the motor turns the axel, this swings the BIOGRAPHIES a little like a giant spider, it developed a humanoid robot and a wobble as if it hasn’t quite have ultimate memories,” says centre of mass forward and causes the ball to rotate in that Matt Denton left school in 1989 and completed a four-year shifts its weight like a predator called Nexi, which has arms that cracked how to balance, it has Denton. “The problem is that direction. Gyroscopes also help to keep the pendulum in position apprenticeship in electronic engineering with Marconi Defence preparing to pounce as it tracks move along with the face to a vulnerability that its designers they would never give you when the ball is moving. Systems. In 1993, he started a degree in computer-based the movement of the person make its body language realistic; say makes it cute. The robot can nuances in each scene. With Although the mechanisms of how the head was attached electronic engineering at the University of Portsmouth. in front of it. “It is much closer with a flick of its eyebrows and hold a basic conversation, make artificial intelligence that might to the robot have not been officially revealed, images that have During his first year, he had a summer job with a small special to what I consider to be real clenching of its fists, Nexi can hand gestures and respond to be possible but it is a long been released indicate that a disk embedded with magnets was FX company at Ealing Studios and didn’t return to complete robotics,” says Denton. skip from joy into anger and human emotions. way off.” attached to the top of the pendulum. With ball-bearings on the his degree. Matt formed Micromagic Systems in 1998 and then sadness. There are even some who are underside of the head, the head could be attached magnetically Then there is Pepper, a hoping robots will become actors has been specialising in control systems for creature effects to the top of the body while allowing it to move freely when ROBOTS IN THE Japanese creation built by in their own right. Researchers HOW TO ENGINEER ever since. the body rotates under it. Tethering the head to the top of the Softbank Robotics, that uses at Osaka University in Japan Lee Towersey built his own R2-D2 in 2009, which has been pendulum in this way also means it shifts forward in the direction REAL WORLD A ROBOT Outside the movie industry, artificial intelligence to give the have created an android they Denton and Lee are now used in a Currys TV advertising campaign along with many of travel when BB-8 is moving and turning. Additional motors at robots have been largely $1,600 humanoid ‘emotions’ call Geminoid F that has been considering publishing the other promotional events on behalf of Lucasfilm. He joined the bottom of the pendulum rotate this shaft so the head can consigned to performing so that it can be used in designed to look and behave just schematics in the future to allow the team of Star Wars: The Force Awakens in 2013 to build and turn, tilt and dip when it is moving. repetitive, mechanical tasks, shops or for customer service. like a human. Actuators powered schools and young engineers operate droids for the film, and has also worked on droids for Even the pattern on BB-8’s body was designed to help viewers but as they start to move into Voice recognition allows it to by air pressure beneath its rubber to build their own BB-8. The Rogue One, Star Wars Episode VIII and the as yet untitled Han connect with the little droid: differently shaped panels were used people’s homes and everyday recognise when it is being skin allow the robot to copy robot demonstrates some good Solo film. on each side to make it easier to follow its movement. life, researchers are taking some chastised and it responds by human facial expressions and engineering principles such *To watch Matt Denton and Josh Lee discuss how they built BB-8 lessons from Hollywood. adopting a crestfallen pose mouth voice recordings. as gyroscopic forces, electrical **To see Matt Denton interact with the Hexapod, go to at the event, please visit http://bit.ly/2ejM6bq Dr Cynthia Breazeal from or, if someone is feeling down, It has already appeared engineering, software and www.youtube.com/watch?v=4oXuSXCKJeY

16 INGENIA INGENIA ISSUE 69 DECEMBER 2016 17