Complex AI on Small Embedded Systems: Humanoid Robotics Using Mobile Phones Jacky Baltes and John Anderson Department of Computer Science, University of Manitoba, Winnipeg, Manitoba, R3T 2N2 Canada Email: jacky,[email protected] http://aalab.cs.umanitoba.ca Abstract ory, and better battery technology have combined to al- low far more effective embedded systems than were previ- Until recent years, the development of real-world hu- ously possible. Consequently, a generation of systems that manoid robotics applications has been hampered by a are lighter and more robust now affords the possibility of lack of available mobile computational power. Un- like wheeled platforms, which can reasonably easily be smaller, lighter, and more adaptable robots. For the same expected to carry a payload of computers and batter- reasons, these small, powerful embedded systems have also ies, humanoid robots couple a need for complex con- moved out of the industrial sector and into the realm of con- trol over many degrees of freedom with a form where sumer electronics, giving much higher computational ability any significant payload complicates the balancing and in embedded devices for everything from video equipment control problem itself. In the last few years, how- to automobiles. In particular, mobile phones have evolved ever, an significant number of options for embedded from basic telephone and contact management abilities to processing suitable for humanoid robots have appeared handheld computers supporting sophisticated applications. (e.g. miniaturized motherboards such as beagle boards), The latter provide particularly exciting possibilities for AI along with ever-smaller and more powerful battery tech- and robotics: they combine powerful computational abilities nology. Part of the drive for these embedded hardware breakthroughs has been the increasing demand by con- with on-board peripheral devices (cameras, accelerometers, sumers for more sophisticated mobile phone applica- GPS units, bluetooth networking) that are in many cases im- tions, and these modern devices now supply much in the provements over what was available just a few years ago and way of sensor technology that is also potentially of use would have to be separately managed. to roboticists (e.g. accelerometers, cameras, GPS). In Our work involves the development of control, planning, this paper, we explore the use of modern mobile phones learning, and vision in humanoid robots. While small em- as a vehicle for the sophisticated AI necessary for au- bedded processors have previously been used to power small tonomous humanoid robots. humanoid robots (e.g. (Yamasaki et al. 2001), Manus I (Zhang et al. 2003), Tao-Pie-Pie (Baltes and Lam 2004), Introduction Roboerectus (Zhou and Yue 2004), and Hansa Ram (Kim et al. 2004)), these examples range in cost from $1000 to Until the last few years, intelligent mobile robotics has been $20,000 US. Currently, we have moved from using these greatly hampered by the size, power consumption, and com- older types of embedded systems to developing sophisti- putational limitations of available mobile computing plat- cated robotics platforms using mobile phones. Used mod- forms. While small mobile robots such as the Khepera ern mobile phones can be had for $100-$200 US (or indeed, have been used in applied research for many years, the ap- even for free as a result of recycling programs) and provide plications of these were limited because of on-board pro- all the facilities necessary to power complex adaptive hu- cessing ability, and the units were expensive (several thou- manoid robots for a fraction of the cost of several years ago. sand dollars each). More typical equipment likely to be en- countered in the average AI lab would be platforms such as Our interest in humanoid robots is in developing the kinds the Pioneer-II, which are large enough to carry laptops or of broad adaptive behaviour that are necessary to support full-size internal computing systems, but remain similarly service robots of the future (e.g. for nursing or firefighting). expensive and carry significant demands due of their size These behaviours include being able to actively balance on (heavy lead-acid batteries and larger motors). uneven surfaces (e.g. move through grass or gravel), plan Conversely, recent years have brought about a revolu- complex motions, such as crawling, carrying, and climbing, tion in available computational ability in embedded systems as well as combinations of these (e.g. pick up dirty laundry from the standpoint of mobile robotics. Smaller, power- from underneath the bed), and interact with other robots or ful and less power-hungry processors, cheaper flash mem- humans (e.g. move furniture in groups). The broad nature of these tasks is extremely challenging to AI in general, let Copyright c 2010, Association for the Advancement of Artificial alone intelligent systems running on small embedded pro- Intelligence (www.aaai.org). All rights reserved. cessors such as mobile phones. 2 built IrDA interface, based on the Microchip MCP 2150 IrDA transceiver, to the humanoid kit. This allows the mo- bile phone to control the high-level motions of the robot. While the Bioloid kit comes with firmware that can record and play back basic motions, this is not suitable for the com- plex motions we require, and so we replace this firmware with our own (requiring reverse-engineering part of the orig- inal Robotis firmware). The firmware also supports a 3-axis accelerometer from Analog devices, so that phones that do not have internal accelerometers can use an external sensor for active balancing. Figure 1: The modified Robotis Bioloid robot STORM Adapting Mobile Phones for Embedded Control Systems There is is a huge variety of mobile phones available on the We have been competing for the last three years at major market, and dozens more are released each year. The cost robotics competitions (RoboCup, FIRA) using humanoids of these devices is extremely competitive compared to many whose main computational demands are supported using embedded systems (given their speed, memory, and included mobile phones. While RoboCup (RoboCup 2009) involves sensing devices), because they are produced in huge volume. mainly soccer and a few challenges closely related to soc- While economy of scale and the ability to have many nec- cer (e.g. a ball throw-in), the FIRA HuroCup (FIRA 2009) essary sensing devices included in an embedded system is competition is specifically designed to encourage the devel- very attractive to a researcher interested in supporting arti- opment of the types of broad robotic skills in which we are ficial intelligence and robotics on such systems, one is also interested. The same physical robot must be able to partic- well advised to heed the old motto: Caveat Emptor.Even ipate in events ranging from basketball free-throws to ob- from the same manufacturer, individual phones often have stacle course runs, to a climbing wall, taking place over ex- different versions of the same OS, support different exten- tended periods of time. The computing demands to support sions, and may sometimes run totally different OSs. The the artificial intelligence necessary for such a range of ac- model number often confuses more than it helps in trying to tivity (managing everything from computer vision, to active decipher the OS that is run by a device. For example, the balancing and intelligent control, to localization and plan- Nokia 6600 and 6680 are Nokia Series 60 devices, which ning) would tax a full-sized desktop system, let alone a mod- is a very good OS for robotics purposes, whereas the Nokia ern mobile phone. 3300 and 3500 are Nokia Series 30 devices, which are not This paper explores our experiences with using mobile programmable. But the Nokia 6230 is a Series 30 device and phones for supporting sophisticated real time artificial in- the Nokia 3230 is a Series 60 device. telligence in the domain of robotic control. We begin by It is also important to realize that mobile phone manufac- describing our typical research platform. Following this, we turers see these phones as finished consumer products, and describe with issues in adapting phones for these purposes, therefore do not expect them to be “illicitly hacked” (from and discuss variations in OS, IO support, and issues in soft- their perspective) to be used as embedded control systems. ware development. We then illustrate the abilities of mobile At best, some manufacturers encourage the development of phones for AI by describing three elements of our work that third-party applications, but these applications often run in a are representative of the difficulty of supporting AI on such sandbox which strictly limits which hardware is accessible systems: real-time computer vision, localization, and. to the application. In spite of these hurdles, mobile phones can provide an Humanoid Robots: Hardware and Software extremely cheap development platform with high speed pro- For a physical platform, we begin with the Robotis Bioloid cessing, LCD, buttons, wireless, bluetooth, infrared and one humanoid robot kit: these provide 18 degrees of freedom, or two cameras in a very small and lightweight package. use reasonably powerful and robust motors given their cost, This section details our experiences with adapting these de- and are far easier to acquire and assemble than building vices for robotics applications, including working with real skeletal components from scratch. The Robotis kit includes time operating systems, developing software, and ultimately a small AVR ATMega128 embedded controller for manag- developing an IrDA interface for supporting IO. ing the individual servos of the robot. In our work, this is only used for low-level position control of the servo motors. A Tale of Caution Figure 1 shows one of our robots, STORM, using this plat- The most ubiquitous development environment for mobile form, along with a mounted Nokia 5500 mobile phone for devices is Java 2ME from Sun.