A Novel Approach to Home Automation: Application Development with Google Glass Rupesh Chinta Roland Fong Isabel Murdock [email protected] [email protected] [email protected] William Kang Quinn Williamson [email protected] [email protected] ABSTRACT 1 INTRODUCTION Recently developed wearable The rapid advancement of modern devices have opened up new capabilities devices has led to an outgrowth of new in human interactions with the ways for people to interact with the world. Home automation is an area of world. Among these advancements is interest for the application of wearable wearable technology, which combines the devices. Currently, home automation is computing power of a smartphone with performed with home control panels or convenience and accessibility. Wearable mobile devices. This paper, on the other technology consists of computing devices hand, presents an implementation of that can be worn, like a belt or a home automation using the Google backpack, and have a form of input, such Glass. With a few simple voice commands as a touchpad1. The rise of wearable or finger swipes, the user can monitor the technology opens up a new wealth of home and control home appliances, thus possibilities for potential applications, bringing a new level of interaction with including sensing the user’s immediate the home. The Glass communicates with environment2, navigating1, and assisting an online server that relays commands to medical professionals3. One application of a microcontroller to execute certain wearable devices is in home automation. tasks. The microcontroller is, in turn, able A primary goal of home automation is to to communicate sensor information to the optimize the efficiency and comfort of Glass through the same server. This home residents. However, current efforts framework serves as a prototype that to fulfill these desires face limitations demonstrates the potential for wearable ranging from restricted portability, such technology to simplify the user’s in home as that provided by wall panels, to a lack experience. In the future, it can be of a nearly hands free experience when expanded to include more features and using smartphones and tablets4. more devices for the user to interact with in the home. Wearable devices solve these problems with constant accessibility and awareness of their context, or the user’s 1 surrounding area1. Because it is 2.2 Components constantly accessible, the interfaces of most wearable devices are designed to The development of this accommodate constant usage5. Google application relies on several programs, Glass, which focuses on simplicity and tools, and hardware. The Android efficiency, best fulfills these necessities. It Development Toolkit (ADT) allows allows users to easily complete daily tasks development of android mobile through advanced voice and motion applications on the Google Glass. The recognition. In addition, the Google Glass Google App Engine, a “Platform as a is comfortable, because it can be worn Service” (PaaS), allows the android like ordinary glasses, providing an application to access an online server. unobtrusive, yet novel experience. Finally, the Arduino microcontroller, an open source electronics platform, controls This research seeks to design an most of the hardware. innovative approach to home automation through Glass application development. The key features of our application, “Start 2.2.1 Android Development Toolkit home auto”, incorporate the Android Software Development Kit (SDK), Google Glass runs on the latest Arduino, and the Google App Engine. Android’s operating system, 4.4.2, also known as “Kit Kat.” This Android Development Kit (SDK) includes a 2 BACKGROUND debugger, libraries, a handset emulator, sample code, tutorials, and 2.1 Why Google Glass? documentation. The Java programming language is used to write Android Google Glass is a wearable device applications. The Extensible Markup with an optical head mounted display and Language (XML), is also used for design in a touchpad on the right temple of the the ADT. XML is a common programming glasses. The user can interact with Google language designed to carry data. In the Glass both by using voice commands and ADT, XML is used to declare static by tapping/swiping the touchpad. Glass structures like that in the Android displays information in the top-right of Manifest. The Manifest holds the the user’s field of view, in a location that application name, version, and icon (not is easily accessible without obstructing applicable with this application) as well the user’s sight. Additionally, Glass can as permissions from the operating connect to the Internet through Wi-Fi and system, used to request additional Bluetooth, and can use this to access features of a device’s hardware or access online devices and servers. personal information and services. The officially supported Integrated 2 Development Environment (IDE) is input and output to and from the server. Eclipse. Its annotation-based code gives it ease of use from a developer’s perspective. In order to code specifically for the Google Glass, the Glass Development Kit 2.2.3 Arduino (GDK), a revision to the ADT, is used. The GDK handles specific Google Glass The Arduino is a single-board functionality, such as voice commands. microcontroller that senses the Additionally the entire project is complied environment by receiving inputs from with the GDK and targets the Google Glass many sensors. It can also output data and instead of the Android phone operating be programmed, allowing it to control system. devices. While there are many different Arduino models, the Arduino Yún is the 2.2.2 Google App Engine microcontroller of choice because it is optimized for wireless connections, which This PaaS allows developers to are crucial to the application. The Yún build and run applications through also contains a USB-A port, micro-SD card Google. A PaaS is a category of cloud slot, 20 digital input/output pins (of computing that allows for the building of which 7 can be used as PWM outputs and applications and of services over the 12 as analog inputs), a 16 MHz crystal internet. Uses of the App Engine are for oscillator, a micro USB connection, an applications that have web service ICSP header, and 3 reset buttons. Arduino components, applications that keep a Yún is programmed with a C/C++ based profile or sync across devices, and language using the Arduino IDE. applications that need authentication. AppEngine Endpoints API parses parameters as JSON (JavaScript Object 3 DEVELOPING A GOOGLE GLASS Notation), a lightweight data interchange APPLICATION format, to translate data between servers. In the case of the application, Google App 3.1 Overview Engine was used to create servers. The servers allow Arduino to communicate Start Home Auto was created to with the application via the Internet and demonstrate the potential of wearable vice-versa. devices in home automation. The application ultimately allows the user to The Google App Engine is remotely control home utilities. The especially tailored to this application project consists of three components: The because it is designed specifically for Google Glass application, an online server interfacing between servers and android hosted on App Engine, and hardware or Google products. The App Engine API controlled by an Arduino microcontroller. also allows flexibility for controlling the 3 3.2 Glass Application motion, it will prompt the server, and the server provides a timestamp for when the The Google Glass runs on the motion was detected. The Glass Android operating system, and is application polls the server periodically programmed using the Android and when it finds that motion is detected, Development Toolkit (ADT) in the Eclipse it will get the timestamp and notify the development environment. Using the user. voice command “Start Home Auto,” the user can open the application and view 3.4 Arduino Microcontroller and Hardware the home screen, a menu that allows the user to access the toggle menu, program The Arduino microcontroller menu, or live stream video. These options toggles a switch, which is directly can be used to wirelessly control connected to the hardware, namely the hardware and monitor the home through lights and the air conditioner. With the live streaming and motion detection. output received from the Google App Engine, the Arduino turns the devices on 3.3 App Engine Server or off. In addition, the Arduino is connected to an infrared sensor, which Google App Engine Server allows acts as a motion detector. When the the Google Glass application and Arduino values of the infrared sensor fluctuate, the to interact. The communication flows in Arduino notifies the server and then both directions. The Glass application can pauses before again checking the sensor. send a command, such as “lights on,” to the server. The command is then relayed 4 RESULTS AND DISCUSSION to the Arduino, which performs the actual command. When the Arduino detects Figure 1. Project Schematic 4 Table 1: Program Classes Classes Description MainActivity Home Page. Allows access to Toggle, Task, and Streaming Activity. LiveStreamingActivity Live stream feed from camera connected to Arduino. MotionNotificationActivity Not on UI thread. Notifies user about motion detected by device on Arduino. TaskActivity Allows addition of programmable tasks (actions that will occur at a certain time) ManageActivity Allows the user to delete existing tasks ToggleActivity Allows access light switch and AC switch ToggleGetService Starts a service that runs in the background which checks whether motion is detected States Holds the states of each toggle-able activity as well as the motion timestamp ToggleGetTask Application can receive information from Arduino through the server. ToggleJsonParser.java Parses string edited in Toggle Activity so server can understand individual commands TogglePutTask.java Sends commands to server through JSON 4.1 Application Activities and Classes this notification. The remaining classes The finished application has a communicate with the Google App Engine simple user interface that includes few Server.