5th International Workshop on Smart Environments and Ambient Intelligence 2013, San Diego (22 March 2013) PERSEO: a system to Personalize the Environment Response through Smart phonEs and Objects Luca Bergesio, Íñigo Marquínez, Ana M. Bernardos, Juan A. Besada, José R. Casar Telecommunications School Universidad Politécnica de Madrid Madrid, Spain {luca.bergesio, imarquinez, abernardos, besada, jramon}@grpss.ssr.upm.es AbstractWhen designing services to control the behavior of a describes a mobile-instrumented interaction concept, which smart environment, it is feasible to rely on mobile devices as facilitates the coordination between the mobile devices con- mediators to empower the user and handle his needs and pre- tent and actions, and the smart objects in the environment, ferences. In this direction, this paper describes a concept for facilitating the use of enhanced interfaces to visualize and mobile-object instrumented interaction through a system that manipulate media data, services and applications. enables the orchestration of a set of smart objects with sensing Section II frames the current research in the state-of-the- and/or media capabilities deployed in a room. These objects art of interaction concepts with mobile devices as enablers. are coordinated to respond accordingly to the physical and Section III details our proposal, describing a validation sce- logical state of a mobile device. A specific case of use is de- nario for deployment. Section IV gives details about the me- scribed; it is centered on a 'smart night table' that coordinates dia management infrastructure. Section V defines the archi- the response of the smart environment, populated with inter- faces such as a TV, tablet, sound system and photo frameworK. tecture components and the workflow to complete the devel- When a user places his smartphone on the smart table, a ser- opment. Finally, Section VI shows our conclusions and fur- vice checKs the active applications in the mobile device and ther work. maKes an external controller to forward actions to the men- tioned interfaces, managing their availability. The prototype II. RELATED WORK IN MOBILE-INSTRUMENTED shows the viability of these mobile-object instrumented perso- INTERACTION nalization and interaction concepts. Mobile personal devices [2] may have a very important role to facilitate interaction in smart spaces. Touching and Keywords-interaction; smart objects; mobile devices; pointing paradigms for physical interaction, together with reasoning scanning and user-centered object interaction [3] enables simplifying the way we interact with the smart space. For I. INTRODUCTION example, Want et al. [4] proposed in 1999 a method to link Nowadays, the most common user devices to ubiquitous- objects to digital resources through RFID tags. Biegl [5] im- ly interact with digital information or with items connected plemented the point&click concept, by using a stand-alone to the Internet are smartphones and tablets. Thus, even if remote control that was capable of obtaining the control wearable devices, such as glasses or textiles, are to be the information for other devices, allowing operational interac- user interfaces of the future, mobile devices with high capa- tion with the help of a simple user interface. bilities in terms of memory, computation and autonomy are Among the previous works that have proposed technical todays option to easily handle digital information or to deal solutions to interact with smart objects through personal de- with items connected to the Internet. vices, [5] uses a laser-equipped device, which provides the The concept of smart space is becoming popular to de- user with information about the objects control commands scribe an environment equipped with smart objects, which by using a small display. When the object detects the laser may be orchestrated to satisfy its inhabitant needs. The con- beam, it transfers the control description to the device by cept of smart object, which may have different levels of using infrared. intelligence, may be defined as a conglomerate of sensors, A framework for mobile devices to interact with NFC actuators, and communication capabilities. In this paper, we tagged physical objects that are associated with web services will use it to refer to a computationally augmented tangible and provide information for their invocation is described in object with an established purpose that is aware of its opera- [6]. In [2], smart objects use formatted SMS containing sim- tional situations and capable of providing supplementary ple commands to enable their remote operation through a services without compromising its original appearance and mobile device. In [7], it is described a system to easily confi- interaction metaphor [1]. gure from the mobile phone a chain of event-condition- How to personalize the behavior of the smart space by action modules that can be executed by a smart object. making easy the configuration of smart objects through mo- bile device is the leitmotif of this contribution. The paper 978-1-4244-9529-0/13/$31.00 ©2013 IEEE 640 III. INTERACTION CONCEPT • Device-hosted content sharing: the method enables faci- litating environment personalization through the use of A. Interaction method media content available in the device. This approach Let us imagine the following situation: a user is inside a makes sense as smartphones are full of digital content room where are available many smart objects with media that the user may be willing to enjoy through different capabilities, such as a television, a tablet and a hi-fi system. smart objects for visualization, mainly to facilitate con- When the user puts his smartphone next to/on/beyond a sumption due to bigger screens or best quality of sound. trigger smart object (not changing the interaction metaphor • Rule-based reasoning: the interaction method is based of this device), the service implementing the proposed inte- on a reasoning paradigm based on handling event detec- raction method is initiated. The contact/proximity event trig- tion, condition checking and action triggering combina- gers the automated checking of the applications running fo- tions through rules. In our system, conditions are de- reground in the mobile device, and initiates the consequent rived of the applications running in foreground, while action, taking into account: a) the configuration that the user actions are related to digital content and application has accepted by default or previously configured (e.g. by handling (e.g. transferring content from the mobile de- using the concept in [7]) for the service and b) his context vice to a different object for visualization). Action de- (e.g. if there are more than one person in the room). For ex- pend on the foreground applications in the smartphone: ample, if the user is watching photos, the content is trans- for example if the user is watching a video, the action ferred to a more convenient interface, in case he is alone or will be to transfer the video to the television; if the user wants to share with the rest of people in the room. is listening to a song, the action will be to play that song In brief, the interaction method that is proposed in this using the hi-fi system. This simple way of handling the paper is based on a usable mobile-instrumented way of inter- logic of the system makes possible to enable easy-to-use facing and sharing in-device content or application-related mechanisms to personalize and configure behaviors actions with smart objects. Section V describes the entities from the user point of view. involved in the method, but at this stage of the paper, we can • Context-responsive: by using the event-condition-action summarize the following basic ones: 1) the smartphone; 2) paradigm, the method enables the inclusion of context the trigger smart object; 3) the responsive smart objects, dep- analysis, in order to adapt the response mobile device- loyed in the smart space; 4) the smart space itself, which is smart object. aware of the status of the smart objects when needed. • Flexible architecture: as it is explained in Section V, the The driving features for the interaction method are: method enables a flexible implementation of its compo- • Mobile-instrumented concept: the smartphone serves as nents, which may be deployed in smart objects, mobile trigger and orchestrator for the interaction between itself devices and infrastructure. and the smart objects in the environment. B. Case of Use • Coordinated response mobile device-smart object: smart objects will respond to the logical state of the mobile To demonstrate the viability of our interaction method, a device, performing different actions depending on the case of use is proposed. We consider a user with a smart- active applications in it or on its configuration (e.g. si- phone inside a room equipped with a television, a photo lence mode). E.g. smart objects will perform differently frame, a tablet, a smart night table and a hi-fi system (Fig- if the weather application is active than if there is an ure 1). incoming call. • Physical-based initiation/control without changing the traditional interaction metaphor: the coordination and control tasks will start when the user interacts with a smart object, trying to preserve the normal interaction metaphor of this object. Additionally, different control actions related to the traditional interaction metaphor of the trigger smart object may be included in the method. Preserving the conceptual models that a user has to inte- ract with an object facilitates learnability. • Interface with heterogeneous smart objects: the concept makes possible to deal with different kind of smart ob- jects. In particular, Section III.a explains a case in which television sets, photo frameworks, hi-fi systems, tablets and computers handle media content from the mobile device, but the proposal is not limited to this and could be extended to other kind of smart objects. For example, if smart metering application to monitor energy con- sumption at home is active, the air conditioning-heating Figure 1.