Proceedings Template - WORD s34

Personalizing Interaction in a Mobile Environment Sebastiano Pizzutilo, Berardina De Carolis, Antonio Petrone and Giovanni Cozzolongo Dipartimento di Informatica - University of Bari Italy

ABSTRACT: With availability of new technologies for interacting with e-services everywhere through wireless devices or active objects, accessing to e- services need to be provided by combining distributed systems technology, user adapting methods and ubiquitous computing techniques. In this optics, context-awareness is increasing of importance in achieving effective interaction by adapting service request and presentation of results not only to the user needs and preferences but also to the situation in which the interaction occurs. In this paper, we propose a multiagent architecture in which users and e-services are represented by agents that negotiate tasks execution and generate results according to “user in context” features.

Keywords Ubiquitous interaction, personalization, autonomous agents, user adapted systems, context adapted systems.

1. INTRODUCTION situational context [1, 5]. The paper is organized as follows: Section 2 outlines the D-Me New technologies (mobile phones, wearable computers, Architecture, in particular describing its personalization “augmented everyday objects”) are available for interacting with component. Section 3 illustrates how it has been applied for services of various kind everywhere and simultaneously with modeling ubiquitous interaction with e-services provided by the other activities. In this scenario, personalization becomes of Department of Informatics in Bari (DIB). In Section 4 increasing importance in achieving effective interaction with conclusions and future work are discussed. these services [1, 14]. However, the mobility of the user and therefore, the possibility to interact with services ubiquitously, 2. D-ME ARCHITECTURE requires taking into account new personalization factors: context-awareness becomes a key feature for ensuring an The idea of delegating complex or tedious tasks to an agent is appropriate response of the application to the user requests. the core of Intelligent Interface Agents research. In ubiquitous According to some definition of context relevant to ubiquitous computing, a user may benefit of a context-aware agent for the computing [2, 4], user’s location, current activity, emotional proactive execution of a task that is scheduled, for instance, in a state, interaction device/s, time of the day, weather conditions To-Do-List [4]. In this case, the user lists the task to be seems to be relevant to personalization. For instance, performed in different contexts and environments and gives to information about the location, time and weather can be used to her/his personal agent the autonomy to perform them entirely or contextualize service request and presentation of results through in part. When the user is in a particular situation (environment, context-sensitive generation of natural language texts [3], time, location, emotional state, etc.) that triggers one of these presentation of graphical maps [7], highlighting of objects tasks, the personal agent requests personalized execution of this available in the surroundings, and other. Users activity, task on the user behalf, by transferring to the environment the it emotional state, as well as the input/output capabilities of the needs. When the task has been performed, the agent device/s they are employing, may influence the way the communicates results to the user: these may be of various information is accessed and presented (for instance, browsing in nature, according to the performed service, and can be adapted a large information space, searching for some specific data or to “user in context” features. For instance, a "remind message" receiving fast and well focused ‘hints’). is generated when some user task is executable in that context; In this paper, we present our approach to ubiquitous interaction "user/context adapted information presentation" is generated with e-services. Starting from previous research works about after an information retrieval task; "assistance on task execution" Intelligent Interface Agents [16, 18], we developed a is generated when help is needed in performing actions in the MultiAgent System in which a mobile student interacts with real or digital world, and so on. ubiquitous campus e-services through an agent (a digital “alter The architectural model of D-Me [20], shown in Figure 1, is ego”) that represents the user in the environment. The agent, that based on this vision. It includes two interacting entities: a D-Me we call D-Me (Digital-Me) [20, 21], perceives the presence of a Agent, representing the user, and the Environment, a physical or smart environment and controls adaptation of tasks to the user logical place in which various services are available. needs and the context. The D-Me Agent: The proposed approach is focused on the concept of task; - knows the tasks in the To-Do-List and how to perform therefore it has been implemented a context-aware To-Do-List them in a given context; application that reminds tasks to the user by considering the - manages the Mobile User Profile (MUP) with its privacy These users can be represented by D-Me agents. requirements; In order to analyze which factors were relevant to our - requests execution of services on the user behalf, and application, a questionnaire was given to 50 subjects belonging - presents messages appropriately. to the following categories: first (23), second (13), last year (7), Ph.D (3) students and professors (5). The number of interviewed people was proportional to the percentage of people of that category, present in the Department. From this questionnaire we identified: - a set of scopes, grouping a set of specialized services; - a set of user features relevant for adapting service fruition; - a set of dynamic context-related features that influences task execution and presentation of results; - a set of possible user tasks executable in the DIB. Let us assume, as an example, that the user is a second-year female student, travelling each day to go to the Campus. Let’s suppose that her To-Do-List includes the two following tasks: "Give back to the library the book on Operating Systems" (with Fig.1 The D-ME overview low priority) and "Find documentation for the Web Programming exam" (with high priority). The environment is ‘active’ [17]: it is populated by several D- In this scenario, My-DIB will behave in the following way: Me Agents and by Service Agents which execute various tasks. When the user comes close to the library, the student’s D- While the number of Service Agents depends on how many Me generates a reminder for giving back the book; it also tasks the environment supports, there is only one Keeper Agent proactively asks the library Service Agent whether Web that acts as a facilitator [10] and knows which (D-Me and Programming books are available and displays Service) agents exist in the environment. It therefore provides to information received according to students rating. If it D-Me environment-related information and the list of agents detects other student’s D-Mes with the same task, it will which could accomplish the required service. contact them asking for more information. According to Users may interact with services in a remote way or by being the user preferences will, eventually, put her in contact physically in the environment and may then move from one with the others. environment to another. Managing inter-environment Let’s see how the previously described agents deal with this communication is the taks of the EnvKeeper. situation. As far as personalization is concerned, in the D-Me architecture we adopt distributed solution to it [15]: on a side, D-Me knows 3.1 Environment Description its user and how to perform delegated tasks; on the other side, An environment is made ‘active’ by giving agents which the environment can provide adapted service execution. Both populate it the capability to understand its features: that is, by entities need to sense and elaborate context information. We modeling it. In MY-DIB we identified user needs for services implemented personalization by means of specialized agents: belonging to the following scopes: learning, social relations, - the UMAgent and the ContextAgent which manage, leisure, food-services and administration. Every scope identifies respectively, user and context modeling; a set of specialized services. For instance, the ‘learning’ family - the D-Me Agent, which manages user and task related includes the following services: register for an exam, get data, and information on how to prepare it, collect learning material and - the Interface Agent, whose role is to communicate user- so on. As we said, scopes are used to identify the user and adapted results. context features that are relevant for performing a task. The D-Me architecture has been developed using the JADE The physical features of the environment have also to be toolkit [12] which is FIPA [9] compliant. Communication described. For instance, the library has ‘learning’ as a scope and among agents has been approached developing specific is a ‘public’ place. In the library there is a PC, identified by an protocols and using FIPA Agent Communication Language ID and an address, that can be used to communicate results. (ACL) compliant messages. The only constraint is that every Therefore, all the tasks in the To-Do-List that are enabled in the entity that supports this communication must share with the library and have learning as a scope will be proactively activated other parties a portion of an ontology [13], whose concepts are when the user is in it or passes nearby. the subject of the conversation. 3. MY-DIB AS A SMART ENVIRONMENT

As mentioned in the Introduction, we developed, as a prototype of the D-Me architecture, MY-DIB that aims at modeling the Department of Informatics Bari (DIB). The system users are students with different level of experience, professors, staff. 3.2 User Modeling The XML structure reflects the user profile ontology used in User modeling (UM) is crucial for the personalization of IDENT IT Y to let the user free to interact with services everywhere and continuously in time [20] causes obvious changes in the way the has to be thought not as a part of a standalone system, but as an M IND independent component able to provide its services to other entities that require them. In this optics, the following problems continuously in time, moving physically in different (user profiles). b) Security; the user model is not always used --- BO DY by the same system, but it “moves” with its “owner” among different environments; this requires the need of establishing privacy and security policies. c) Consistency; the user may interact with more than one environment at a time; then, it is necessary to develop a strategy for keeping individual user PERSONALITY information consistent. Possible solutions to these problems are represented by a …. centralized, distributed or mobile approach [15]. All these the user to access her/his model after having been recognized by the system. In the distributed solution, user information are ..... stored in different servers, reducing in this way the computational load but presenting problems of Figure 2. Example of a second-year student MUP. redundancy/incompleteness of user information and consistency. My-DIB and includes four main sections: IDENTITY (with The mobile approach (the user “brings” always with her/himself identification data such as the user name, id, password, and the user model, for instance on an handheld device, and, when email), MIND (background knowledge, interests and know- the interaction with an environment starts, the user profile is how), BODY (disabilities or preferences in using a body part passed to the environment user modeling component) seems to during interaction) and PERSONALITY (personality traits and be very promising since it presents several advantages such as: habits). Every slot in the MUP can be protected by giving a the continuous availability of user information, wireless data ‘scope’ validity to the corresponding XML tag and can be made communication, absence of information redundancy and easy ‘not public’ by setting the ‘public’ attribute to ‘false’. For management of consistency. However, we cannot assume that instance, in Figure 1 the user interest towards web programming the user will have an handheld device (i.e. the interaction could can be shared with other agents only in the ‘DIB’, while her potentially happen using a key car) and this type of device still interest in pop music is always public. presents hardware-related limits (capacity, computational speed, When users access some environment service through their D- battery,… ). Me Agents, data that are relevant to adapt service execution are In our architecture, the user is represented by an agent that has to passed to their UMAgents, which start the modeling process. In dialogue, communicate and negotiate with other agents in the the considered example, when the student is in the DIB user environment. In this view, the user profile should preferably be identification data, interest in web programming and pop music stored on the client side and should communicate the needed will be considered, together with the preferred interaction personal information to the environment. Once the users access modalities (visual interaction). When interaction ends, the through their D-Me Agents some information service, data that environment sends back to D-Me a portion of the model, are relevant to personalize the interaction for that particular updated according to what has been inferred: D-Me stores this service are passed to the environment UMAgent which starts the data in the MUP as ‘inferred’. We did not consider consistency modeling process. issues so far; however, since we provided the D-Me model with In order to accomplish this task, D-Me manages a Mobile User an infrastructure able to support federation of environments we Profile (MUP), containing information about its user, that moves are considering how they could exchange information about the with the user. In the present prototype, data in the MUP are user. As for the underlying architecture, agents that interact to collected in two ways: the user can input information through its accomplish the user modeling task communicates using ACL. interface, while other information (i.e. temporary interests) can This enables us to overcome problems due to agents using be derived from tasks scheduled in the To-Do-List. Figure 2 different representations for user profiles [13]. shows an example of MUP. 3.3 Context Modeling In My-DIB, context is grounded on the concept of ‘tasks executable in an environment’. Given a task in the To-Do-List, suggested material to prepare the exam. Then, the D-Me agent its execution and results may be influenced by the context in will ask the user for confirmation before sending the email . which interaction occurs. In particular: i) environment-related - notification messages. Proactive task execution is features (scope, noise and light level); ii) dynamic user-related notified by D-Me, for instance, in the previous case, if the agent features that identify the physical and social surroundings of the has the autonomy to send emails it will not ask for permission user (emotional state, location, activity the user is performing, and will just notify it. time and weather conditions; iii) device employed and its state at the considered time (battery, connection, and so on.). These - remind messages. This is the typical message factors are sensed and controlled by dedicated Sensors Agents, generated for the first task in our example. which communicate relevant changes to the Context Agent. In User and context related factors are taken into account in the scenario example: the Sensor Agent controlling the user generating the communication about a task in the following location detects the user presence in the DIB and in particular way: her relative position to key places such as the library. The 1. user preferences and features: results of information Sensor Agent controlling the device detects that the user has got provision tasks are filtered, ordered and presented a PDA. according to what has been inferred about the student The context situation relevant at time ti is represented in an starting from user profile data (interest, know-about, know- XML structure compliant to the context ontology. how). Possible user disabilities are taken into account for media selection. 3.4 Task Modeling 2. activity: this influences information presentation as follows. D-Me may execute the tasks in the To-Do-List that are enabled If the user is doing something with a higher priority respect in the given context. To model the ‘task-user-context’ relation, to the one of the communication task, then the message is we employed an extension of Petri Nets which was developed postponed until the current activity ends. If the by our research group in a previous project [6]. For instance, in communication regards the current activity, media used in the previously described situation the first task in the To-Do-List the message take into account the available body parts. corresponds to the D-Me goal: Remind(U, Do(Task, env, Cti)), Therefore, a voice input is preferable to a textual when, for where U denotes relevant user features, Task denotes the task in instance, the user is running with her/his PDA asking for the To-Do-List, env denotes the environment and Cti the context information about the next train to catch. at time ti. In this case, when the user comes close to the library, D-Me generates a reminder for giving back the book that is 3. location of the user in the environment: texts, images and presented appropriately by the Interface Agent. other media may be selected according to the type of environment (public vs. private, noisy vs. silent, dark vs. The second task in the To-Do-List activates another D-Me goal: lightened, etc.) in which the users are and, more precisely, Search(D-Me, News(U, Task, env, Cti)): D-Me proactively asks to their relative position to relevant objects in the the library Service Agent whether Web Programming books are environment. available and displays information according to student ratings. 4. emotional state: factors concerning the emotional state influence the level of detail in information presentation 3.5 D-Me Interface Agent (short messages are preferred in stressing situation), the This agent has the role of interacting with the intrusiveness (bips and low priority messages are avoided user for communicating results of tasks or for when the user is already nervous), and the message content. asking information/confirmations required for For instance: if a user requests information in conditions of task execution. Figure 3 shows the D-Me emergency, the agent will have to avoid engendering panic, communicating to the user the result of the by using reassuring expressions or voice timbre [3]. Search task. 5. device: the display capacity affects the way information is selected, structured and rendered. For instance, natural In My-DIB application, we consider the language texts may be more or less verbose, complex following families of communication tasks: Figure 3. figures may be avoided or substituted with ad hoc parts or with written/spoken comments [7]. - request for input. If, for instance, the to-do-list includes the task: “Sign for an exam next week” and To accomplish the communication task, we implemented two the user is at work, D-Me will ask information about “which behaviours of our Interface Agent that map typical steps of a exam ” and “at what time” the user wants to do it. Natural Language Generation system [19].. The first behaviour implements Content Generation that decides, starting from - information provision: Information may be presented when explicitly requested by the user or proactively prompted ServiceAgents results, what to communicate. The second one implements the Surface Generation that displays the content by D-Me because related to the current user task. As we anticipated, in the case of the second task in our example, D-Me according to the factors influencing personalization. The Interface agent applies the following strategy: XML-annotated will display information about Web programming books present in the library according to student rating. filtered results are passed to the Content Generator that, using an approach based on ontology sharing like the one explained - request for confirmation. Let’s take as an example the previously, understands the meaning of the data to be presented previous task, let’s suppose that one of its steps involves and uses it as a factual knowledge base for the selected contacting by email the course professor for indication about presentation strategy. The result is a communication plan that is passed to the Surface Generation component, which decides how to render it at the surface level taking into account the rules [8] Greenberg S. (2001). Context as a dynamic construct. described above. In our system, these transformation rules are Human-Computer Interaction. Vol. 16. encoded in XSL. [9] http://www.fipa.org. 4. CONCLUSION [10] http://www.ai.sri.com/~cheyer/papers/aai/oaa.html [11] http://www.protege.stanford.edu Ubiquitous computing gives new challenges to adaptive system [12] http://sharon.cselt.it/projects/jade/ research. Indeed, adaptation to user and context features is seen as a promising solution for achieving effective interaction with [13] Heckmann, D. Ubiquitous User Modeling for Situated information services accessible everywhere and continuously in Interaction. 8th International Conference on User Modeling time. As suggested by Dey [4], delegating to an agent tasks 2001, Proceedings. LNCS 2109 Springer. pp. 280-282 execution and presentation of personalised results may be a [14] Jameson A. Modeling Both the Context and the User. solution for dealing with the complexity of this problem. Personal and Ubiquitous Computing. Vol 5. Nr 1. Pp 29-33. In this work we presented a multiagent system supporting 2001. personalised interaction between mobile users and services of a [15] Kobsa A., Generic User Modeling Systems. UMUAI vol. II smart campus (My-DIB). Every user is represented by a D-Me Agent that, according to the content of a “To Do List”, performs nos.1-2 pp.49-63. Kluwer Academic Publisher. 2001. tasks on the user behalf by negotiating services with My-DIB. [16] Maes, P. "Agents that Reduce Work and Information Interaction between these two entities is personalised according Overload," Communications of the ACM, Vol. 37#7, ACM to policies that are implemented in a distributed way. Press, 1994. Obviously, some work has to be done to adapt the general [17] McCarthy, J. F. Active Environments: Sensing and architecture to My-DIB prototype. In particular, we have to Responding to Groups of People. Journal of Personal and complete the porting of web-accessible DIB services to the D- Ubiquitous Computing, Vol. 5, No. 1, 2001 Me architecture. [18] Lieberman H. and Selker T. Out of Context: Computer Another aspect of the personalization of interaction that we are Systems That Adapt To, and Learn From, Context. IBM considering is the possibility to model group of users of My-DIB Systems Journal, Vol 39, Nos 3&4, pp. 617-631, 2000. in order to adapt presentation of information both to the group and to personal needs. [19] E.Reiter and R. Dale. Building Natural Language Generation Systems. Cambridge University Press. 2000. [20] Weiser M. The Computer for the 21st Century. Scientific REFERENCES American, september 1991 [21] B.De Carolis, S.Pizzutilo and I.Palmisano D-ME: Byun H.E. & Cheverst K. User Models and Context- Personal Interaction in Smart Environments. In User [1] th Awareness to Support Personal Daily Activities. Workshop Modeling 2003 ( 9 Int.Conf. of UM03, Johnstown, on User Modelling for Context-Aware Applications. 2002. PA, USA, 22-26 June 2003) eds. P.Brusilowsky et al., Springer . [2] Chen G. and Kotz. D. A Survey of Context-Aware Mobile Computing Research. Technical Report TR [22] B.De Carolis and S.Pizzutilo Ubiquitous Interaction: http://citeseer.nj.nec.com/chen00survey.html. Adapting to the User in Context on “ Advances in [3] De Carolis B., de Rosis F. and Pizzutilo S. Adapting Communications and Software Technologies” ed. Information Presentation to the "User in Context". IJCAI N.E. Mastorakis, V.V. Kluev, WSEAS Press, 2002, 2001 Workshop on AI in Mobile Systems, Seattle, 2001. pp.219-224 (ISBN: 960-8052-71-8) [4] Dey A. K.Understanding and Using Context. . Personal and Ubiquitous Computing 5 (2001) 1, 4-7. [5] Dey A.K. and G.D. Abowd: CyberMinder: A Context -Aware System for Supporting Reminders. Proc. Symposium on Handheld and Ubiquitous Computing, Bristol. (2000). [6] de Rosis F., Pizzutilo S., and De Carolis B.. Formal description and evaluation of user adapted interfaces.International Journal of Human-Computer Studies, 49,1998. [7] G. De Salve, B. De Carolis, F. de Rosis, C. Andreoli, M.L. De Cicco. Image Descriptions from annotated knowledge sources. IMPACTS in NLG, Dagsthul, July 25-28, 2000.